We know that differential attainment exists in surgical examinations. We also believe that it is not a result of learner deficit. Instead, we think that examination outcomes in surgery shine a light on inequalities that learners experience throughout the training environment.
Starting with the MRCS examination, we want to understand the exact reasons behind these inequalities and the resulting differential attainment. We aim to have closed the attainment gap within five years.
What is differential attainment?
Differential attainment is the systematic difference in examination outcome between different groups of students depending on their protected characteristics and socioeconomic background. We know that differential attainment exists in a range of medical examination settings, but that it is particularly pronounced in surgery.
What is the scale of differential attainment in surgical exams?
Our understanding of the scale of differential attainment in the MRCS comes from the significant body of research carried out by the Intercollegiate Committee for Basic Surgical Examination (ICBSE) on behalf of the four surgical royal colleges.
Their work has provided us with a clear quantitative analysis of the effect of both protected characteristics and socioeconomic factors on candidate performance in the MRCS exam. From their work we know that:
- Men are significantly more likely to pass the MRCS Part A at the first attempt than women.
- White candidates are more likely to pass the MRCS Part A and Part B at the first attempt than ethnic minority candidates.
- Candidates under the age of 29 are more likely to pass the MRCS Part A and Part B at the first attempt than those aged over the age of 29.
- Those from less privileged socioeconomic backgrounds perform less well in the MRCS than candidates from more privileged backgrounds.
The ICBSE research also made it clear the next steps were to better understand the exact causes of differential attainment, starting with the hypothesis that it was likely to be a result of an ‘accumulation of educational disadvantage over time.’
What are the causes of differential attainment?
To better understand the causes of differential attainment, we have carried out two pieces of work.
The first looked at literature on differential attainment. This pointed us to six key factors that we believe can contribute to differential attainment:
- The training environment and trainer / trainee relationships
- A trainee / candidate’s support network
- How a candidate approaches examination preparation
- A candidate’s experience on the exam day itself
- How a candidate responds to failure and feedback
- A candidate’s wellbeing at the time of the exam
Our second piece of work surveyed recent MRCS candidates. Our aim was to test whether the six factors identified in the literature resonated with candidates. Where issues existed, we wanted to understand what could be done to tackle them.
From the survey, we were able to identify five areas that we believe we should target in order to reduce differential attainment:
- Improve access to peer support in examination preparation
- Improve access to examination preparation tools
- Improve access to examination information
- Improve access to support for trainers supporting their trainees in examination preparation
- Increase the diversity of the examiner panels
What we have done to try and close the attainment gap:
In order to respond to the five areas identified in our survey, we have:
- Created a new RCS England mentorship scheme. This provided both one-to-one and group support to MRCS candidates from backgrounds that had been less able to benefit from senior, trainer, and peer-to-peer support when preparing for the MRCS.
- Ensured wider access to examination preparatory resources. We wanted to ensure that access to our resources was not determined by financial means, and that as many people as possible had access to resources written by people who had significant experience of preparing candidates for the MRCS.
- Increased the diversity of the RCS England examiner panel, particularly by appointing more women and non-binary examiners and examiners from ethnic minority backgrounds. This helped to ensure that candidates from all backgrounds felt a sense of inclusion and belonging on examination days.
- Improved the detail and availability of MRCS information for candidates. We worked closely with our colleagues at ICBSE to ensure that information about the MRCS was as detailed and accessible as possible, including clearer guidance on the syllabus and what to expect on the day of the examinations.
- Improved the detail and availability of MRCS information for Educational Supervisors. We wanted all trainers to be equipped to provide practical support to candidates preparing for the MRCS examinations, including subsequent attempts.
We have launched these projects throughout 2023 and 2024. If you have any questions or would like to get involved please contact us at diversity@rcseng.ac.uk.
