The USMLE recently released the percentile rankings (PDF) for scores on all three board exams taken between 2015 through 2017 (updated May 9, 2018). The USMLE does not publish percentile rankings with individual scores and only provides this kind of data to the public every couple of years.
In the tables below we have kept the data from previous reports, but the graphs and analysis are updated to reflect the 2015-2017 data.
This information is very interesting because we can see the actual distribution of scores compared to a normalized distribution, or standard bell curve, that we typically assume when calculating a percentile rank for USMLE scores.
USMLE Step 1
Let us get right into the data. The following is the table showing the mean and standard deviation on USMLE Step 1 for US and Canadian students.
|USMLE Step 1|
|Calendar Year||Mean||Standard Deviation|
The are some questions that we will try to answer with this information:
- What is the actual distribution of scores? Is it close to a normalized distribution or is it skewed?
- What do the different distributions really tell us?
- How accurate is our percentile calculator?
Using the data provided by the USMLE we can create a graph demonstrating the percentile rank corresponding to a given score (blue) versus the calculated percentile rank (red). The calculated rank is created with the score, mean, and standard deviation and assumes a normalized distribution. You can mouse over the following graphs to see the percentiles for any of the provided USMLE scores.
So to answer our first question, no, the actual scores do not conform perfectly to a normalized distribution but are pretty close. If you look at the graph you can see that at extremes values (very high or very low scores), the actual percentile values are higher than the calculated ones. Also, percentiles closer to the mean tend to be lower than the calculated values.
What this means is that the distribution of USMLE scores has a slightly negative skew, towards the lower scores on the left. To put this simply, overall, slightly more students did well on the exam, scoring above the mean. Fewer total students did poorly, scoring below the mean, but some of them did really poorly, scoring well below the mean, creating the skewed “tail” to the left on the next graph.
Another way of looking at the same data is to graph it as a normalized or “bell” curve. Visualizing both the actual distribution curve (blue) next to the calculated normalized curve (red) makes it easy to see that the actual distribution of scores is skewed to the left.
So what does this mean? First, we know that between 2015 through 2017, more medical students taking Step 1 scored above than the mean than scored below than the mean. However, fewer students did really well (above 250) and more students did really poorly (below 205) than would be expected from a perfectly normal distribution.
Therefore, if you used a percentile calculator to estimate your percentile rank and you did really well (scored above about 250), congratulations, your percentile rank is even higher than the calculated one! On the other hand, if you did poorly, you can be somewhat comforted by the fact that the calculator underestimated your rank.
I talked a lot about the differences between the actual percentile rank and the calculated one to highlight that the score distribution is not a perfect bell curve. However, the actual percentiles are very close to a normalized distribution. The calculator estimates the percentile based on a perfectly normalized distribution and yet it does a pretty good job at estimating the percentile rank: at any given score, the highest deviation from the actual percentile was +5.23 points and the overall accuracy was ±1.54 points.
In the future, I would not expect the actual distribution to change much, so for calculating percentile rank of scores after 2017, you can be confident the calculated percentile is accurate within a few points. If your score is on the extreme ends of the spectrum with either a very high or low score, the calculated percentile likely underestimates your actual ranking on Step 1.
USMLE Step 2
The following table is the overall data provided by the USMLE for Step 2.
|USMLE Step 2|
|Academic Year||Mean||Standard Deviation|
Again we can graph the actual percentiles reported (blue) and our calculated percentile rank (red) for each of the given scores.
On USMLE Step 2, the distribution of scores also has a negative skew. Compared to Step 1 the differences are less obvious. Again, the calculated percentiles are slightly higher than the actual percentiles between 225 and 255. This shows us the negative skew towards the lower scores. Again this means that more people scored above the mean than scored below it.
The calculator does pretty well, with the significant deviations occurring near the mean. The largest difference is +4.93 percentile points at 240, but the overall accuracy is ±1.2 points.
USMLE Step 3
The following table is the overall data provided by the USMLE for Step 3.
|USMLE Step 3|
|Calendar Year||Mean||Standard Deviation|
From the data presented in the table it is easy to see that the average score on Step 3 is lower compared to the first two exams, but the distribution of scores is also narrower (lower standard deviations).
On USMLE Step 3, there is no significant skewing of the score distribution. The actual and projected percentiles are nearly the same. The largest deviation from the normalized curve is at score 225 and is +2.13 percentile points. All other differences are < 2 with an average accuracy of ±0.46 points.