The 2010 CrossFit Games
The ultimate proving grounds of the world’s fittest athletes.
July 16-18, 2010 • Carson, CA
The Home Depot Center Sports Complex
Select a 2010 CrossFit Games event
United States Qualifiers
Self-Reported Versus Rigorously Judged WOD Results
Hari Singh of CrossFit NYC delivers a dose of reality
Tell a member of the CrossFit community that Chris Spealer says his “Fran” time is 2:11, and you’ll probably get one of two responses: acceptance of a perfectly plausible fact; or a brief pondering over whether it really takes Speal that long to do anything. (My own reaction fell into the first category, after which I ambivalently noted that my own “Fran” time is approaching a major benchmark: three times Speal’s time.)
However, many CrossFitters do wonder about the accuracy of self-reported WOD results posted on the Main Site and elsewhere. The most cynical seem to assume that virtually all self-reported scores are suspect. At the other end, many people simply accept that on average the results are accurate, and that while some scores may be nonsense, most seem reasonable, particularly those of athletes who are recognized as members of the community.
The data from the 2010 CrossFit Games allows us to make a direct comparison for two WOD’s (1RM dead lift and “Fran”) for one group of athletes, the Masters competitors. Though it is possible to draw different conclusions, nothing in the data appears to be inconsistent with the theory that experienced CrossFitters do the best they can to accurately report their results. Or, to put it another way, CrossFitters seem to tell the truth, as best they know it.
During the course of a WOD like “Fran,” it’s easy to lose count, fail to go below parallel, lock out, or get chin over bar. These things typically happen without the athlete knowing it. For a great example, see the Main Site video on 29 July 2010, where Miranda Oldroyd does “Isabel” in 2:22. After seeing the tape, she realizes that she missed a rep. If it happens to an athlete like Miranda, it happens to everyone. The video is worth viewing, just to see how Miranda deals with her mistake. (For a great example of the opposite type of error, see the video from the 2010 European Regionals, where Annie Thorisdottir self-reports missing and then redoes a rep on the CTB pull-ups that the judge had, in Annie’s opinion, incorrectly counted.)
On a WOD like “Fran,” we would expect good judges to catch all of the things that the athletes themselves do not, and that for the Masters, the Games results would be somewhat worse than the self-reported results. John Matzner (CrossFit Potomac) a competitor at the 2009 CrossFit Games suggests that for a WOD like “Fran” the adrenaline rush during competition may actually work against some athletes, causing them to go out too fast and leave less than enough in the tank for the final rounds. This phenomenon may have been in play during the first WOD of the 2010 Games when several competitors went from doing nine muscle-ups nearly unbroken in the first round to missing multiple single attempts in subsequent rounds.
During the 2010 Games, the Masters competitors did “Fran” on Sunday, after having a full day off on Saturday. Thus the effect of fatigue should have been minimal. Before the Games, 22 of 26 athletes (11 of 15 men and 11 of 11 women) self-reported their “Fran” times. On average, the 22 athletes did just 35 seconds worse than their previously reported results. While there are many possible explanations for the 35 second average difference, I am convinced that exaggeration of prior performance accounts for little or none of it. If this group of athletes were prone to exaggeration, then surely they would also have also exaggerated their 1RM dead lift numbers. The data show the complete opposite.
Before the Games, 22 of 26 (12 of 15 men and 10 of 11 women) reported their 1RM dead lifts. On average these 22 athletes pulled 9.77 pounds more than their previously reported results. In fact 19 of the 22 (including all 10 women) matched or exceeded their self-reported results.
What accounts for the relatively high accuracy of self-reported WOD results in comparison with other data on (e.g. data on dating sites, which is notoriously inaccurate)? As many of us have observed, CrossFitters are an incredibly honest group. To CrossFit requires a willingness to tell the truth; to accept the fact that we are not good at everything and that we are (at least for now) terrible at some things. CrossFit is quantitative, not qualitative. No CrossFitter says, “I seem to be lifting more weight.” Instead they know—and want to know--with precision exactly how much more weight. In other areas, three-quarters of the population can tell themselves that they are above average, but not in CrossFit.
There are always friends and family who will tell you that you are smart, beautiful, and special, even if you are not. People you CrossFit with will tell you the truth. You didn’t get below parallel. You didn’t get your chin over the bar. People who CrossFit don’t want to be lied to, and the data show, they don’t lie.
Apparently it really does take Speal that long to do “Fran.”