In my previous post I shared my thoughts on the current debate about running form and footwear – that post was triggered by reading the comments related to Christopher McDougall’s recent article in the New York Times Magazine. A number of commenters criticized McDougall for basing his article off of a few individual anecdotes (his own personal history included). Several people asked for the scientific “evidence” that barefoot running form is better, ignoring the fact that this was a popular magazine article and not a peer reviewed journal article (and that several notable scientists were quoted – e.g., Benno Nigg).
I’m going to avoid specific reference here to the NY Times article as others have done so in depth (e.g., this post by Alex Hutchinson). Rather, the constant back and forth about science vs. anecdote deserves some commentary, and so I’d like to share a few of my thoughts on this more general issue.
1. Anecdotes have value. No, anecdotes are not the same as a controlled scientific studies, and the plural of anecdote is not fact. But, anecdotes are examples where something has (or has not) worked for an individual. Someone tries something out, say a particular shoe, and they observe a positive benefit. Does this mean that every person is going to obtain the same benefit from trying that shoe? No, but for people suffering from a similar chronic injury, anecdotal reports of positive experiments can be very helpful. How often have you provided advice to a fellow runner in need? If a friend asks you how you overcame your own bout with ITBS or plantar fasciitis, would you simply say “I’m not telling you because there’s no scientific study proving that my approach is best.” Doubtful. We runners constantly share what works and does not work amongst our community, whether it be about injuries, training methods, or which race is best run for a BQ. It’s what makes the running community so vibrant, and such an amazing group to be a part of.
Though they may not be considered to be as valuable as randomized, controlled experiments, anecdotes do play an important role even in science and medicine. The case study is a classic learning tool in medical schools, clinicians often rely on past, personal experience with “what works” in devising treatment plans for a given patient, and case reports are commonly published in the medical literature (here’s a recent one on the effect of foot strike modification on knee pain outcomes in 3 patients). Personal and clinical experience are based on a foundation of anecdotes, and to discount this reality is a mistake.
All of this being said, it is also critical to not take lessons learned from anecdotes and apply them too broadly. Just because something worked well for one person does not mean it will work well for all. Should everyone go out and run on their forefeet under all circumstances? No, absolutely not. For some people this is probably a recipe for disaster (e.g., women who wear high heels all day), and even back in the 1950’s marathoners did not typically land on their forefeet – very few marathon runners are true forefoot strikers. Figuring out how broadly to apply anecdotal experience is where scientific study comes into play. We observe that some intervention works through anecdotal responses, then do careful, controlled testing to see just how broadly this response occurs when replicating the same intervention in different individuals. Maybe the anecdotal benefit was a fluke, maybe not. But both anecdote and controlled, rigorous study have a role in science and medicine.
2. Science has limits too. Controlled studies are of great value, but their application has limits. Pertinent to my thoughts here are two quotes that Steve Magness recently posted to twitter:
“If you are advising or treating individuals according to the average effects, you may be doing the wrong thing” via Evolution in 4 Dimensions
“Averages mask individual variation- that’s the whole point of an average.”
The problem with scientific studies, as I’ve discussed previously on this blog, is that all to often they compare group means. Lets say we took 90 new runners and randomly assigned them to either the Nike Pegasus, Nike Free 3.0, or Vibram Fivefingers (30 in each group). We devised a careful progression in mileage, and tracked their injury status for 6 months. Suppose 50% of Vibram runners reported an injury in those 6 months, whereas 40% of Pegasus runners reported an injury, and only 20% of Nike Free runners reported an injury. Suppose the differences in injury risk in the three shoes were found to be statistically significant. Based on a result like this, should we conclude that the Nike Free is the best shoe and the Vibrams should be pulled from the market? Perhaps, and I’m sure Nike marketing would be all over this, but the results only indicate that the Free performed better on average than the other two shoes. What if the 50% who were not injured in Vibrams were able to run for the first time in their life without significant pain? Should we advise them to stop immediately because the shoe performed worse than the others? Should we tell the 60% of runners in the Pegasus who had no problems that they should switch to Frees, or should we just advise that they stick with the shoe that’s currently working? What if the 20% who got hurt in Frees had really nasty injuries and we could determine that they were caused by the shoe being wrong for their foot? I think you get the point. Studies like this can provide helpful guidance, but they generally don’t have much to say about what is best for the individual. When it comes to running, individuals don’t care about average group responses, they care about what is going to keep them running injury-free out on the road or trail.
Again this does not mean that a study like this does not have value. In fact, a study just like this is currently being conducted by Michael Ryan at the University of British Columbia – see video below:
The value here is that this type of study let’s us hone in a bit on why some people do better in one shoe than another. It provides us better guidance on where to steer new runners who are looking for a shoe – i.e., it helps us better define a starting point. That’s why I found Ryan’s study on pronation control systems in shoes to be so valuable. It’s not that the stability shoes or neutral shoes never work – for many people they work just fine. But, for others they don’t, and Ryan’s study showed that the way we assign a shoe to a runner appears to be deeply flawed. Given this, I’d generally take the anecdotal experience of a running partner over most over most other sources when it comes to choosing a new shoe. My friends know what I like and dislike in a shoe, and they know my needs/preferences better than any scientific paper or store clerk. That’s not to say that a knowledgeable store clerk is not someone to listen to – it’s just that it can be hard to know at first who you’re dealing with. Is it someone working a summer job who is following an outdated company fitting policy, or is it an open-minded individual with long and deep experience with lots of shoes? The latter is what you want when you walk into a running store.
At the end of all this I hope I’ve made the point that anecdotes should not simply be dismissed, and in some cases they might be even more valuable than the results of a scientific study. As a scientist myself it might be a bit heretical to say something like that, but it’s what I believe. McDougall’s conclusions may not have been based upon a foundation of studies that clearly demonstrate that barefoot-style running is indeed better, but neither have the shoe companies designed shoes based upon rigorous and open scientific efficacy trials. Don’t believe me? Here’s a quote from Ryan’s recent paper in the British Journal of Sports Medicine: “…despite over 20 years of stability elements being incorporated in running footwear there is, as yet, no established clinically based evidence for their provision.” And what of anecdote? Running magazines and books are full of advice based on them, it’s not just McDougall’s article. In fact, l admit to being guilty of it myself – all of the shoe reviews I’ve ever written amount to nothing more than personal anecdotes.
Maybe if McDougall hadn’t referred to the 100ups as foolproof or if he had attached a disclaimer that “individual results will vary” this would have appeased some of the more critical readers. I do agree that there is risk associated with advocating too hard for a particular running style or footwear type – some people get enthused with a new idea, push too hard with it, and get hurt. Determining just how strong we should push one type of footwear over another is where I do think the more rigorous scientific studies like Ryan’s will help to shake things out. But, I’m certain that his results will not end this debate :)
You might also like:
I Think I’m a Pseudo-Heel Striker!: Study Suggests That As Many as 25-33% of Heel Strikers Exhibit a More Midfoot Loading Pattern
More On Running Cadence: Comparative Data from Amby Burfoot and Alex Hutchinson
Proprioreception: Additional Thoughts on Balance, Stability, and Running from Jay Dicharry
Running Form, the Origin of Minimalism, and the Elite vs. Recreational Divide
Effects of Running Speed on Foot Strike Patterns and Identification of Multiple Heel Strike Types
About Peter Larson
Great article. In the end only one anecdote matters: YOUR OWN. If barefoot running works for you, go for it! But if not, thats fine too, don’t worry about it.
Smith
Totally agree the only thing that matter in the end is your own experience. I can tell after wearing orthodics for over 25 years and then ditching them and going to mimimalist shoes I have had my first injury free year of running since 2007.
Good points on the value of anecdotal and scientific evidence – not every scientific article is correct (and, in fact, many are retracted for various reasons), and anecdotes can form the basis for a lot of different practices, even in science (see anyone teaching you a short-cut in setting up, running, or working-up a chemistry experiment, for example).
I also find it interesting how many of the naysayers who say they tried barefoot running and had many injuries ignore the fact that our running in shoes is a learned behavior (conscious or not), one that needs to be “unlearned” before running properly.
It will be interesting to see what happens with kids like yours who are given more “minimal” type shoes as they grow up, and if their running form changes over time. A long-term, prospective study like that (which I’m sure parents would be happy to participate in if the shoes were provided…) may provide the best evidence regarding how more built-up shoes change our running form.
Anecdotes are powerfully influential, especially when they come from people we trust, most notably ourselves. We tell stories to ourselves to validate our decisions. We narrate our lives according to how we want the story told, not how it actually happened. Alex Hutchinson’s post reminds us of that, noting how McDougall did not in fact run injury-free after adopting a new form, but in retrospect may have conveniently forgotten his 2 year bout of plantar fasciitis. I’m as a guilty of that as anyone. I think back on a great year of running, easily forgetting the 7 weeks off with a fibular stress fracture, or the chronic Achilles tendonitis. Minimalism gave me new speed and overcame a running plateau, but also created new weaknesses and injuries. The joy of running so easily overwhelms the pain. Why else would we sign up for a new race just a few days after painfully finishing another marathon or ultra?
I’ve been thinking about placebo lately, and it’s effects on running form. I think it’s particularly pronounced in anecdotal evidence. For example: heel-toe drop. So many shoe reviews note heel-toe drop, but then state that the shoe “feels flat” or the heel “feels high.” Is anybody studying whether actual vs. stated heel drop causes runners to change form. Could we tell Group A that their shoes have 10 mm of drop and document heel stiking, but we tell Group B (wearing the same shoes) that their footwear has only 4 mm and we see more forefoot striking? I think we might, especially if the study subjects were informed about minimalist running and were expecting a certain result. The human body has an amazing ability to sense subtle differences and adapt. Barefoot running teaches us that. But the human brain also has an amazing ability to fool itself and the body. We can see this in the runners in FiveFingers who continue to heel strike, but are convinced that minimalism alters their form.
My biggest worry about anecdotes, though it applies to controlled studies a well, is the conflation of cause and effect. Correlation does not equal causation. Inevitably, we forget that. We think that the only part of our training that we alter is our footwear, and we attribute to that change a decrease in injury. Outside of a laboratory (and even inside a lab, for that matter), it is difficult to truly isolate a variable. With a change in shoe, such as from stability trainer to minimal flat, likely comes a change in training intensity, mileage, periodization of mileage, socks. Therefore I will always give greater credence to anecdote, if adequately documented. An experiment of 1 still needs good experimental design, or the results lose value.
As always, Pete, another excellent, thought-provoking post.
Great thoughts Pete and this is a terrific summary of the pros and cons of various approaches to developing evidence in support of a theory. The third paragraph discussing anecdotes is key – it is important to not apply the results too broadly, and the same can be said of scientific research as well. All studies have limitations, and often in the summary of such studies by the media, these limitations are omitted. Yet there often lies the key in understanding how broadly applicable the findings are. If the study, for example, focuses on elite athletes, are the findings really relevant for the everyday Joe?
I’d have less problem with McDougall’s approach if he were a bit more nuanced and open in his statements. Granted, this may make him a less entertaining writer, but at the end he needs to decide what he wants to be – a storyteller, or a researcher. He walks the line between both and it hurts his credibility, especially as the latter. As I wrote in my own review, maybe he is the Steve Jobs of the footwear industry, and maybe he’s just ahead of his time. But Jobs was wise enough to step away for a bit when he realized the world wasn’t ready for what he was preaching. It’s not clear McDougall would do the same.
Fantastic read! One thing though is that, as we throw in the “results may vary” type statement, we run the risk of the perception that the thing being discussed will likely work for the vast majority. And, in this case at least, I’m skeptical that it will. As a disclaimer, I’m a severe over-pronator who didn’t get relief from shin splints until I started using orthotics. And, since then, my trusty Brooks Adrenalines, sans orthotics, have played a huge role in keeping me on the roads. I’m just one guy though and that’s my personal experience, making me one case where the results have definitely varied, which leads me to my next point.
It’s been my experience that most people who get into running later than, say, 30 or so, do so because they want to lose weight. And, if that’s true, then it follows that a fairly high percentage of beginners will be moderately to severely overweight. Imagine the impact that will have to be absorbed by the legs, hips, connective tissues, etc… when a 240lb+ person, who possibly hasn’t seen anything faster than a walk in a good long time, goes out to hit the roads for the first time. Perhaps I’m shortsighted or simply unreasonable here but I can’t imagine that a minimalist shoe would be the way to go in a situation such as this. I’ve spoken with my shoe guy, who sells just about everything, about this a few times and he’ll tell me story after story of overweight people who wanted the latest cool stuff on their feet, only to abandon running within a few weeks due to injury.
Again, this is merely “it seems to me” type logic with a little bit of “some guy said” thrown in, and I’m totally open to being wrong, but I’d be surprised if a minimalist approach would benefit 50% or more of those who purchase running shoes. I don’t fault the science or theories, just our own capacity to stay away from couches, cheeseburgers and supersized portions of pasta. I’d like to say that, in a perfect world, we’d have an undisputed one size fits all solution but that would make blogs on running pretty boring.
Thanks for the comment. I will throw one thing out there though – there have been studies that have found no link between BMI and injury risk in runners, and the speculation was that it’s because they runner more slowly and probably lower mileage than more fit runners. What’s more, impact is increased with a heel strike in a cushioned shoe over a forefoot strike in a minimalist shoe, so the idea the big runners need big shoes may not hold water. I know big guys who run barefoot without a problem, so this may be just another example where what seems like common sense may not be entirely accurate, and there are anecdotes pointing both directions. McDougall himself is a pretty big guy, and was even larger when he wrote Born to Run than he is now – he’s lost a lot of weight over the past few years.
I will also say that a lot of the speculation that suggests that injury rates remain high because modern runners are out of shape may be off a bit as well. Many of the injury studies that are out there are looking at experienced runners and not beginners. Seems to me that a fit runners are not escaping the injury bug.
Sent from my iPad
Well shoot, nothing like having the wind taken out of your sails. Keep up the awesome work and thanks for your dedication. Also, congrats on the Runner’s World article a while back!
It’s a commonly held belief that injury rates have stayed the same because more people are running, and more of them are less fit. It’s one of those things that sounds great and makes sense, and it’s said all the time, but I’m just not aware of any evidence to support the belief – needs to be studied in more depth. All I can say is that I know a lot of very fit runners who deal with lots of injuries, so it’s not just clydesdales who are skewing the stats.
Sent from my iPad
I truly appreciate this balanced approach to viewing research.
I try to remind my students regularly that correlation does not always equal causality, and that while we spend quite a bit of time discussing the characteristics of a population “on average” there are always outliers. (I’m a demographer by day and teach Intro to Sociology by night…) You can wind up in quite a bit of trouble if you presume, for example, that all people from France have exactly the same values, motivations, and lifestyles. It may be true that many French residents enjoy good wine and cheese, but it is just as true that there are lactose-intolerant non-drinkers in France.
In the same vein, there is a wide spectrum of running styles and body types. Just because a technique works for “most people” does not mean it works for all. I particularly liked the quote “If you are advising or treating individuals according to the average effects, you may be doing the wrong thing.”
Interesting post! I am just starting to try out barefoot running and as a blogger, I’m trying to be careful because I know what works for me won’t work for everyone. But at the same time, it is good to hear people’s experiences and then ultimately each person can try for herself/
Im confused. First you,along with many other barefoot/minimalist proponents, blast mainstream shoe companies for not using scientific data to back up their claims that their shoes reduce running injuries.
Now, after McDougall’s article gets criticized for an over emphasis on anecdotal evidence, you say that It’s ok to select footwear based not on science, but on feel? Btw I bet the shoe designers at Brooks can cite a ton of anecdotal evidence that shoes like the Beast do reduce injuries.
And if a runner is wearing the Beast and is not injured, by all means keep doing it. I just don’t buy that there is any good evidence that every person with a flat foot should wear the Beast, and in fact there is published evidence that people who should be wearing the Beast according to prevailing wisdom get hurt in them. I’m a proponent of variety, and for too long we,ve been told that we need shoes of a certain type, and there is no good data to support that model. In the absence of such data, anecdote has a role, as does personal experimentation. I absolutely believe that science is important, and there is a lot of good science supporting the fact that a more barefoot-style stride can have positive benefits on the loading of our joints when we run. At the end of the day, doing what works for you is all that really matters.
By the way, McDougall quoted Benno Nigg in his article – he’s one of the leading biomechanists in the world and was heavily involved in the development of the pronation control model. He now apologizes for this and says pronation should not be much of a concern. There was more than just anecdote in Chris’ article.
Pete — I always think your posts are interesting and well written, so I apologize in advance for mentioning a grammatical hobbyhorse of mine, but the phrase is “home in”, not “hone in”.