Amazon: 10% off running shoes for Prime members - click here to view current selection.
Running Warehouse: Great prices on closeout shoes! View men's and women's selections.

On the Limitations of Science in the Study of Running

Histogram of sepal widths for Iris versicolor ...

Image via Wikipedia

As runners, we often want concrete, black and white answers to questions about things like shoe design, running form, training methods, and so on. Does running in a minimal shoe make you less prone to injury than running in a traditionally cushioned, heel-lifted shoe? Does switching form to a midfoot strike make you faster? Will wearing compression sleeves make you recover faster after a race? Unfortunately, answers to questions like these aren’t simple, often because there are too many competing variables that make clear-cut conclusions difficult. Furthermore, an answer to one of these questions might not apply to to every single person – humans are variable, and changes will affect people in different ways. A given shoe might be great for one person, but terrible for another – it’s really hard to say what the result will be with any certainty.

As an example, I ran the Manchester City Marathon last November in a fairly minimal racing flat (Saucony Grid Type A4). My legs suffered during the race, and I irritated something in my left foot which cause me some trouble for about a week afterward. The easy and most tempting conclusion was that the shoes were to blame – I wore a flat for a very long, intense race, and it was the wrong choice. However, the culprit could just as easily have been the difficult, hilly course combined with the fact that I had just run an intense, BQ effort marathon a month before. In fact, in reading my race report from the same race the previous year, I experienced similar leg issues in Saucony Fastwitch lightweight trainers, which are much more cushioned and have a standard heel lift. Furthermore, a few weeks before the Hartford Marathon in 2009 I had irritated my foot in seemingly the same way as I did in Manchester, so maybe I’m simply susceptible to irritating what I think is my peroneus longus tendon. So where does this leave me? It leaves me to conclude that I have no idea exactly how I hurt my foot, why my legs suffered, or whether the shoes were to blame or not (or how much blame I should attribute to them). Such is the difficulty of trying to pinpoint clear causative factors when it comes to running injuries and the like.

One way to get at answers to questions like these is to turn to scientific experimentation. If we could run more controlled comparisons and hold as many variables constant as possible, maybe we could come to firmer conclusions. This is great in theory, but even science has its limitations. Let me walk you through a hypothetical example. Suppose we were interested in whether a given shoe could encourage a flatter, more midfoot foot strike (a very big goal for a lot of people these days, and a big marketing claim for manufacturers). Let’s take the two shoes I mentioned above, the Saucony Fastwitch and the Saucony Grid Type A4. Both are very lightweight, but they differ in some basic structural properties. The Fastwitch has a standard heel lift, whereas the Grid Type A4 has a much lower heel (see pictures below). We could have 20 people run by a high-speed camera in each shoe, measure the angle between the foot and ground for each, and compare the results between the two shoe conditions.

Saucony Fastwitch

Saucony Grid Type A4

It’s quite possible, perhaps even likely, that the runners would on-average exhibit a flatter-footed, midfoot style landing in the flatter shoe (the Grid Type A4). However, the key phrase here is “on-average.” I would almost guarantee that there would be individuals who would buck the average trend and perhaps not show much difference between the shoes, or even show the reverse pattern. Therein lies a problem with scientific studies of running – in order to derive meaning from messy data, we use statistical analyses to compare distributions of numbers and means for groups, and often lose sight of the individual. So, we might conclude from this study that the Grid Type A4 is a better shoe, on average for this sample, at encouraging a midfoot strike, but that does not mean that it will serve this purpose well for every single individual. Perhaps other factors associated with the design of the shoes (e.g., midsole firmness, heel shape, forefoot width, etc.) will uniquely affect how isolated individuals will land in them.

In reading scientific studies of running biomechanics I see this often – a study will demonstrate a pattern, such as that a particular shoe design reduces ground impact loading rate (how rapidly the foot impacts the ground), but when you look at the variation among subjects it falls out over a fairly wide range. In other words, the results are non-systematic, and the individual gets swamped by the general trend. However, that doesn’t mean that the outlying individual who does not exhibit reduced impact loading rate in the shoe, or who might even exhibit increased loading rate, is unimportant. Individuals that don’t follow the general trend don’t want to get hurt either and would not benefit from following the advice suggested by the data – this is why scientists are often bad at making concrete statements, and why I always hesitate to do so on this blog. We tend to hedge and qualify statements by saying that our data “suggest” a given pattern, or that a “significant effect” was found. We rarely “prove” anything. I’m not saying this is bad, it’s just reality.

As a scientist I believe strongly in the value of controlled experimentation, but I also believe that we need to constantly keep in mind its limitations. Neglect of the individual in favor of the overall trend is one of them. Humans are variable, we are not machines, and we shouldn’t be treated like them. Science can be a very useful guide, but rarely does it provide absolute, definitive answers. It’s very hard to know in a black-and-white way what the effect of any shoe design, form change, or training method will be for any given person. This is the reason why scientifically informed, yet careful and open minded self-experimentation is perhaps ultimately the only thing that can truly work on an individual level.

Enhanced by Zemanta

Track and Field Shoe Sale at Running Warehouse!

Running Warehouse Track Shoe ad
Running Warehouse Runblogger Sidebar

Save $$$ On Shoes Gear:
Running Warehouse: Great prices on closeout shoes! View men's and women's selections. 25% or more off clearance running shoes - click here to view current selection.
Connect With Me On:
Facebook - Runblogger | Twitter - @Runblogger | Instagram - @Runblogger

Recent Posts By Category: Running Shoe Reviews | Running Gear Reviews | Running Science
About Peter Larson

This post was authored by Peter Larson. Pete is a biology teacher, track/soccer coach, and dad (x3) with a passion for running, soccer, and science. If you'd like to learn a little bit more about who I am and what I do, click here, or visit


  1. I think there’s a significant issue with respect to trying to run any sort of analysis on running data: margin of error.

    Sure, a race chip time is accurate – but how reproducable are the results? How deep a data set is needed from a runner to establish reliable, reproducable data?

    And that’s just one issue w/r/t timing. What about other, difficult-to-quantify data, like pain?

    The difficulty in having rigorous data is a big hurdle to a lot of research – at least when trying to state a reproduceable thesis.

  2. Sports injuries is related to the field of excercise immunology; I think the reason for getting injuried can also related to one’s health and immune system and not only biomechnics, making the subject even harder. One question may be how for instance stress fractures are related to Vitamin D staus and bone health.

  3. While I understand that humans are highly variable, I would also think that there are some fundamental biomechanical principles that would lead individual runners to increased efficiency and decreased rate of injury. Perhaps with enough “scientifically informed, yet careful and open minded” individual experimentation we would see results converge on these principles.

    Also I believe that each individual’s running form is heavily influenced by prior experience and preconceptions. How do you eliminate those variables? Bring out the mind eraser!

    Anyway, another great post. Sounds like a perfect introdution to your book…

    • Pete Larson says:

      Absolutely – I do believe there are general principles that apply. What
      those are is the big question. For example, running science has focused in
      large part on heel-toe running for the past 30 years because that is what
      most people do, but now with the minimalist movement and recognition that
      form may have been overly tied to our shoes during that time period, a lot
      of the general principles we have developed are being called into question
      and there is a lot of uncertainty right now. Does what we have learned
      about pronation in heel strikers apply to mid or forefoot strikers for
      example – probably not entirely, so you need to be careful when you apply
      principles generated by a lot of the past studies. I can virtually guarantee
      that the population in most of the injury studies we have consisted mostly
      of heel strikers wearing typical running shoes. Do a similar study on
      minimalists in flat shoes and predominant injury types are likely to change
      (things like met stress fractures, calf/Achilles injuries probably become
      more common).

      It’s a complex, yet exciting time to be trying to figure this all out, and
      each runner is part of the learning process.


      • I agree. So have an individual experiment with different running forms (scientifically, carefully, & open-minded) and see which form is most efficient and injury free. Apply this to many individuals and compare. That is when the basic principles emerge and that is something we can learn from.

  4. This is why I get so frustrated with my past running injuries – there’s simply no great way to pinpoint exactly what caused it. In all likelihood, it was a combination of factors. In practical terms, how do you use this information to create better training? My initial thought is to introduce a lot of variety into your program – hills, shoes, running surfaces, paces, etc. so the body gets stronger and better at adapting. When you’re doing the same run, in the same shoes, at the same pace over and over again is when I get nervous about injuries.

    • Pete Larson says:


      Yes, I agree totally. Repetitive, unvarying stress is a killer. Add in a
      biomechanical flaw, a bad shoe, or excessive fatigue and something is bound
      to break. I’m a big believer in variation – I do it with footwear and
      elevation, but mixing in trails, cross training, strengthening are probably
      also good approaches.


  5. FoCoRunner says:

    Nicely explained, Peter. When I taught a course or two awhile back, I used to tell the students that biological data are messy. The general trend in the data can be informative for sure, but often we learn a lot more about a phenomenon that is important in individual cases when we ask what is causing the messiness (variation).

    • Pete Larson says:

      Either that or we trim the outliers because they cause us to violate
      assumptions of normality or homogeneity of variance 🙂 Thanks Mark!

  6. Runnermatt says:

    I was thinking along similar lines this morning. I think the response to this is runners seeking a “magic bullet” for their own problems. (to be fair, there are times when I’m as guilty as anyone) I saw where the Newton guys stated the he could improved one’s marathon time by 30 minutes just by switching shoes. To be honest, that is why I bristle at the whole barefoot/minimalist running movement- too much focus on one aspect.

    My general approach is an experiment of one. I am not afraid to try varied approaches and analyzing if they are working. While results are nice, I simply enjoy the process of becoming a better runner. Even if I make a few mistakes along the way.

    • Pete Larson says:

      “Experiment of One” is exactly right – Dr. Sheehan was a smart guy. I
      advocate for minimalist shoes because I want more variety in the
      market. Are they for everybody – probably not, but if traditional
      shoes aren’t working for you (which they clearly don’t for some
      people), or you like to experiment, why not give them a try.


      • Pete,

        I didn’t mean my post as a dig or anything. I think you’ve viewed the issue from more side than most. I also think you wisely note that shoes are just part the equation. (the same could be said of high-mileage for example). But I don’t see that everywhere.

        I agree that I am personally loving the variety of shoes that has bubbled to the mainstream. It was always kind of there but racing flats were overlooked for years.

        • Pete Larson says:

          I didn’t take it as a dig, no worries. There are a lot of people on
          both sides who are dogmatic that minimalism is either the only way or
          that it is crazy. I prefer a middle ground, though my personal tastes
          are more toward the minimal end of things. It’s true the flats have
          always been there, but even today, if you go to a mall or big box
          sporting goods store you rarely see them on the wall. Went to my local
          Dick’s a few weeks ago and of 92 men’s running shoes on the wall, not
          one was a flat or a minimalist style shoe. Lots of people still have
          no idea that other options are out there.


        • Pete Larson says:

          I’ll add this too – I asked the clerk at Dick’s if they carried Nike
          Free and I don’t think he even knew what I was talking about. He said
          they just put out what Nike sends them, and they display things as
          Nike specifies. Nike is good to us was his comment.


  7. Very good post, and fully agreed. As you noted the problem and challenge for all of us as Sheehan-esque ”experiments-of-one’ is to draw valid conclusions as to what works and what doesn’t as since we only very rarely modify just one variable (e.g. the heel-to-toe drop of our running shoe) while not simultaneously varying others (e.g. our weekly mileage, our training intensity/amount of speed work, etc.). Consequently, after observing a step change in our performance, or conversely suffering an injury, it’s tempting but very difficult to separate from the likely causes which was the most probable in having had the greatest impact. While your process is absolutely correct via making inferences only after referencing your own training diaries, even with that resource it’s still very difficult to draw a firm conclusion when multiple variables are at play. Consequently, even recognizing the inherent variabilities it would be very interesting to see some scientifically controlled studies looking, for example, at large populations of runners to establish whether, for example, a high heel-to-toe drop “typical” training shoe leads to a greater than expected injury frequency or a slower average pace than a control group of a comparable population of runners with “minimalist” shoes providing a very low heel-to-toe drop.

    • My lengthly running “career” has allowed me the opportunity to modify one variable at times. One thing that has absolutely helped in this is having a benchmark race. I do the same 5 miler here every year for the last 12 years and a few years before that. This, coupled with some comprehensive logging, helps. As does educating myself about as many running studies as I can find ( and not just ones I want to agree with)

  8. Laminator says:

    Excellent post Pete. I like how you are convincing people that scientific experimentation is never “clean” and that despite what we would like to believe, there’s hardly ever going to be one shoe or one method of running that’s going to work for everyone. Even for one single individual, the body changes over time so what works at one time might not be so good a year or two years or five years down the line. In scientific studies, we are always just looking at averages and percentiles and nothing is ever 100%, so how can we know we have the “truth” about anything…much less advocate it for others. We can only dispense advice that will “more likely work”. That is often the best we can do.

    That’s why for shoes, i advocate for shoe rotation. As for your specific examples of Kinvaras, Fasttwitch and Grid A4 Racers, I am and have been running in all three and have personally found that only the Kinvaras hold up in the longer distance races. I use the last two only for 5K-10K distance races.

  9. This actually makes for a very interesting study. I agree with you that everyone is different, and science does have its limitations. However, it would be nice to know whether my choices for shoes are all based on facetious believes in my head, or that I am actually making the best choice based on the best knowledge out there. And honestly, from an economist point of view, this is a totally do-able study. The setup would require a group of ppl that we can track for a year (ideally 1000+), we need to collect information on what shoes they have worn, how many miles they have run on each shoe, what kind of injuries have they had in the past and what kind of injuries they have now, and maybe other things that they do that are running related – i.e. stretching, taping, icing, etc. Hmm too bad I’m not in academia anymore, but this could be very very interesting.

  10. Greg Strosaker says:

    There is no such thing as a truly “controlled experiment” for such an activity as running (despite what some studies will claim), as there are far too many variables to possibly address and control. Even if you were to track one runner over time, the variability in their own mood, energy, nutrition, hydration, physical condition (wear and tear, illness, etc.) would make it impossible to draw clean and neat conclusions. As data-driven as I am, even I recognize that common sense, trend evaluation, and just plain gut instinct play a big role in reaching conclusions about what works for an individual runner or the running community collectively. Thanks for the “authoritative” reminder of such truths, Pete.

Shop Running Warehouse – Summer 2019

Speak Your Mind

 Notify me of followup comments via e-mail

You can click here to Subscribe without commenting