If you’ve ever wondered, “How does Josh’s mind work?” this is the episode for you. We talk about what models are, why they matter, the model Josh uses for building mental models, and — in addition to a wide-ranging conversation — how it all applies to cycling.

Got a question you’d like to ask? Text or leave a voicemail at the Marginal Gains Hotline: +1-317-343-4506 or just leave a comment in this post!

Subscribe using your favorite podcast platform (but be sure to rate and review us on Apple Podcasts).

Very nice.

You’ve probably heard the quote by the statistician George Box, “All models are wrong. Some are useful.” I prefer another statistician’s quote on models. Sam Karlin said: “The purpose of models is not to fit the data but to sharpen the questions.” Jim Martin’s model helps us focus on newer, sharper, questions.

My brother is an engineer and he once proclaimed, “Engineers think equations approximate reality. Physicists think reality approximates an equation.” I was trained as a mathematician so he waited two beats to let that sink in before finishing, “Mathematicians haven’t yet figured out there’s a connection.” So I have to work doubly hard to remind myself there’s a connection.

Interesting show with many things to ponder. One thing Josh said sounded off to me. Speaking of his implementation of the Martin model, he said something to the effect that he was able to calculate TT times to within 1% so there’s still something missing. I think it’s important to draw a distinction between model form error and parameter estimation errors. The fact that the predictions came within 1% says to me that it is less likely that the physics model is missing something of any importance than that he is seeing the limit of his parameter estimation. I don’t know how sophisticated his method is to determine and apply CdA, Crr, rho, wind speed and direction, and drivetrain losses along the route, but I’d be surprised if these were within 1% given the variability of these on the road. For example, even if extremely accurate measurements of CdA were determined for different body positions (after all no one can maintain the exact same head position for the duration of a time trial), the exact distribution of these positions couldn’t be known. Similarly for all the other parameters in the Martin model.

It would be interesting to know the distribution of model errors for different predictions of TT performance using measured power. A narrow distribution would indicate that the effect of errors in parameter estimation are limited to the ~1% quoted by Josh. A wide distribution would say the impact of parameter errors is greater with the more accurate predictions being the result of errors canceling and the larger ones the result of errors adding. I suspect it’s possible for someone with a greater data analysis background that I (Robert Chung) to determine the magnitude of these parameter errors from the model error distribution function.

This is a complicated question, and the best I can do is give a simple response, beginning with a twist on Karlin: the purpose of models is to sharpen the questions but the test of a model is how well it fits the data. That said, the ways that a model can fail to fit the data are numerous: there’s at least measurement error, parameter estimation error, and model specification error. Worse, a prediction can also err even if the model were correct if the putative inputs were wrong. So, in a large sense, that you can come within 1% of a model prediction is suggestive but unless you have some idea of why it failed you won’t know whether it was a model specification failure or a measurement error. However, you’re exactly right, errors are clues because model specification errors manifest differently than measurement errors. So, like NTSB crash investigators, we need to do forensic analysis of a failure to figure out what went wrong and figure out what we can learn from it.

I know that’s an unsatisfying response but I hope it’s a helpful response. I’ll add that VE profiles are a type of error analysis. That’s why I envisioned them mostly as a diagnostic for fit and deviations from fit from a particular model of cycling drag.