AnaLINitcs, Cognitive Bias and The Power of Context.
Everyday, our world is becoming more predictable. The sheer volume of data created in recent years, combined with the power of statistical modeling has created a world where we our analytical power is infinite, and our ability to predict future outcomes is growing exponentially, in terms of both data sources available and the reliability of those predictions.
In the context of this increasingly analytical world, the most shocking stories are those which break the mold, which defy the increasingly powerful data fueled logical treasure trove. Jeremy Lin exemplifies the power of this storyline. While this idea may explain his popularity, it does not explain why he went largely unnoticed by NBA scouts and analysts.
In the last five years, basketball has relied more and more on inferential statistics and predictive modeling to drive personell decisions. The explosion of data driven analysis, and the increasing desire to create a model where past performance strongly indicates future results matches with larger analytic trends across industries and verticals.
As someone who works with data to describe and help predict human behavior, I kept asking myself, how did these NBA data monkeys miss Jeremy Lin? Then, last week, I read about a Fed Ex driver with a basketball analytics hobby blog who predicted the potential value of lin based on a score called "RSB40" (rebounds, steals, and blocks per 40 minutes). Well, that seems like a mighty powerful inferential statistic if you ask me. Here's the full table from a cache from Hoops Analyst:
Someone saw Jeremy Lin coming. So why didn't any other NBA scouts? In my mind it is the power of social and cognitive biases and their impact on analytical thinking.
So what kind of social and cognitive bias surrounding Jeremy Lin could have outweighed the power of this empirical data?
Where to begin.
Well, there's Harvard. There's his race. All of these things have been written about profusely, so I will spare you. For great insights into the cognitive and social considerations around Jeremy Lin, these are two of my favorite pieces of writing (Grantland and Slate).
For me, the fact that pretty much every basketball number cruncher missed Jeremy Lin has made me realize more than ever that the value of empirical analysis is only as powerful as your understanding of the context within which it exists, and this context is largely framed by stuff that is hard to quantify.
This is how analysts might describe the process of using statistics to determine future performance and value of a single player.
In reality, there is a lot more to learn. There should be equal focus between developing predictive models and to understanding the context in which they exist:
Analysts, thinkers, writers, academics,need to focus equally on numbers and on context. Then we'll be better at understanding the world around us and maybe find some more Jeremy Lins along the way.