Survivorship Bias

Winston Churchill was, perhaps inadvertently, an advocate of survivorship bias.

Survivorship bias is a type of selection bias where the results, or survivors, of a particular outcome are disproportionately evaluated. Those who "failed", or did not survive, might even be ignored. Focusing on the survivors can result in a false, or incorrect, estimate of probability. For instance, some people might look to those who've won the lottery when trying to find "tricks" to win it themselves. If half of the winners drove red cars before they won the lottery, someone might decide to get a red car to better increase their chance of winning the lottery. In reality, this is coincidental.

This particularly pernicious type of bias is common in everyday life. Educational systems and the media tend to highlight those who've been successful in a particular profession. Young athletes admire and emulate the big name all-stars. While this isn't, strickly speaking, wrong, it's often more useful to look at all of those who had potential, tried, and failed; then to work to avoid the mistake that led to their failure. Admiring celebrities and success stories is great, but there is much to be learned from those who tried and failed. Survivorship bias isn't only limited to celebrity idolization, it was famously addressed by statistician Abraham Wald to protect fighter planes, effects perception of market performance, and international calculations of average lifespan.

The American military called on statistician Abraham Wald to help them determine what parts of the planes they should reinforce. Due to weight constraints, it couldn't be everything. Initially military engineers wanted to reinforce those places where surviving planes had bullet holes, figuring that these were commonly hit spots. These spots were the wings, tail, and cockpit.

However, Wald pointed out the survivorship bias implicit in their logic. A plane could actually get shot multiple times in the wings, tail and cockpit, and still fly. It was the rest of the plane that needed reinforcement.

A common phrase in any investment prospectus is "past performance is not indicative of future returns" (or a variation thereof). Yet many investors still rely on a fund or market's historical performance or fail to factor survivorship bias into their judgment.

For instance, the S&P 500 is one of the leading stock indexes in the world, indexing to 500 stocks that its governors feel best represent the overall market. However, an individual investor would not necessarily generate the same returns as the S&P 500 simply by investing directly into the currently indexed 500 companies. This is because the S&P cuts and replaces companies overtime. It selects only those it feels most fit, biasing towards those companies that are succeeded and survive.

Suppose we were mutual fund managers looking to invest in the best fund managers we could find. We could look at the ten-year past performance of all managers we're considering today, but this is biased upwards. Why?

---

By only looking at funds that are alive today, with ten years of performance, we ignore all of those funds that failed in the past ten years. Intuitively, we want to believe that the "living" funds are run by better managers. But in a random sample with some probability of failure, there will be funds that last ten years because of luck, and luck is not a reasonable investment strategy.

It would be best to include failed investment funds in our selection process, at least, to determine an expected return, factoring both those that succeeded and failed into our calculations. $_\square$

Additionally, there is also significant periodical and academic literature debating whether funds do^{[1] [2]} or do not^{[3] [4] [5]} outperform the market. The consensus seems to lean into the conclusion^[6] that they do not. However, This is significantly complicated by the fact that many funds fail quickly, so survivorship bias presents a view of success that may not be true on average.

One common mistake is to assume that average life expectancy is constant--that a person's life expectancy doesn't change no matter how old they are now.

To put this another way, if the average life expectancy from birth of a population is $82.4$ years, how much longer does a $72$ year-old have to live?

---

The intuitive answer is $10.4$ years. However, the average, according to the US Internal Revenue Service's Life Expectancy Tables ^[7] is 15.5 years. In fact, even at 82 years of age, someone is estimated to live an average of 9.1 more years.

This is because an average life expectancy at birth includes the full population set, including those who die at birth or die of an accident in middle age. As an individual ages, their life expectancy changes. In fact, higher infant mortality rates tend to lower life expectancy calculations. For instance, the country of South Africa had, in 2014, an average infant mortality rate of 34 deaths per 1,000 births^[8], and one of the lower life expectancies in the world, at 57 years of age^[9]. Yet 11.5% of their population is above the age of 55^[10].

1. Jaffe, C. 90-of-fund-managers-beat-the-market. Retrieved May 25, 2016, from http://www.marketwatch.com/story/90-of-fund-managers-beat-the-market-but-their-shareholders-dont-2015-01-21
2. Johnson, S. Active-fund-managers-really-can-pick-stocks. Retrieved May 25, 2016, from http://www.ft.com/cms/s/0/7883ef7e-f3d9-11e4-a9f3-00144feab7de.html
3. Sommers, J. How-Many-Mutual-Funds-Routinely-Rout-the-Market?-Zero. Retrieved May 25, 2016, from http://www.nytimes.com/2015/03/15/your-money/how-many-mutual-funds-routinely-rout-the-market-zero.html
4. Domato, K. When-It-Comes-to-Fund-Performance,-History-Is-Often-Written-by-the-Winners. Retrieved May 25, 2016, from http://www.wsj.com/articles/SB10000872396390444025204577545460615317218
5. Brown, S., Goetzmann, W., Ibbotson, R., & Ross, S. Survivorship-Bias-In-Performance-Studies. Retrieved May 25, 2016, from http://www.eco.sdu.edu.cn/jrtzx/uploadfile/pdf/empiricalfinance/04.pdf
6. Buttonwood, . Practice-Makes-Imperfect. Retrieved May 25, 2016, from http://www.economist.com/news/finance-and-economics/21611090-even-experienced-fund-managers-dont-beat-market-practice-makes-imperfect
7. Internal Revenue Service, . Publication-590-B-(2015)-Appendix-B.-Life-Expectancy-Tables. Retrieved May 29, 2016, from https://www.irs.gov/publications/p590b/index.html#en_US_2015_publink1000231236
8. The-World-Bank, . Mortality-rate,-infant-(per-1,000-live-births). Retrieved May 29, 2016, from http://data.worldbank.org/indicator/SP.DYN.IMRT.IN?order=wbapi_data_value_2014+wbapi_data_value&sort=desc
9. The-World-Bank, . Life-expectancy-at-birth,-total-(years). Retrieved May 29, 2016, from http://data.worldbank.org/indicator/SP.DYN.LE00.IN?order=wbapi_data_value_2014+wbapi_data_value+wbapi_data_value-last&sort=asc
10. Wikipedia, . Age-and-sex-distribution-South-African-National-Census-of-2011. Retrieved May 29, 2016, from https://en.wikipedia.org/wiki/Demographics_of_South_Africa