(Interestingly, it appears Rumsfeld did not pen the koan himself; correspondent J.S.S. noted that the original source may be Landmark Education of Seattle, Washington.)

I recently read two fascinating accounts of why we have such a difficult time knowing what we don’t know: it’s called survivor bias, and what that means is we only get information from the survivors, not those who perished and vanished from the records.

A mid-list author recently explained how listening to the handful of authors who make it big financially is completely misleading: Survivorship bias: why 90% of the advice about writing is BS right now.

Here’s how this works: big-bucks Author Z says, here are the 10 steps you need to take to become as successful as me. The list always mentions perseverance, being nice to your readers, writing 1,000 words a day and so on.

The 99.9% of writers/authors who make less than $10,000 a year from their writing (and the 99% who make less than $1,000) take this list as a script or program that if followed, will yield great success. But the 99% follow the script and do the 10 things and discover they are still unknown and not making any money.

The numbers of people who become successful in these sorts of high-competition careers is vanishingly small–hedge fund managers are an example, along with musical acts, artists and writers. A handful at the top make most of the money, a relative few make some money (let’s say a middle-class income) and then 99% make near-zero.

Longtime correspondent B.C. recently sent me an analysis of how many mutual fund managers outperformed a “dumb” low-cost index fund. B.C. found that 66 managers out of 26,507 outperformed the SPX over 5 years. That mirrors the distribution of outsized success among hedge fund managers, a few of whom net $500 million each annually while the rest underperform a plain old index fund.

The conclusion is that luck is the ultimate factor in these signal-noise levels of success, and being in the right place at the right time with the right product/idea can’t be replicated by following a script or list of tips.

The basic idea here is that studying the aircraft that come back riddled with holes from bombing missions leads us astray when we try to analyze the damage: what would really help us is studying the planes that were shot down, but they are unavailable for study.

The same principle applies to restaurants: the few that become roaring successes are endlessly studied, but the causes of success and failure are actually buried in the stories of all those that failed and close their doors. But nobody collects that data, for a number of reasons, including the study of failure isn’t sexy and won’t sell magazines.

We know that we learn from mistakes and failures, yet the study of failure is never recorded or saved unless the company or individual “came back from the dead,” for example, Apple after Steve Jobs returned to lead the company from the abyss in 1996.

6 Comments on "The Unknown Unknowns And Survivor Bias"

BillT on Mon, 17th Jun 2013 12:24 pm 



Connections are often overlooked in success. Such as:

Family business connections: You have a better chance if your name is Rockefeller, Carnegie, Clinton, etc. It opens doors.

Top school credentials: Harvard, Yale, M.I.T., etc. Opens doors.

School connections: Famous or connected school chums to open those doors.

Family wealth: See first connections.

I.Q. over 100: Self explanatory.

All the not-as-good-as-their-parent siblings who get a chance because their parent(s) made it. Often found in the arts more than finances.

Luck … Luck … and More Luck!

(It also helps to have done your Homework before you try, then go for it! Failure is a learning experience.)

Jerry McManus on Mon, 17th Jun 2013 4:59 pm 



I have no data to back this up, but I suspect a large percentage of those small few who make it big did so not because they were looking to get rich quick, but because they were passionate about their craft.

Not sure about hedge fund managers, but how many times have you heard writers, artists and musicians say “I would be doing this anyway”? And of those how many toiled for years in obscurity before they were “discovered”?

How many of the so-called “failures” don’t really give a flip about that? They simply want to do what they love and they don’t care what anyone thinks about it.

Plantagenet on Mon, 17th Jun 2013 5:37 pm 



What BillT ascribes to luck is largely a matter of innate ability.

IQ is a heritable trait. Smart people have more ability. Smart people are better able to get into good schools, and more likely to do well in those schools. Smart people then go on to better careers and make more money.

dave thompson on Mon, 17th Jun 2013 8:24 pm 



We are a quantum field of opportunities. Success can be measured in many ways. If all you look at is monetary gain very few reach the uber rich no matter how much you might try. I look at my success at not having to worry about facing Mondays as a dread the utmost in achievements. Food, shelter, clothing,all taken care of for today and the foreseeable future………. PRICLESS!!!!!!!!!!!!!!

rollin on Mon, 17th Jun 2013 10:03 pm 



Failures are studied thoroughly by the military, engineers and scientists. Apparently in civil society, the norms of society prevent much examination of failure. Failure is a major sin in the society and is usually swept under the rug.

BillT on Tue, 18th Jun 2013 4:47 am 



Jerry, The really rich are there because they are ruthless and uncaring about anything else. Yes, that might be what they actually ‘like’ to do, but it is evil.