Survivorship Bias

It is a kind of selection bias, when we can observe only a part of the population because only this part survived a process or an event. For example, if we look at successful companies to study how to build a successful company we would miss not so successful and closed companies. So we wouldn’t take into account probably useful information about reasons which led to their lack of success.

This bias may appear more often than we think. Imagine an article about studying a foreign language. Quite often polyglots and people with good proficiency in languages write articles like this. Which means that these people may be more inclined to study languages than an average person. Which in turn means that the advice from the article may not work for many people. And that may be the problem with many articles like this.

Do you want to bake a cake? Do you have a good recipe? It may be written by a professional cook. Not so bad per se, but the cook may miss something important about the very basic stuff. Do you want to read a good magazine? Existing magazines may exist because sponsors want to support them and not because magazines are good. The really good magazines might be commercially disastrous, so they had closed a long time ago. You have got a recommendation about a book. Its popularity can be caused by chance, so other better books might exist. And so on.

Fortunately there are ways to avoid this bias. If you know about it, you can check any suspicious selection process. And you should check any selection process if you are going to use the results of this process. For example, when you want to study a foreign language, you may find some guidelines. Do these guidelines reflect the experience of people like you? Even if you think so you may want to double-check them before investing your time and other resources.

You can look for reviews and comments on these guidelines. If you have a book that contains a method of studying you can check its reviews. And look for negative reviews first, because they may have some useful impressions and experiences, while positive reviews often have praise without anything substantial.

In some areas you can look for scientific articles. For example the topic of studying foreign language may be a good candidate to check not only separate articles, but also meta-analysis articles.

To avoid this bias you need to look for ways to discover missing part of a selection. I don’t have a complete list of methods on how to do it, because it depends on a specific case. But I can advise you to understand how a certain selection is formed. Then you can understand how biased it is and start looking for missing members.