On the Invisibility of the
Nassim Nicholas Taleb
NASSIM NICHOLAS TALEB is an essayist and mathematical trader. He is interested in the epistemology of randomness and the multidisciplinary problems of uncertainty and knowledge, particularly in the large-impact hard-to-predict rare events ("Black Swans"). Taleb is the founder of Empirica LLC, a trading firm and risk research laboratory and Adjunct Professor at the Courant Institute of Mathematical Sciences of New York University.
Taleb held senior trading positions with trading houses in New York and London and operated as a floor trader before founding Empirica LLC. His degrees include an MBA from the Wharton School and a Ph.D. from the University of Paris. He is the author of Dynamic Hedging and Fooled by Randomness (2nd Ediition, published April 9th).
How Black Swans make their way out of history books. Methods that help you avoid drowning. The drowned not voting. Nietzsche delivering a special medicine. Casanova’s étoile. New York is invincible. Recurrence (Nietzsche again).
One of history’s principal jobs seems to bury, sometimes quite deeply, as many Black Swans as it can.
On the Effectiveness of Prayers
More than four centuries ago, the English essayist Francis Bacon had a very simple intuition. The idea is so trivial that he puts to shame almost all empirical thinkers who came after him until very recently. Even then, only a very small minority seems to get the point otherwise a book on Black Swans would have been much shorter. Bacon mentioned a man who, upon being shown the pictures of those worshipers who paid their vows then subsequently escaped shipwreck, wondered where were the pictures of those who happened to drown after their vows. The lack of effectiveness of their prayers did not seem to be taken into account by the supporters of the handy rewards of religious practice. “And such is the way of all superstition, whether in astrology, dreams, omens, divine judgments, or the like”, he wrote in his Novum Organum. That was written in 1620.
This is a potent insight: the drowned worshippers, being dead, do not advertise their experiences. They are invisible and will be missed by the casual observer who will be led to believe in miracles.
This is how superstitions (and much beliefs) are formed. Consider the following extension: I was thinking about calling my third cousin Antiochus this morning when the phone rang. Miracle! It was him on the other line! This confirms my developed sixth sense! This is a great omen except that perhaps I should wake up and take into account the number of times when I thought about calling him without his calling me; the times when he called me without my thinking about calling him; and, most significantly, the numerous occurrences of my not thinking about him, and him not trying to call me.
You just saw how the problem applies to the remembrances of your own thoughts: You cast away those that do not seem relevant, and only remember those that drew your attention. This distortion of the drowned worshippers hides among us in the most unexpected of places (as I am writing these lines I am surprised by my discovering , even after much of a lifetime of immersion in the topic, even more unsuspected applications). It pervades anything that may have a connection to the notion of history. By history, I don’t just mean those learned-but-dull books in the history section: It concerns the perception of events in your own life (your personal history) and how you infer things from experience. It extends to the ascription of factors in the success of ideas and religions, to the illusion of skills in many professions, to success in artistic occupations, to the nature v/s nurture debate, to the mistakes in using evidence in the court of law, to the illusion of the “logic” of history and of course, most severely, in our perception of the odds of a Black Swan.
Collecting The Wrong Stamps
You are in a classroom listening to someone self-important, dignified and ponderous (but dull), wearing a tweed jacket (white shirt, poker dots tie), pontificating for two hours on the theories of history. He throws at you very heavy artillery: Hegel, Fichte, Marx, Proudhon, Plato, Toybee, Spengler, Fukuyama, Schmukuyama, Trukuyama. He seems deep and knowledgeable, making sure that no attention lapse will make you forget that the approach is “Hegelian”, whatever that means. Then you realize that a large segment of what he is saying reposes on a simple optical illusion! But it will not make a difference: he is so invested in it that if you give him all the right arguments he would react by throwing even more names at you.
The point here is not yet that history may or may not have laws (we will have a bit on that in future chapters); the point is that our perception of the process hides significant parts in it. It looks like a law but it may not be one. What I will set to show in Chapter x is that much of the knowledge by historians is like stamp collecting: interesting, aesthetic, worth investigating, worth teaching, but not to be used beyond that as it can be dangerous. This chapter focuses on how the historian, by easily missing seeing the past black swans, may be collecting the wrong stamps! It is easy to omit to look at the cemetery while concocting historical theories.
We call this distortion a bias, i.e. the difference between what you see and what is there. This bias has been rediscovered here and there over the past century across disciplines, often to be rapidly forgotten (like Francis Bacon’s insight). It bears the name wrong reference class in the philosophy of probability, anthropic bias in physics, and survivorship bias in statistics. Drowned worshippers do not write histories of their experiences (one may need to be alive to do so); so it is with the losers in history, whether people or ideas. Remarkably, historians and thinkers in the humanities who need it the most do not seem to have a name for it (and I looked hard). As to journalists, fuhgetaboudit! They are industrial producers of the distortion.
The term bias also indicates the condition’s quantifiable nature: it may sometimes be easily corrected by taking into account both dead and livings in the assessment of the properties instead of just looking at the living.
The manifestations are even rampant in things that may not appear to have history in them (yet your knowledge of them is in some manner historical). Did you ever go to the local YMCA pool with the hope of acquiring a swimmer’s body? Did you ever switch lanes while driving on a highway under the impression that the one next to you is moving faster than yours[i]?
Sir Francis, Egomaniac and Skeptic
Sir Francis merits a word here; he is an interesting and endearing fellow in more than one respect. First, he harbored a deep-seated, skeptical, nonacademic, anti-dogmatic, and obsessively empirical, nature which, to someone skeptical, nonacademic, anti-dogmatic, and obsessively empirical, like this author, is a quality almost impossible to find in the thinking business. (Anyone can be skeptical; any scientist can be overly empirical; it is the rigor coming from the combination of skepticism and empiricism that’s hard to find). Furthermore, even rarer, Bacon offered the combination of an independent thinker and a man of deeds; he was Lord Chancellor under Queen Elisabeth, the combination of the highest governmental rank with that of chief justice. He even went as far as considering the academia an impediment to learning; he specified the idola theatri (idols of the classroom[ii]) as an inhibition to knowledge. Indeed he fought the doctors sitting and “thinking” in scholastic bliss he saw little difference between scholarly knowledge and superstition unless there was some empirical validation and thus created the modern scientific method.
Second, he was said to be of an immodest nature (he considered that all of knowledge was his province, and his own private province at that) egomania can have some charm. He did not aim too low: his Novum Organum was meant to be a replacement to Aristotle’s magisterial Organon (consider that there were not that many books to refer to during the Middle Ages). Indeed there is this romantic charm of the gutsy skeptic who starts out by taking on the big guns of the day. As we will see in Chapter x, Karl Popper took on Plato (and of course the more minor figure of Hegel); Bacon and Al Ghazali took on the biggest gun of all, Aristotle; and Charles Sanders Peirce took on everyone before him (including Aristotle).
Third, Bacon was imprisoned for taking a bribe from a defendant in a trial, a very modern attribute yet it is mentioned everywhere as one of his characteristics as if it had anything to do with the quality and validity of his thought. Indeed, he had to be quite a potent thinker for Bertrand Russell’s A History of Western Philosophy gives him ample coverage yet hardly contains a single favorable sentence about him and his ideas.
It is most ironic that while I open my examination of the causes of Black Swans with Bacon’s intuition of the invisibility of the drowned worshippers, his introduction of excessive and naive empiricism may be the major cause of the underestimation of large unexpected deviations. It feels quite incongruous to have to convince the reader of an intellectual hazard by quoting insights from the main perpetrator. In fact we will see that the whole metaphor of Black Swans was developed in response to inadequacies in his theories, what is well known as the Baconian flaw of deriving general laws from finite set of observations. Confident knowledge without empirical validity will cause the underestimation of errors (that was what Bacon attacked); likewise would naive generalizations from observation alone (which is what he got us to do). Consider the situation before Sir Francis: Aristotle held that women had fewer teeth than men. Nobody would have dared to challenge Aristotle’s authority “who are you to question Aristotle?”; the instinct to seek empirical validity was stifled under the weight of scholasticism. Bacon, with his big ego, came and said: look at the world around you; examine the data, just look, look, look. The problem is that his method led us to observe, something good, but derive general conclusions from observed data, something that can be dangerous. You see a number of white swans and become comfortable with the general statement that all swans are white. So long as people develop confidence with that they know, no matter how they got to know it, whether from Aristotle or from their observations, there will be Black Swans.
The rest of this chapter will go deeper and deeper into the extensions of this bias of the drowned worshippers. Its focus will be on how our perception of history causes us to see fewer outliers that there actually were, causing illusions of stability. I discuss in the next chapter how my encountering a version of his bias got me settled into my choice of career of professional Black Swan specialist and came at the right time to save me from duller pursuits. The range from the benign to the malignant. I call them: a) Nietzsche’s Error (where we think that something “helps” when it may not), b) The swimmer’s body (instances where we get the causal effect backward), c) the Giraffe’s rubber neck ( the cause behind the confusion between Lamarckian and Darwinian evolution) and d) Casanova’s misperception (the most unsettling of all: how we develop illusions of stability).
A Health Club for Rats
When I was in my early twenties and still read the newspapers (though never steadily), I came across an article discussing the mounting threat of the Russian Mafia in the United States and its displacement of the traditional Louie and Tony in some neighborhoods of Brooklyn. It explained their toughness and brutality by their being “hardened by their Gulag experiences”. The Gulag were labor camps in Siberia where criminals and dissidents were routinely deported. Sending people to Siberia was one of the purification methods initially used by the Czarist regimes and later continued and perfected by the Soviets. Many deportees did not survive these labor camps.
Hardened by Gulag? The sentence jutted at me as both profoundly flawed and widely accepted by conventional wisdom as being a reasonable inference of the kind intelligent and educated people generally make. Yet I could not for the life of me consciously figure out exactly why it bothered me. It sounded right and reasonable, yet it still bothered me. I never suspected then how much mileage I would get out of it.
There is nonsense in it hidden under a cosmetic wrapping. The following thought experiment will show it. Assume that you’re able to find a large assorted population of rats; fat, thin, sickly, strong, well proportioned, etc. (You can easily get them from the kitchens of fancy New York restaurants). With these thousands of rats, you build a heterogeneous cohort, one that is well representative of the general New York rat population. You bring them to my laboratory on East 59th Street in New York City and we put the entire collection in a large vat. We subject the rats to increasingly higher levels of radiation (since this is a supposed to be a thought experiment there is no cruelty in the process). At every level of radiation, those who are naturally stronger (and that is the key) will survive; the dead will drop of your sample. We will progressively have a stronger and stronger collection of rats. Note the following central fact: every single rat, including the strong ones, will be weaker after the radiation than before.
An observer endowed with analytical abilities who probably got excellent grades in college would be led to believe that treatment in my laboratory is an excellent health-club replacement and one that could be generalized to all mammals. Their logic would run as follows: Hey, these rats are stronger than the rest of the rat population. What do they seem to have in common? They all came from the Taleb workshop.
Next we pull the following trick on The New York Times: We let these surviving rats loose in New York City and inform the Chief Rodent Correspondent of the newsworthy disruption of the pecking order in the New York rat population. He will write a lengthy (and analytical) article on the social dynamics of New York rats that includes the following passage: “those rats are now bullies in the rat population. They literally run the show. Strengthened by their experience in the laboratory of the reclusive (but friendly) scientist-philosopher-speculator Dr. Taleb, they...”
Literature can be quite detrimental to your health, particularly when someone invested with literary authority makes a medical statement, a well sounding one, so well sounding that it automatically endows him with medical competence, and his statement becomes quoted in all manner of circumstances. As I mentioned with the availability heuristic, the mechanism of the soundbite (in the media) and metaphor (in more respected texts) does not just vitiate your inference, but does so in a well known manner. It just augments the perception of causality.
Say you are struck with a high fever of unknown origin, with worrisome attributes. I bring to your attention the works of one Herr Doktor Professor Hans Schmidt who was an eminent professor of internal medicine at the University of Basel between 1869 and 1898 and specialized in fevers of unknown origin. No matter how eminent I will describe him you will think that by directing you to his research I am making a particularly silly joke. Yet his Treatise on Special Fevers has been so influential! Meanwhile you may accept a medical quote by another resident of Basle between 1869 an 1898, an eminent professor of Greek. Medical advice from the 19th Century?
Just consider that the eminent professor of Greek, the German poetic philosopher Friedrich Nietzsche wrote the following aphorism: “what does not kill me makes me stronger”[iii]. (Actually Nietzsche may be neither poetic nor philosophical as he is usually considered a philosopher by poets and a poet by philosophers). This statement is quite hackneyed, so hackneyed that there are some situations where it is almost impossible to not hear it mentioned. It is cited by those who undergo difficulties in life and find consolation in the promise that they will get something out of their trials, such as strength of character and the ability to overcome future trials. It is also part of the popular belief that stresses cause organisms to compensate by getting stronger, which may work in a limited way with what is called hypertrophy. This is after all how muscles grow: they tear down and get stronger upon their rebuilding. The organism toughens up to adapt to the stresses it is submitted to. It is the reason people are supposed to go to the gym, children are expected to do math exercises, and older people are recommended to play bridge. Companies that survive trials get better; they become “more competitive”. Commuting, I was told, “builds character” (it doesn’t on that, later).
By some coincidence I was around that Gulag epiphany period in the middle of reading Gibbons and stumbled on a statement in his magisterial work to the exact opposite effect: he used the antonym mollified to explain the Romans’ gradual decline. With his characteristic elegance, he wrote:
If the decline of the Roman empire was hastened by the conversion of Constantine, his victorious religion broke the violence of the fall, and mollified the ferocious temper of the conquerors.
Now according to two writers, Christianity mollifies and Gulag hardens our human race.
The Memory of Rats
Three quick points to sum up the rat experiment. First, we just pushed one step beyond the drowned worshippers problem. The worshippers did not drown because of their prayers; these were in all likelihood ineffectual, while here, with the Gulag and the rat experiment, we have a perception of a benefit when in fact we were subjected to harm. As with the worshippers, deceased rats become invisible after their funeral and do not run around the place eager to show you their attributes; only living rats trumpet theirs. Out from sight, out from mind; they are theoretical constructs that may be easy to forget about or miss seeing altogether. It is not jut us who scorn what we don’t see, but much of the inferential methods in use by scientists are built on the observable (and observed) not the nonobservable (hence abstract). It is difficult to look at the composition of the cemetery, at the ideas that failed, at the things that you forget about because you do not notice them. Note that here the attributes of the dead are completely different from those of the living (they were weaker to start with). That difference we will call observational bias; as we said, it is easy (sometimes very easy) to quantify. How? It matters to get the attributes of the base cohort! Just subject all the rats to a thorough physical examination before the treatment and look at the relation between initial health and survival. This is a mistake that professionals tend to readily make. While it has been corrected in medicine (experiments use a base cohort and refer to it) and other experimental sciences the equivalent of the missing rats went completely missing from the social sciences analyses such as economics and sociology.
Second, what we just built in the laboratory on 59th Street is a full-fledged, albeit simplified, simulator of history. We put an initial population through a “historical process”; we could have possibly created a multitude of historical processes (one per rat). We observed what came out in relation to what went in, instead of just acting as spectators and selecting causes to it from the most visible elements. We know the cause because we are the cause. This is the difference between forward and backward causation. We can assess the situation from two perspectives: the historian (or journalist) and the experimenter. That contrast will be the subject of chapter x; it is at the core of the difference between what we will call soft and hard knowledge.
Third, and perhaps the most worrisome aspect of all, human nature prevents us from seeing the point. By a mental mechanism called the availability heuristic, which we will discuss at length in chapter x, humans are designed to heed what they see and not heed what does not vividly come to mind. We may have a blind spot with respect to the cemetery; even those who might be walked through the reasoning will often unconsciously slip into considering that the rats were made stronger in my laboratory. Why? because the unconscious part of our inferential mechanism (and there is one) will ignore the cemetery, even if we know intellectually of the need to take it into account. Again, scorn of the abstract. We will dive into it in our later discussion on cognitive biases.
There is a vicious attribute to the bias: it can hide best when its impact is the largest. Owing to the invisibility of the dead rats, the more lethal the risks, the less visible they will be. Consider that the more injurious the treatment the larger the surviving rats and the more fooled you will be about the strengthening part of the story. A necessary ingredient for this difference between the true effect (weakening) and the observed one (strengthening) to exist is either a) a disparity, or variance, in the base cohort or b) randomness, or variance, somewhere in the treatment. Variance here has to do with the degree of uncertainty inherent to the process. I will explain the effect of the variance in the base cohort with the following arguments. Consider that if all the rats were of the same strength then the bias would not have existed; they would have been all dead or all alive. As to the variance in the treatment: If you pick thousands of identical rats, you would get a difference in the end result by randomly changing the radiation treatment for each one of them. You would then only see the winners and become confused as to the properties of the losers. Variance in the base population or in the treatment is critical here. I will keep returning to the point throughout the entire book.
Why is Nietzsche’s aphorism dangerous? Because there are situations where your own past experience may be irrelevant, entirely irrelevant for any purpose. How can someone’s experience be irrelevant? Why is the knowledge about your own history something to avoid sometimes? How can information be worse than no information? Clearly if you are not capable of interpreting it and get a biased message from it, you are worse off without experience than with one. Consider the feeling of indestructibility it confers, the exhilaration of the agent led to believe that he can survive the trials because his personal experiences revealed to him an intrinsic property when in fact he may have just survived because of luck.
The Swimmers Body
What do the popular expressions of the “swimmer’s body” and the “beginner’s luck” have in common? What do they seem to share with the concept of history?
There is a belief among gamblers that beginners are almost always lucky. “It gets worse later, but gamblers are always lucky when they start” is something you hear. This statement is actually empirically true: A researcher would confirm that gamblers had lucky beginnings (the same applies to speculators). Does it mean that each one of us should become a gambler for a while until he stops being a beginner?
The answer is no. The same optical illusion prevails: those who start gambling will be lucky and unlucky, say in equal numbers (actually given that the casino has the advantage, slightly more people will be unlucky than lucky). The lucky ones, under the feeling of having been selected by destiny, will go on to continue gambling; the others, discouraged, will stop and will not show up in the sample. They will probably take up, depending on their temperament, bird watching, scrabble, or other pastimes. Those who continue with their gambling habit will remember having been lucky as beginners. The dropouts, by definition will no longer be part of the gamblers community. This explains the beginner’s luck.
Look at the analogy with what is called in the common parlance a swimmer’s body, a mistake I shamefully fell for a few years ago (in spite of my specialty in this bias; I did not notice that I was being fooled). When asking around about the comparative physical elegance of athletes, I was often told that runners were anorexic, cyclists bottom heavy, and weight lifters insecure and bulging, a little primitive. I inferred that I should spent time inhaling chlorine in the New York University pool to get those “elongated muscles”. Now consider it backwards. Assume that people's genetic variance allows for all type of body shapes. Those born with a natural tendency to develop a swimmer's body become better swimmers. These are the ones you see in your sample going up and down the pools. But they would have looked the same if they lifted weight. It is a fact that a given muscle grow exactly the same way whether you take steroids or climb walls at the local gym. I was fooled by my own specialty!
There have been numerous studies of millionaires and their traits, with signs of success from these. They look at what those hotshots have in common: chutzpah, risk-taking, optimism, etc. and consider that these traits, namely risk-taking help you become successful. Had they looked at the cemetery (though people who fail do not seem to write biographies) they would have figured out that the same applies to the population of losers. The entire mechanism of biographies, aside from the arbitrary ascription of a causal relation between specified traits and subsequent events, seems to perpetuate the bias owing to the simple fact that people do not write the life events of failures.
Lengthened by Rubbernecking
We will discuss the mechanics of evolution a little later in this book, but a point worth considering here. The bias easily explains much of the misunderstanding of evolution that plagued us for millennia. Evolution is a historical process fraught with uncertainty and our perception of its workings is subjected to the same bias. For a long time we held the belief that giraffes developed a longer neck by dint of stretching it (the local journalist would call it “lengthened by rubbernecking”) and that they transmitted the elongated neck to their progeny. In other words, the environment acts directly upon the organism as a mechanism of adaptation, just like with the hardened by Gulag or Nietzsche’s aphorism. This is called the Lamarckian view on the acquisition of traits and their inheritance, after its formalization by the 19th Century biologist Jean-Baptiste Lamarck: people saw giraffes with tall necks and for more than twenty centuries assumed that reaching out for branches elongates one’s neck. The Darwinian thesis is similar to our rat experiment: that the giraffes with shorter necks escape our analysis because they drop out of the gene pool as they do not tend to leave descendants[iv].
What we did with the rat experiment is simulate a survival of the fittest. If instead of being impatient we let the rats reproduce we would have seen second and third and fourth generations of effectively stronger animals. This confusion between epigenetic and genetic in the transfer of the information from the environment across generations is quite endemic. Remarkably it comes from the same observational bias.
Casanova’s Unfailing Luck
This brings us to the gravest of all, the illusion of stability and one that is my personal specialty.
The adventurer Giaccomo Casanova, later self-styled Chevalier Jacques de Seingalt, the wannabe scholar and legendary seducer of women, seemed to have a Teflon-style trait that would cause envy on the part of the most resilient of Mafia dons: Misfortune did not stick to him. Casanova is not known for his seductions so much as from his ten-volume Memoirs of My Life written in bad, sometimes extremely bad, French.[v] Aside from the lessons in how to be a seducer, the Memoirs provides an engrossing account of a life of successions of dips and bouncing back. Casanova felt that every time that he got into difficulties, his lucky star, his étoile, was watching after him and managed to pull him out of trouble. After things got bad for him, matters somehow recovered by some invisible hand and he was led to believe that it was an intrinsic property of Giaccomo Casanova to recover from hardships by running every time into a new opportunity. He somehow met in extremis someone who offered him a financial transaction, a new patron that he has not betrayed or someone generous enough and with weak enough a memory to forget past betrayals. Could Casanova have been selected by destiny to bounce back from all hardships?
Not really. Consider the following: of all the colorful adventurers who have lived in our planet, many will be crushed occasionally, many will bounce back repeatedly. It is those who survive who will tend to believe that they are indestructible; they will have long enough an experience to write books about it. Until, of course...
Actually these adventurers who feel singled out by destiny abound, simply because there are plenty of adventurers, and we do not hear the stories of those who happened to be down on their luck. As I started writing this chapter, I recalled a conversation with a woman about her flamboyant fiancé who, the son of a schoolteacher, managed through a few financial transactions to catapult himself into the life of the character of a novel, with handmade shoes, collectible cars, etc. The French have a word for it, flambeur, which is the mixture of extravagant bon-vivant, wild speculator and risk taker, all the while bearing considerable personal charm; a word which does not seem to be available in Anglo-Saxon cultures. He was spending his money very quickly and as we were having the conversation about his fate (she was going to marry him after all), she explained to me that he was undergoing slightly difficult times, but that there was no need to worry since he always came back with a vengeance. That was a few years ago. Out of interest, I just tracked him down (trying to do so tactfully): he did not recover from his latest blow of fortune. He also dropped out of the scene and can no longer be seen among other flambeurs.
How does this relate to the dynamics of history? Consider what is generally called the resilience of New York City. For seemingly transcendental reasons, every time it got close tot the brink of disaster, it managed to pull back and recover. Some people truly believe that it is an internal property of New York City, i.e., replace the “managed” in my previous sentence by “manages”. The following quote is from a New York Times article (found on Arts and Letters Daily):
Which is why New York still needs Samuel M. Ehrenhalt. An economist who turns 77 today, Mr. Ehrenhalt studied New York City through half a century of booms and busts, until he retired as regional commissioner for the federal Bureau of Labor Statistics in 1995 (...)
"We have a record of going through tough times and coming back stronger then ever," he said.
Run the idea in reverse: think of cities as little Giaccomo Casanovas or rats in my laboratory. Just as we put the thousands of rats through a very dangerous historical process, put a collection of cities in a simulator of history: Rome, Athens, Carthage, Byzantium, Tyre, Catal Uyuk (the first known human settlement in modern day Turkey), Jericho, Preoria, and, of course, New York City. Some cities will survive the harsh conditions of the simulator. As to others, we know that history might not be too kind. I am sure that Carthage, Tyre, and Jericho had their local, no less eloquent, Samuel M. Ehrenhalts, saying: “Our enemies have tried to destroy us many times; but we always came back more resistant than before”.
This bias causes the survivor to be an unqualified witness of the process. Unsettling? The fact that you survived is a condition that may weaken your interpretation of the properties of the survival, including the notion of “cause”.
You can do so much with this statement. Replace retired economist Samuel Ehrenhalt with the chairman of a company discussing his corporation’s ability to recover from past problems. How about discussing the “resilience of the financial system”? How about a general who had a good run? But things can get worse.
I take more random examples from the press (borrowed again from Arts & Letters Daily). In an article Stability, America's Enemy, by one Ralph Peters [vi](never heard of him), has for thesis that stability is not necessarily good for America, as the U.S. benefited from past turmoil and he supplies seemingly convincing well-written arguments. Problem: 1) if these events did not turn out to yield to stability, he would not be writing the article; 2) It would suffice to have one single event leading to instability to invalidate the many that lead to stability --simple properties of disorder. Conclusion: all he could say is that the U.S. benefited from past turmoil that subsequently lead to a period of stability, a mere tautology.
[THE REMAINING SECTIONS WILL BE SIMPLIFIED]
I wanted to stay closer to earth with this book without having to bring higher-up metaphysical or cosmological arguments to this discussion. But a hint of the higher up extension might be necessary.
A recent wave of philosophers and physicists have been studying what is called the self-sampling assumption, which is a generalization of the principle to our own existence. Consider our own fates. The odds of any of us being in existence are so low that only a creator or some higher up design could have explained it. Thinks of the probability of the parameters being right there to induce our existence. So either it has to be an event of massively low probability (so low that I can hardly explain it here, say one in several trillion trillion trillion ... and keep going) or it had to be designed which seems to provide a much more credible explanation. It cannot be an accident.
Not so... The odds of such accident of fate are grossly underestimated from such perspective. Think again of all the possible worlds as little Casanovas following their own fate. The one that is still alive by accident will feel that given that he cannot be so lucky there has to be some transcendental force guiding him and supervising his destiny. Otherwise the odds would be too low. For the surviving Casanova, the odds will seem low. For someone who observes all adventurers, the odds of having a Casanova are not low at all: there so many of them and someone is bound to win the lottery ticket.
This induces a bias: don’t compute the odds from Casanova’s perspective, but from that of the pre-observer. Likewise can no longer compute probabilities without wondering that the conditions that we are in existence imposes restrictions on the process that led us here: it had to become from the rosier realizations of the process, the ones the most favorable for our existence.
This in itself greatly weakens the notion of “because”. Recall that we are explanation-seeking animal and tend to think that everything has an identifiable cause and grab it mentally. Yet there may not be a because, to the contrary, frequently a nothing, not even a spectrum of possible explanations. Here, whenever our survival is in play, the notion of because is weakened. To see how the mind needs causation and cannot really grasp these non-explanations (and how journalism falls for the fallacies of necessary explanation) I give an excerpt from a book review by the journalist Nicholas Fearn reviewing of the last book by the late philosopher Robert Nozik's[vii] Invariances: The Structure of the Objective World
Thus our universe might not be the best of all possible worlds, but it is at least one of the fittest. If the laws of its parents were not stable enough for them to mature to "reproductive age" (whereupon they produce black holes), the world in which we live would not exist. This may not be the kind of answer some people were looking for. It recalls the notorious attempt to account for the existence of human beings by saying that if the laws of nature were not such as to allow for our arrival on the scene, we would not be here to ask the question. This is like someone asking how they survived a car crash and being told that if they had died, they would not be around to examine their good fortune.
Isn’t it interesting how we mistake the notion of weakened explanation for reverse explanation? I am bringing the article by the journalist because it illustrates exactly the problem of philosophy: philosophy is asking us to question the very core of the because. Ironically Fearn is the author of a philosophy-for-dummies type of book.
There is the toxicity of the category of people in search of explanations. They cannot get the point that invoking the anthopic bias is not claiming causality, but quite the opposite. I did not survive the car crash because I am here today, but the fact that I am here provides a bad conditioning to the inference. It shows the lameness of looking for causality and explanation as a necessary interpretation.Besides everyone seems to want an answer. A book (this one) that asks questions is never considered a contribution.
Another question: why didn't the plague kill more people? Backward causality: had it killed more people the observer (us) would not be here to observe. So it may not be necessarily the property of diseases to spare us humans.
More Extensions (Incomplete)
Let us go back to Giaccomo Casanova’s unfailing luck as a framework that applies to a generalized framework of scientific method for the analysis of history. We will even call this computational epistemology as it can be done by computer (it is more fun and more precise than listening to boring, slow-talking philosophers with their noses glued to the lecture notes). You generate artificial histories, say millions of Giaccomo Casanovas, and watch the difference between the attributes of what you generated (since those you know the exact properties) and those an observer of the result would obtain. Such difference will be the exact bias.
First result: the observed Casanovas will fare better (since the others will drop out of the range of visibility). Second, the observed Casanovas will seem to have less volatile lives (we do not observe the dead). Finally: the observed Casanovas will exhibit a pattern of recurrence: whenever things go bad they will tend to come back from the brink. Why? Because those who subsequently fared poorly dropped out of the sample. You may do poorly. Those who did poorly then had a comeback will show up in the sample again. Others will be gone.
The engine of computational epistemology I used in finance to debunk some of the attributes propounded by Wall Street. The fund management industry claims that some people are heroes and extremely skilled since they have year after year outperformed the market. They will identify these “geniuses”. My approach has been to manufacture cohorts of purely random investors and show how it would be impossible to not have their geniuses just by luck since we are not observing the cemetery and missing out on the failed investors. Of course an explanation will be readily provided for the success of the lucky survivor by the mechanism of retrospective determinism.
Back to Nietzsche (there will be more visits later). Somewhere he wrote about the idea of eternal recurrence, the constant repetition of history ad infinitum. If you extend time a process examined by Poincaré as the recurrence theorem; if you wait an infinite amount of time by the side of the road a random event might inflate your tires! We will examine it again with my paradigm of the Library of Babel, but the notion is extremely common in the perception of historical events. It has been used by the historian of religion Mircea Eliade in opposition to the prevailing view promoted by Hegel that history had a course, followed its own logic, and moved by stages, each an improvement over the previous one. Basically the split is between ideas of history following a pattern of reversion, the other that it follows a pattern of trend. I set to prove, using the bias of this chapter, that both parties are fully wrong.
Benign Cases Of The Survivorship Bias
Note that this bias is generally mistaken for its exact opposite: Darwinian fitness. At a conference I saw a presentation by a colleague discussing his methods in picking fresh talent. He believed he could find operators who had the ability to survive a hurdle, meaning that he believes that he can find persistent "winners". What I call survivorship bias is the discussion of traits in a cohort that are no different from those in the general population, but who became successful out of randomness.
[i] See the wonderful and comprehensive discussion in Bostrom (2002).
[ii] Theatri is generally translated by theaterit refers to the classroom, or more generally, to formalism.
[iii] "Aus der Kriegsschule des Lebens. - Was mich nicht umbringt, macht mich stärker." Translation: "Out of life’s school of war. What does not destroy me, makes me stronger." Note: Often quoted as "That which does not kill me, makes me stronger". Source: Götzen-Dämmerung (Twilight of the Idols), Aphorism 8.
[iv] The following episode might illustrated how deep-seated and prevalent the mental bias. One April evening, the novelist Salman Rushdie sat on the throne at the 92nd Street Y and discussed, as part of the Paris Review Writers at Work series, his career and recent book. Rushdie is a man of immense personal charm and a seductive and animated speaker, which explains why females mob him at parties (they actually mobbed him after that discussion). By some heuristic his presentation skills make his statements extremely convincing. At some point he discussed the fictional character Malik Sowanka, the protagonist of his then latest novel, Fury. He gave a lengthy exposition of the determining events in Sowanka’s childhood “that shaped his character”. Events, determining events that shaped what? Sowanka’s character? How do you know that these types of events are the thing that shape one’s characters? Could it be that he was predetermined genetically to have a given temperament and these events happened to coincide with a given disposition? Everybody finds it obvious today, after the virulent nature-nurture debates, that events don’t necessarily shape one’s character. They might, but they do not necessarily do so. Many people would tell you that the mothers of autistic children were until recently made to feel guilty by psychologists who accused them of being “refrigerator mums”these children would have been no less autistic with an “oven mum”.
So the Rushdie statement appears obviously wrong and few outside leftovers from the 1960s and 1070s (and those who still write conventional biographies) would pay attention to it. Yet by some mechanisms those who can readily identify the flaw of unconditionally imparting the role the environment make the dead rat mistake. It is difficult to shed our predilection for the concrete.
[v] . He simply fell in intense love, although transitory, with each of his conquests. Casanova seemed undiscriminating and extremely democratic in his mate selection, given the range of his conquests. He did not look at a women as a whole, but focused rather on an attractive part in her.
[vi] Stability, America's Enemy, by Ralph Peters, PARAMETERS, Winter 2001-02
[vii] in The New Statesman (Feb 11 2002).