Apr

19

The confirmation bias is the tendency to search for or accept as valid information that is consistent with a prior belief and to exclude or reject information that challenges the prior belief.

As powerful a resource as the world wide web/ net is for doing research it seems that unless one is very careful this bias can impact the search words one uses thus strongly promoting the confirmation bias. Since the net has articles and posts that support virtually every point of view it is inevitable that point of view will strengthen over time as more and more articles are found that confirm the belief that biased the search.

Thoughts? 

Nick White responds:

I think your query is an excellent one. There will be other readers of the site of vastly greater eminence and skill than me in the field you have brought to our attention. However, I'll take a dilettante stab at responding by highlighting a coupe of good pieces of literature on the topics of biases / scientific method… and look forward to the honor of being corrected and informed by my betters. I'll also note that I'm not a professional scientist– merely of the armchair type, and a bad one at that. Secondly, I will cite quite a few names and sources in my response; these readings have been for personal pleasure rather than as part of a course where I've been goaded into critiquing them. In other words, my interpretations of their work might be a load of old tosh. That said….let's get on with it!

To me, your query goes to the heart of the philosophical foundations of the scientific method– and therefore difficult to answer either succinctly or with conviction. Happily, greater minds than our own have wrestled extensively with your topic and, I believe, have some useful answers for you. My aim is to present a thumbnail sketch of the thumbnail sketch of their work. I recognize that boring one's audience is the worst of solecisms so, at the risk of vast oversimplification, I will state my conclusion up front, lest anyone wish to proceed no further (likely a good idea): I believe it is fairly safe to say your answer lies with the tenets of "skeptical" empiricism. That is, one applies as best as possible the criterion of "falsifiability" in their work and research. The trick, as ever, is actual application.

Before we go on, though, I should like to define terms. You say that confirmation bias "is the tendency to search for or accept as valid information that is consistent with a prior belief and to exclude or reject information that challenges the prior belief". I'd like to demarcate that a little bit for our discussion. Richard Thaler summaries much when he writes that, "[belief perseverance means] people are reluctant to search for evidence that contradicts their beliefs. Second, even if they find such evidence they treat it with excessive skepticism. Some studies have found an even stronger effect, known as confirmation bias, whereby people misinterpret evidence that goes against their evidence as actually being in favor." In contradistinction, the behavioral literature seems to distinguish your definition as motivated reasoning, that is, "thinking biased to produce preferred conclusions and support strongly held opinions". Yet these biases themselves may simply be symptomatic of more problematic and pernicious dysfunction in our mental machinery. What could these biases be that "sum to" motivated reasoning?

A non-exhaustive inventory might contain at least five other biases in particular. First up is survivorship bias - that only the winners and survivors get to tell their story and present their data (irrespective of how one frames the enquiry to them). Next, we must account for Kahnemann / Tversky / Slovic's availability and anchoring biases. Availability is "the ease with which relevant instances come to mind". Anchoring is our propensity to estimate solutions with disproportionate reliance on– and influence from - the initial conditions. Fourth, we have the work pioneered by Kelley in the area of attribution, "man …infers causes for the effects he observes. The causes he attributes determine his view of his social world, and this view may determine his behaviour". Fifth, we have K/T's "errors of prediction" which, inter alia, states three principles. First, people rely too much on their "prior" intuitions when making assessments -even in the face of new, objective information. Second, people do not vary their predictions in line with the validity of the information on which their predictions are based and, fInally, people place more confidence in predictions based on highly correlated predictor variables than rational analysis affords them. The preceding doesn't even begin to touch on the personality dimensions of bias - that is, why our egos are constantly on the hunt to be "proved right", or the evolutionary ones - that biases are survival mechanisms!

So, given this diagnostic, what conclusions can we draw so far? It seems we can say that we're wired to confirm our hypotheses because it's convenient, it's fast and it's pleasant. As a result, we make judgment and inference errors that could be avoided if we had more robust methods to compensate for them. But why is confirmation "bad"? What's wrong with the proposition of reinforcing your beliefs and proving your hypotheses with more evidence of the same?This dilemma is not a new one. English polymath Francis Bacon highlighted the problem in the 17th century– and it has been a source of debate for much of the period since (incidentally, he was all for confirmation; though I put this down to his being a lawyer and the legal climate of his times). Today, the problem lies within the domains of philosophy; principally epistemology (what do we know, and how do we know that we know that we know it?) and the problem of induction; in other words, the realm of * falsifiability*. Sir Karl Popper (your principal, go-to guy on philosophical questions of method) argued forcefully that a hypothesis is not empirical -let alone scientific - unless it is falsifiable. What does this mean? For example, an unbroken string of sightings of white swans does not confirm the hypothesis that "all swans are white". But a single sighting of a black swan shows the hypothesis to be false. Conversely, claiming "there may be aliens in space" is not falsifiable. This is important because of the elusive nature of truth and certainty (at least this side of Heaven). One can never prove anything in this life with absolute certainty. All we have are probabilities. Once that notion is at the heart of one's scientific investigations, the door to statistical introspection swings wide open.From that point, at best, we can say with certain degrees of confidence that something "isn't" something else.

Ultimately, I can do no better than quote Sir Karl:

*Science is not a system of certain, or well-established, statements; nor is it a system which steadily advances towards a state of finality. Our science is not knowledge (epsiteme): it can never claim to have attained truth, or even a substitute for it, such as probability…

*We do not know: we can only guess. And our guesses are guided by theunscientific, the metaphysical faith in laws, in regularities which we can uncover - discover.

*But these marvelously imaginative and bold conjectures of ours arecarefully and soberly controlled by systematic tests. Once put forward, none of our anticipations are dogmatically upheld. Our method of research is not to defend them, in order to prove how right we were. On the contrary, we try to overthrow them. Using all the weapons of our logical, mathematical and technical armory, we try to prove that our anticipations were false– in order to put forward in their stead, new unjustified and unjustifiable anticipations, new ' rash and premature prejudices' as Bacon derisively called them.

*The advance of science is not due to the fact that more and more perceptual experiences accumulate in the course of time. Nor is it due to the fact that we are making better use of our senses….bold ideas, unjustified anticipations and speculative thought are our only means for interpreting nature…and we must hazard them to win our prize. Those among us who are unwilling to expose their ideas to the hazards of refutation do not take part in the scientific game.

Thus, I would contend alongside Popper (oh, how I deign!) that empiricism untempered by proper falsification might be many orders of magnitude worse than no empiricism at all. The safeguard you seek in your query is to, as widely as possible, practice rigour in all your habits and research - most importantly, the principle of falsification. Around these parts, the Chair et al have been known to throw out more than the occasional, "um, have you tested that?" to those making a particular claim of one sort or another. It's not for show.So, that's my thoughts on your query. We may avoid confirmation bias and the multiplication of its effects in the following (non-exhaustive) ways: Know which biases may impact your research. Run Popper's Logic of Scientific Discovery over your processes; especially the criterion of falsifiability. Will bad research persist? Sure. It has for centuries. Will it become ubiquitous? Not if science and scientists are doing their jobs properly.What practical steps can we take to effect these principles in our daily work and lives? Bischoff has provided some helpful guidelines: Know your cognitive frailties. Actively seek contrary evidence…force yourself to do it, or have a mentor well versed in creative destruction of your bad hypotheses. Put confidence estimates around the quality of information / data you have obtained. Educate yourself constantly (but don't rely on it too much - heavily discount your own smarts).

From Taleb: know in which domains you can safely apply induction (largely stable, natural phenomena) and which ones may get you into hot water (complex / contingent outcomes relying on inferences drawn from limited observations…because of the massive distortion and impact of rare events in the distribution of outcomes). Relentlessly build in redundancy to all you do and hypothesise. Be humble, and know that you know nothing– even on your best day. Visit graveyards and consider the untold stories of those who "didn't" make it -only survivors get to tell their stories with conviction and credibility. Consider alternative histories, and adopt the probabilistic mind-set. Apply the lessons you have learned from your textbooks to your whole life, not just the narrow, specialized context in which you learned them.I hope this sparks some good discussion! I'd be very interested to hear critiques, comments, additions etc. I'm attempting to stand on the shoulders of giants, so any misinterpretations, misquotes, misattributions or any other mis's are entirely my own.

Source / reference list:

- Kahnemann, Slovic, Tversky (eds): Judgment under Uncertainty: Heuristics and Biases, Cambridge

- Kahnemann, Tversky (eds): Choices, Values and Frames, Cambridge- Slovic et al: The Perception of Risk, Earthscan

- Taleb: The Black Swan, Penguin- Popper: The Logic of Scientific Discovery, Routledge

- Thaler: Advances in Behavioral Finance (Vol II), Princeton Publishing

- Peterson: Inside the Investor's Brain, Wiley- Forbes: Behavioural Finance

- Gauch: Scientific Method in Practice, Cambridge.

- Chamley: Rational Herds: Economic Models of Social Learning, Cambridge 

Pitt T. Maner III writes:

A quick overview of a few of the issues you have discussed is presented by James Montier in his latest book, The Little Book of Behavioral Investing– How Not to Be Your Own Worst Enemy. Problem examples in the book illustrate many behavioral traits that one can become susceptible to. Montier shows that one must be ever vigilant and self-aware of narrow-thinking, over-optimism, faulty statistical reasoning, over-conservatism, majority group thinking and assorted biases. One quote from the book reads, "Question authority, but don't accept the answer".

Montier's own soft spots, however, may be an over attachment to value investing and Graham techniques. More knowledgeable critics can decide.

Here is a two part interview with Montier related to the book.

Part 1
Part 2

A snippet from the 2nd part of the interview:

Miguel: Give us some insights – how can we become critical thinkers.

James Montier: Critical thinking is really all about being a contrarian in thought. Learning to be skeptical, to question what you hear, and evaluate it based on merit, rather than emotional appeal. In essence taking a contrarian view point requires us to learn three skills.

The first is highlighted by the legendary hedge fund manager Michael Steinhardt, who urged investors to have the courage to be different. He said, “The hardest thing over the years has been having the courage to go against the dominant wisdom of the time, to have a view that is at variance with the present consensus and bet that view.”

The second element is to be a critical thinker. As Joel Greenblatt has opined, “You can’t be a good value investor without being an independent thinker—you’re seeing valuations that the market is not appreciating. But it’s critical that you understand why the market isn’t seeing the value.”

Finally, you must have the perseverance and grit to stick to your principles. As Ben Graham noted, “If you believe that the value approach is inherently sound then devote yourself to that principle. Stick to it, and don’t be led astray by Wall Street’s fashions, illusions and its constant chase after the fast dollar. Let me emphasize that it does not take genius to be a successful value analyst, what it needs is, first, reasonably good intelligence; second, sound principles of operation; and third, and most important, firmness of character.”

Chris Cooper writes:

I recommend the blog called Overcoming Bias for discussion of many other biases and their application to real-world problems. You may also like Less Wrong.


Comments

Name

Email

Website

Speak your mind

3 Comments so far

  1. Diego Joachin on April 19, 2010 3:48 pm

    There is a collective knowledge issue you are ignoring.

    It’s like saying if the S&P 500 drops 8% in a single then this is about confirmation bias since you are still having your pay check at the end of month.

  2. Don Chu on April 20, 2010 7:49 pm

    Well said, Mr White. Just a few further extraneous points.

    Don’t forget Popper’s Falsification arose primarily in criticism to that particular influential school of thought at that time, the Logical Positivism of the Vienna Circle, and especially its empirical-rational criterion of meaning, Verificationism. Popper circled the Circle and took aim at what he considered to be their weakest point, verificationism in its ‘weak’ form, or if you like, its strong and naive form — Conventionalism. Popper dismissed the positivists’ psuedo-verificationism as incapable of truly verifying an inductive inference and thus failing the central question in the original Problem of Induction as articulated by the early empiricists as per Hume; and proposed his criterion of empirical falsification as the only true solution to the problem of induction.

    So, with that in mind, we can see that Mr Aronsen’s musings that confirmation bias “will strengthen over time”, essentially is an echo of the weak verificationalist-conventionalist process of reinforcing an ultimately flawed viewpoint with each additional positive example/sighting thereof. Which Mr White rightly responded to with Popper’s idea of empirical falsification.

    But here a little caution on the subject of Probabilities. The step from Popper’s empirical falsification to “probabilities” as “statistical introspection” is not as clear as it may seem. While all of us in the present generation and world may accept the frequentist/statistical inference process a la the Frequency Interpretation of Probability as a given, there are actually other flavours*. And in his time, Popper in fact questioned the validity of the frequentist interpretation and propounded his own Propensity Theory of Probability (primarily in response to nagging problems in quantum theory, towards providing a dispositional account of quantum states in the Schrodinger wave-function).

    One last comment: after reading Popper’s LSD, Taleb’s BS is probably not needed.

    [*For other flavours of probability: Bayesian as mentioned on this site I think, and check out Russell and Keynes.

    As for considering “alternative histories”, try the original purveyor: Michel Foucault and his Excluded Histories.
    ]

  3. dave whitesel on April 24, 2010 2:22 pm

    Very nice subject, and wonderful replies, Nick and Pitt.

    Diego…information is often Produced then planted for later view by the original issuer or proxies. Which sets off all sorts of alarms, that indeed confirmation bias can be simply bought and already paid for. Meeting terms of falsifiability…Si or NO?

    Then Don Chu, brings this “Popper in fact questioned the validity of the frequentist interpretation and propounded his own Propensity Theory of Probability (primarily in response to nagging problems in quantum theory, towards providing a dispositional account of quantum states in the Schrodinger wave-function). ”

    Which kinda sorta does obliterate many of our bias’ for probability outcomes.
    Have you ever read The Moon of Hoa Binh? By William Pensinger and Nah Trang (sp?)
    under the terms of a wave function, value is a measure of the capacity to integrate the subsystem-supersystem composite, within factors of relative-state, superintegration and overdetermination. Where value is within the observer, as an index of gathered awareness.

Archives

Resources & Links

Search