The Drunkard’s Walk, from Duncan Coker

July 6, 2010 |

I am just finishing The Drunkards Walk by Leonard Mlodinow. I learned a few things. He gives a nice history of statistical thought highlighting that events are often more random than we might believe. It is always good to be critical of ones research.

There is an interesting chapter on conditional probability and how it is often misused by doctors and lawyers. For doctors by misinterpreting false positive or negative test results. Say one tests positive for X with 95 % confidence and an incidence rate of X disease of 1 per 100,000. Sounds bad, but in counting of those who test positive per 100,000 there will be 1 real case and 5000 false positives. So if you test positive there is a 99% chance you do not have the disease. In law there is a prosecutor fallacy. Here the prosecution presents a history of a spouse with domestic abuse in a murder trial to make their case. This is countered using deceptive statistics by the defense. The defense argues that only .04% go on to commit murder so this is very rare. Maybe true, but what the prosecutor should have said is that in cases where a spouse is subjected to domestic abuse and murderer, about 90% of the murders are committed by the domestic abuser.

There are examples from the investing industry showing how track records can be meaningless. In one example if there are a thousand investment managers over some 10 year period, the odds are over 75% that at least one will outperform the average every year over that period, by chance alone. Or the odds that good hitters in baseball might go on a streak and break hitting records by chance. The success of motion pictures is largely random, but credit or blame for a CEO is based on this. More likely is a reversion to the mean where the least successful studio over x years is the more likely to succeed in the future.

Towards the end of the book he discusses confirmation bias and how from a random series of numbers we often find patterns to confirm our pre conceived beliefs. I found this section very relevant to trading. Maybe a good way for me to avoid this is to assume my trading is completely random and look for evidence to the contrary. Or in a trading scenario to test the opposite of what you would expect as the Chair recently suggested. There are interesting studies showing a random series of X's and O's that seem to exhibit patterns.

The book is well written and full of other interesting examples like the Monty Hall example, and how a girls name can affect the odds of two girls in a family. The book also provides a laymen's history of some of the statistical pioneers like Pascal, Bayes, Bernoulli, Galton and others. Most important, I took away the importance of rigor and a critical eye toward my own statistical research and trading.

`SELECT * FROM wp_comments WHERE comment_post_ID = '4979' AND comment_approved = '1' ORDER BY comment_date`