# The Improbability Principle, from David J. Hand

October 6, 2017 |

## Victor Niederhoffer writes to David Hand:

I am good friend of Steve Stigler and recently read and recommended your book. I came across an interesting coincidence in our mutual field. Every day I post a colored graph of 4 possible outcomes of directions of bond and stocks previous day. 11 of the last 16 occurrences have been yellow days with stocks up bonds down. The binomial prob of that is 1 in 10 million or so. I point out that events have to happen. And this is one of many billions of starting stopping pts and outcomes. Still it seems like an anomaly as I point out, the more important question is what does it portend for future. What's your view? Random or not?

## David Hands replies:

Hi Vic,

Thanks for recommending my book!

Can I first check the basis for your calculations. (I may have misunderstood what you meant.) If we take a simple model in which the probability of each of the four types of up/down pairs is equal, and the days are independent, then the probability of getting 11 out of 16 having (stocks up; bonds down) is Choose(16,11)*(0.25^11)*(0.25^5) = about 1 in 4000?

But you presumably chose (to comment on) the pattern (stocks up; bonds down) after having seen the data. So if instead we say what about the probability of any one of the four patterns coming up 11 out of the 16 times, then we have four times the probability. So, now it's 1 in 1000.

That sort of calculation would be ok if we simply had a set of 16 days to look at. But, of course, we are scanning across time. The longer we go on, the more we should expect apparently anomalous sequences to crop up. For example, we should ask not 'what is the probability of getting 11 out of 16 the same?' but 'what's the probability of getting 11 out of 16 consecutive days the same over the past 1000 (or however many) days?'

All the best

David

Professor David J. Hand Imperial College, London

## Pitt T. Maner III adds:

A 1 hour lecture by Prof. David Hand on this subject (2014) is available here.

I was watching Professor Hand's lecture and thought it amusing that he found himself in a situation where a man with his same name was staying at the same hotel at the same. This reminded me that at the University of Alabama about 39 years ago I had, if my memory is right, a Professor Hand for an advanced, introductory chemistry course who was a Harvard graduate. Ironically, the chemist Dr. Hand liked to grade on a curve and on his first test the grade for a "C" was 35% instead of the normal 70%! The first question on this first test involved multiplying/dividing two large numbers and determining the number of significant digits–this took about 15 minutes of the allotted 1 hour test time to do with a calculator but was only worth 5% out of the 100% perfect test score– such a tricky fellow. Now the professors get rated online by the students!

Name

Email

Website