Daily Speculations The Web Site of Victor Niederhoffer & Laurel Kenner Dedicated to the scientific method, free markets, deflating ballyhoo, creating value, and laughter;  a forum for us to use our meager abilities to make the world of specinvestments a better place. Home Write to us at: (address is not clickable)

Dr. Alex Castaldo

5/12/2005
Backtesting When We Don't Know the Direction of the Expected Effect

A useful fact to know is that if a random variable is distributed as N(0, sigma) the expectation of its absolute value is sigma*sqrt(2/pi) or approximately 0.798*sigma.

The latest application I came across is as follows. Let R(i) be the daily returns (or daily changes) adjusted for long term drift. The R(i) have a zero mean by construction and a standard deviation sigma (approx 1% for the S&P 500 for example).

Suppose a decision rule selects N days as a subsample of interest. If the rule is no better than random, the average of the chosen R(i) will have expected value zero and standard deviation (approximately) sigma/sqrt(N). [I say approximately because I am neglecting a finite sample correction we might want to make]. In practice this sum S will not be exactly zero; if it turns out to be positive, we can say We have discovered a decision rule for going long that yields S, if it turns out to be negative we can say We have discovered a decision rule for going short that yields S. This is the way the result will inevitably be presented: in the most favorable light.

As a result, even for a random rule, we can expect to see a return 0.798*sigma/sqrt(N) and we should accept a rule as useful (nonrandom) only if it exceeds this value by a sufficient margin. (The margin being expressed as a certain number of standard deviations).