When we do a study based on historical data and find a statistically significant result at the 5% level, we really are saying that there is less than a 5% chance that this study is completely attributable to chance. But if we observe some pattern in recent market action and then study it, that can be a problem: the multiple hypotheses problem.

One might think that if only one test is done that only one hypothesis was tested. Sometimes this is true. Other times traders will be intense students of the markets and notice a recurrent pattern. The trader then forms a hypothesis based on this pattern. It is properly tested on the most recent data and shows itself to be statistically significant.

There are two problems with this approach. First, if "the most recent data" include the same patterns that were observed and used to form the hypothesis then we are subject to the multiple hypothesis issue. This is true because that exquisite pattern-matching machine called the human mind continually looks for non-randomness and meaning in everything it sees. The mind tries out incredibly many hypotheses all the time. Most of us cannot even guess how many hypotheses our mind tries out before we identify one as interesting. So including the data, which formed the hypothesis, implicitly includes an element of multiple hypothesis testing.

The other problem is that we already know that the data will validate our study because it was used to help form the hypothesis. So it is not independent data but inherently biased. Thus our significance tests will be biased toward acceptance.

The best way to do these kinds of studies is to form the hypothesis on one data set and to test it on another completely different data set from another period.

Bruno Ombreux adds:

Or consider the same period but another market. For instance, if some phenomenon shows up in US stocks, test it on French and German stocks, too. There must be a reason for the putative phenomenon, either microstructural, behavioral, or economic. If so, it should show up in several markets. This extends the amount of testable data. One must be cautious with microstructure however, because it can differ. 

Philip J. McDonnell responds:

I do not agree with the idea of testing on data from different markets during the same time period, because many markets are highly correlated on a coterminal basis, sometimes as much as 90%. So it is really not an independent test on independent data.

But when one uses different time periods the correlations drop to near zero. So we can conclude that the data are truly out of sample.

Bruno Ombreux replies:

Dr. McDonnell is 100% right, but I still think it is not completely worthless to extend the sample to other markets. If you test a hypothesis on the US market, you'll be interested in the cases when you reject the null. Now, you test the German market and you still reject the null. You're right — not very useful. But if you fail to reject it on the German market, you need to come up with a very good explanation why it would work in the USA and not in Germany.

This is not nearly as good as different time periods, but it can be useful and increase understanding. 

Yishen Kuik adds:

I like to take an idea that has demonstrated its worthiness in actual trading in the US, then port it to other countries to see whether it works or not. If one has a group of countries for which the idea works and another for which it does not, it becomes interesting to try to figure out what members of each group have in common.

Nigel Davies remarks:

Presumably you're also taking account of time zones here. I've noticed that other markets tend to be led by the US during the day session (and even a couple of hours before its open) and have their measure of independence at other times. China is probably leading the overnight action now and Europe dominates during its morning. So perhaps it's not so much cultural as different time snapshots showing a certain similarity.

Martin Lindkvist extends:

Like the human flus that originate in Asia, many market ones seem to come from there too. Now, last night's Chinese flu seems to be of the same strain as that of late February. And as such, the market's immune system should be better prepared now. Perhaps a bit of coughing, and some sneezing for a little while, but not much of a fever this time? 

Henry Carstens adduces:

From a book recently recommended to me: "Routine design involves solving familiar problems, reusing large portions of prior solutions. Innovative design, on the other hand, involves finding novel solutions to unfamiliar problems." To borrow a quote from a friend, "Better necessarily means different." 





Speak your mind

3 Comments so far

  1. David Whitesel on April 20, 2007 12:37 pm

    Here is a very interesting quotation about time, with a little about the author afterword.

    “If you take a moment of time, this moment; it cuts through the entire physical universe as we’re talking. It holds all of space in itself. But one point of space doesn’t hold all of time. In other words, time is much bigger than space.

    A line of time is then an occurrence, and a wave of time is a recurrence. And then if you get out from the circle of time, which Nietzsche saw, the eternal recurrence –”

    Dr. Charles Muses — a mathematician, philosopher, and computer scientist who died in 2000 — was author of Destiny and Control in Human Systems and The Lion Path.

  2. SesDissefum on November 16, 2007 5:10 am

    Two new studies show why some people are more attractive for members of the opposite sex than others.

    The University of Florida, Florida State University found that physically attractive people almost instantly attract the attention of the interlocutor, sobesednitsy with them, literally, it is difficult to make eye. This conclusion was reached by a series of psychological experiments, which were determined by the people who believe in sending the first seconds after the acquaintance. Here, a curious feature: single, unmarried experimental preferred to look at the guys, beauty opposite sex, and family, people most often by representatives of their sex.

    The authors believe that this feature developed a behavior as a result of the evolution: a man trying to find a decent pair to acquire offspring. If this is resolved, he wondered potential rivals. Detailed information about this magazine will be published Journal of Personality and Social Psychology.

    In turn, a joint study of the Rockefeller University, Rockefeller University and Duke University, Duke University in North Carolina revealed that women are perceived differently by men smell. During experiments studied the perception of women one of the ingredients of male pheromone-androstenona smell, which is contained in urine or sweat.

    The results were startling: women are part of this repugnant odor, and the other part is very attractive, resembling the smell of vanilla, and the third group have not felt any smell. The authors argue that the reason is that the differences in the receptor responsible for the olfactory system, from different people are different.

    It has long been proven that mammals (including human) odor is one way of attracting the attention of representatives of the opposite sex. A detailed article about the journal Nature will publish.

  3. seoindex on June 6, 2008 1:56 pm

    I would like to see a continuation of the topic


Resources & Links