Any reader who has not looked at a price chart in the past 90 days please stand up and identify yourself. For that person and that person alone can cast a stone (at technical analysis).
Gary Phillips writes:
I look at charts all the time, but that's really not the point. For someone who is as truly blessed with the ability to determine causality as yourself, you must realize that charts are not predictive in of themselves.
Larry Williams writes:
Parts of charts are most definitely predictive. Patterns repeat. And I agree that so much of TA is misleading and based on whims and fancy yet there are parts that really do work.
Ah, the chart debate has returned.
While surely an example of survivor bias, I have witnessed industry greats use charts and technical analysis as part of their speculative arsenal. Of more interest is that these people used their own personally derived versions of these methods and not the versions available at no cost to everyone. I dare say that the creators of well known indicators have ways of using them that they would never reveal (rightly so!).
A few points about charts:
1. At the higher frequency end, in the OTC macro markets, ALL of the chart services are wrong and ALL of the chart services are correct. Each has its own price, so there is no 'right'. This probably doesn't matter to most and doesn't fatally damage the pro chart school.
2. Some market extremes are written out of history for various reasons (regulatory, legal, error, political correctness and vested interest). The move toward full electronic trading might alleviate some of these in future.
3. Commodity prices on charts…. Should we adjust them by inflation? What are we actually looking at? What are we comparing.
4. Equally spaced data? What to do with price action measured in equal intervals (say, for example, 5 minute charts) when the price doesn't change during the period but the recording software has to put a number in there so it averages, uses the last price, the first price of the next period etc…
5. There is a reason why the big quant firms have interesting individuals whose life's passion is ensuring data is clean/ accurate.
6. It is probably a fair point to state that the recording of price information has improved since, say, the 1970's. The tricks now are more to do with latency of its delivery and the subtle recursive methods some providers appear to use to set their lows and highs. As an example, watch EURUSD spot today if you have something approaching Direct Markey Access and if you watch closely enough you may note that the high as printed on your screen (for eg.) sometimes moves higher a few seconds after the price has actually moved lower. A less charitable person than I would suggest it was to ensure all the stops on the banks' electronic platforms could be said to have been done within 'the range' ( whatever that is ). I guess it might just be an optical illusion generated by my mind's inability to accept being stopped at the high. Ha!
SideBar on this last thing– one great method market makers employ to get stops done is to drastically widen their spreads when near stops. ( Much small print allows stops to be done if inside the spread for ' risk management' purposes ). This may go some way to explaining the mystery of the changing highs/lows after the fact….
John Bollinger writes in:
I don't understand. If charts aren't predictive why in the hell do you all waste your time looking at them? Do you have so much time on your hands that you can engage in frivolous pursuits at work? If you gonna talk the talk, walk the walk. If you think charts aren't helpful, STOP LOOKING AT THEM.
Rocky Humbert writes:
While I am in agreement with the inestimable Mr. Bollinger that looking at charts has utility, I would be cautious about the term "predictive."
When I go to the doctor's office, her nurse always takes my temperature. My temperature is not so much "predictive," but rather it is informational. In numerous ways, looking at charts are like taking a patient's temperature.
I wish I could claim credit for this insight, but I can't. It's from Bruce Kovner (who I still consider the best trader/investor from a risk-adjusted return perspective of the past 30+ years.)
Ed Stewart writes:
It seems to me that body temperature is predictive of future temperature change do to homeostasis. The breakout from the range where homeostasis functions is going to be predictive of body temp = ambient temp if there is not a reversal or intervention.
Rocky Humbert replies:
Fair point. But you don't need to take a patient's temperature to know that EVENTUALLY body temperature = ambient temperature.
Keynes figured that out when he wrote that "in the long term, we're all dead." (See: JM Keynes "Tract on Monetary Reform, (1923) Chapter 3)
Kovner's actual quote was in reference to so-called fundamentalists who scoff at charts. He said, "Would you go to a doctor who didn't take a patient's temperature."
Gary Rogan writes:
You don't need to take the patient's temperature nor to study medicine to know that eventually the body will assume ambient temperature, but there are clearly situations when the current temperature is highly predictive of the timing, barring an intervention. As such, this whole analogy and the corresponding point just don't work.
A more expanded quote by Keynes reads as follows: The long run is a misleading guide to current affairs. In the long run we are all dead. Economists set themselves too easy, too useless a task if in tempestuous seasons they can only tell us that when the storm is past the ocean is flat again. He was in fact arguing for short-term action based on predictions even though in the long run the economy will recover. So it a way it's almost the opposite point to what Rockstergeist indicated he was making.
Craig Mee writes:
No doubt with the right risk management you can make money trading in many ways, but surely the best outcome is to not leave plenty on the table and have a lot of what ifs in the outcome, together with an ordinary win loss ratio while still banking a healthy return. In the pursuit of excellence, it doesn't seem winning and the above go hand in hand. Though possibly for others this isn't an issue, and probably quite rightly it's all about the bottom line. Hence the saying, "trade the way that you're comfortable with".
Gary Phillips writes:
Considering the maelstrom of controversy and unchecked emotion the subject elicits, perhaps TA should join sex, politics, and religion on the list of banned subjects for this site.
John Bollinger replies:
Careful, the site will become very quiet as the best part of what is discussed here is technical analysis in one way or another as a survey of the literature will confirm.
1. The January barometer has become a Judas goat for the weak to be slaughtered having failed big when down the last 3 times, in 2009, 2010, and 2014 with average subsequent rises in double digits each time (after holding in 2008) but failing in 2005 and 2003.
2. The stock markets swoon in last few hours on Friday, Jan 30 was 10th worst in last 15 years.
3. Some constructal numbers of the week: gold below 1300, SPU below 2000, and wheat below 5.00, and vix above 20.
4. The best book on science I have read is Michael Munowitz Principles of Chemistry. Some other great books I am reading is Paco Underhill Why We Buy (does for buying what we should do for the market in terms of scientific analysis), Russ Roberts How Adam Smith Can Change Your Life (applies the theory of moral sentiments to how to live happily in current days), Paul Moskowitz and Jon Wertheim Scorecasting (applies sabermetrics and counting to our favorite sports shibboleths), Michael Begon, Townsend, and Harper Ecology 4th edition (the best selling standard ecology book these days) and William Esterly The Tyranny of Experts (how planning leads to poverty compared to the invisible hand), Chris Lewit The Secrets of Spanish Tennis (gives some great footwork drills the Spanish use to rise to top), Lamar Underhood The Duck Hunter's Book (the most beautiful writing about fauna I have ever read and reread that makes you long for the beauty and poetry of bygone pastimes) Uri Gneezy and John List The Why Axis (uses pseudo experiments in real life and contrived anthropogical settings to attempt to prove liberal shibboleths like why genetics and incentives don't matter), David Hand The Improbability Principle (why miracles are likely by chance). That's enough.
5. The service rate paid by the world's most sanctimonious billionaire has risen from 2.5% to 9.5% on quarterly ebit this last reported quarter.
6. The ratio of stocks to bonds is at a 1 year low.
7. Gold is playing footsie with 1300 and SPU with 2000
8. Crude broke a string of 15 consecutive weekly declines with a 7.5% rise this week finally showing that futures moves to telescope reductions in supply the way Heyne elegantly shows they do.
9. The pythagorean theory of baseball runs scored for and against is a statistical due to random numbers, completely consistent with chance and has nothing to do with any recurring tendencies or baseball tendencies.
10. When my kids and relations start calling me worrying about how far the stock market is likely to fall, it's bullish. Conversely when they all start apps, it's time to wonder whether that goose has been plucked.
As to point 1.
I posit that all 'indicators', techniques and strategies in the public domain are worse than useless as presented. Within this I include everything preprogrammed into trading software like Bloomberg or Tradestation, the 'January effect', every indicator written about in Futures magazine etc… There are a few public strategies that some firms have made money from but the volatility is enormous and no note is made of survivor bias of others who used the strategy. There are then the preprogrammed techniques available that can be very useful but only as part of a bigger trading process. These last are probably less pernicious than claptrap like the RSI.
It belittles us all to discuss these things.
Consider it this way– everything that makes its way into a magazine or gets programmed into trading software is detritus from the core of truly predictive strategies.
If there is anything to be gained from this it is that you have to do your own homework.
Larry Williams writes:
With all due respect you are way off base on this issue; you mean to say OBV is useless, that seasonals have no value that volatility breakouts are worthless, that Bollinger bands are junk and select price patterns have no value? COT is just a joke, that watching spreads and premiums is the same as an Ouija board? Delivery intentions tell us nothing and advancing stocks, volume and Open Interest reflect nothing?
There are lots of great tools in public domain, just as there are good saws and hammers but it takes a good carpenter to make them work.
Anatoly Veltman writes:
Paragraph 1 falls apart on many levels: so what that "it" failed in 2009 and 2010 at price levels triple and double the 2015 level? So what that "it" failed in 2014 - then via principle of alternating years, "it" better work in 2015! But most of all: in day and age of still ZIRP manipulation, what historical market stats? The 2009-2010 were onset of QE, and 2015 is sunset!
Ed Stewart writes:
Taking into account changing cycles, I tend to disagree. I think there is quite a bit of stuff in the public domain that is very worthwhile.
For starters, a careful reading of Victor's book revealed many more specific ideas than it seemed on a casual reading, which I'm sure many/most here know. I have actually made more than decent money with a few ideas (gasp!) I found in the first market wizards book. Larry's book is a bit of a brain dump (which I always like, no offense there), but once again I found some good ideas in it.
I made (for me, not relative to a big fund manager) very significant profits in 2012-2013 using concepts that I first learned about (If I recall) on Falkenstien's blog, and for a time I tried to get a fund started to trade that market. My thought is that sometimes the market is rich for a particular approach do to a counterparty paying a massive premium, consequently sometimes these things go on even when everyone doubts them (which is why they might keep working).
I think the key to public domain stuff is that if one gets the concept behind a good rule-set there might be 1000 other rules related, waiting to be discovered that might be more attuned to the current cycle of market behavior.
Another is in combining ideas. For example in my way of seeing things there are environments were "naive" strategies are very effective - it is a matter of if u can catagolize that environment and then if there is some persistence to it in the next period (My finding is that there often is), though never perfect.
One last thing I learned is (perhaps contradicting the above) Don't ever write anything and assume that no one will reverse engineer and map out every qualitative thing you write. I had a trading blog that admittedly was mostly goofy stuff i wrote to draw free traffic from google, but also some pretty good core ideas I have made good hay with. Then one week I got emails from two different guys (one a big algo firm, the other an execution algo guy at MS) basically saying, "hey, I mapped out these ideas ideas, they really work - thanks!". The next week I took the blog down. So my conclusion is while some good stuff is in the public domain, don't put anything of value in the public domain yourself, even in vague terms not intended to attract a sophisticated audience.
Stefan Martinek writes:
From whatever I tested, +90% does not hold or does not improve the base case. Few areas are fine despite being in public domain. They can be further developed. It also helps to start PC at least 250-350 times per year, and make tests before forming opinions. There are so many people with beliefs but when you ask them "show me the codes", there is nothing to show. Sometimes an argument goes that you can take anything and make it working, making the dog fly; I agree but I do not think it is a good use of time.
I have been in Chile and Argentina the last month learning about markets from pursuing large brown trout.
Here's a great lesson. Other fishermen were fishing in the deep pools where one would think fish hide. I found my 11 lb Brown about 12" from the shore. He gladly took my fly and what a time we had untill I released him back to the waters. I hade numerous encounters with these lunkers, lurking where bugs fall into the water right next to the shore. No one else fished there.
Casting away in shallow waters.
Happy trails to all
January 22, 2015 | 3 Comments
The RSP (equal-weighted) S&P index ETF is well-known. Less known is the RYE (equal-weighted energy sector ETF). It has only existed since about 2006.
Equal-weighted ETF's give a larger weighting to smaller-capitalization stocks and, to the extent that individual stocks approach zero, they engage in the Rocky pastime of "scaling down to oblivion". That is, If cap weighted indices "ride the trend," equal-weighted indices sell the winners and add to the losers on each rebalancing.
Might anyone have some insights about whether such a practice is inherently superior or inferior over time? And especially for a (distressed) sector index?
Kora Reddy writes:
But the academic literature suggest otherwise: "equal-weighting is a contrarian strategy that exploits the "reversal" in stock prices" (see this pic).
Except in Australia, equi-weighted outperformed the cap-weighted in major countries.
Gordon Haave writes:
I wrote about this 6-7 years ago when the first Wisdom Tree stuff came out and they were talking about how equal weighted was superior to cap weighted and showed the back-tested numbers. All they were really saying is that "over time small caps beat large caps" which isn't exactly news.
To call a equal weighted index and "index" is itself misleading. A cap weighted index is "the market" or some approximation thereof. Theoretically every single market player could go passive and be in it. You can't do that with an equal weighted index (or at least not without distorting prices).
As to your idea of how they have to double down on the loses that is somewhat limited by the fact that once the name falls out of the index it is dropped.
Larry Williams comments:
Along that line Our work shows it is better to invest equal dollar amounts vs equal share amounts
Gibbons Burke adds:
I know a fund which used to invest 90% of client stake in SPX via SPY. A couple of years ago they switched to 10% equal dollar investment in each of the nine sector select spdr ETFs, with the intent of rebalancing to equal dollar allocation annually. They found, in testing, the strategy provided an average of 200 bps of boost each year over the cap-weighted all-SPY investment.
Regarding a depressed sector, is there any truth to the adage: "Buy the stock that has gone down the least, and also the one that has gone down the most". The strong stock will come back smartly and the oversold weak stock will come up from being smashed on a higher percentage then the middle of the pack.
So if this is true you could design your own basket of strong stock leaders in the depressed sector mixed with oversold beaten down stocks that pass a screening survival test.
Erich Eppelbaum adds:
Theoretically speaking, re-balancing a portfolio by using the winnings to buy more of the losers is at the heart of the only portfolio selection methodology that I know of that mathematically guarantees to asymptotically outperform the best stock included in the portfolio (See Thomas Cover's Universal Portfolio seminal 1991 paper): pdf link.
I don't know if in real life the portfolios resulting from this methodology are inferior or superior over time to those created by rebalancing based on allocating more to the winners (such as a market cap weighted portfolio); I would assume that any result would depend heavily on the rebalancing costs and slippage (the liquidity of big vs small stocks matter, especially when trying to push size), and I would assume that the slippage incurred in a market cap weighted portfolio would be less than that incurred in a equal weight portfolio (less small company shares to buy/sell).
In reference to a previous post, another thing to consider is that perhaps there are many effects at play other than the small-cap "more-risk-more-reward" effect. For example, a sell-the-winners-buy-the-losers methodology could be profiting partly by say the volatility harvesting effect described by Claude Shannon.
This brings up another question: The volatility harvesting effect becomes greater as the volatility of the portfolio's underlying stocks increases. In the stock market, volatility usually increases when the market falls. Could this mean that an equal weighted/rebalanced portfolio would outperform a market cap weighted portfolio during bad times? and would the opposite be true during good times? Would be interesting to test…
Here is some intriguing research from a book my psychiatrist son is working on:
"They found asymmetry between buy and sell order placement. Sellers consistently place their sell orders "further from the market" (i.e. further from the best quote) than buyers do with their bid orders. On average, that is, buy orders are "closer to the market" than sell order. Asks were placed, on average, 23.4% further away from the market than bids were."
Perhaps this has to do with endowment effect? The fact that people often demand much more to give up an object than they would be willing to pay to acquire it.
This paper examines the predictive power of money supply growth for stock returns. An understanding of any such predictability would be useful for market practitioners and policy makers given the monetary expansion of recent years. Furthermore, knowledge of this relationship can aid our understanding of the causes of stock price movement. Using monthly data over the time period 1959:1-2012:12, we illustrate that money supply growth has negative predictive power for stock returns over and above the predictability contained in more standard predictive variables, including the dividend yield, interest rates and output measures. Of particular note, the predictive power is strongest for future returns measured over a one- to ten-year horizon and suggests increasing money supply is associated with lower risk. Additionally using both a dividend growth predictive equation and a VAR we report results suggesting that predictability also occurs through the cash flow channel. Finally, by using rolling window forecasts we can determine that money supply growth contains incremental information over standard forecast variables and that the result is robust across the sample.
Big move covers small move is the "secret" to most magic effects. As a small move that covers a large one would have a very small ability to "predict" stocks.
A disturbing chart: "This is Probably the Second Worst Time in History to Own Stocks"
Bill Rafter writes:
The trouble with the chart is that the regression fit was done cumulatively, resulting in older data being subject to look-ahead bias. Thus only the current values are useful, and one wonders exactly how useful. As Steve has commented, the way to foil that is to use a moving regression fit in which the values are static over time, always taking the last point in the fit. Thus all data, past and current are relevant and can then be used in statistical studies.
The question that then comes up is which lookback period do you use. Wherever possible all lookback periods should be adaptive, the question then being to what input. In shorter term price data the market will tell you the relevant lookback period. I have never tried determining lookbacks for longer term data because (a) I don't expect to live long enough to take advantage of it, and (b) too many things can happen in the short run to screw up a good plan. Most people don't marry someone in their 20s based on the supposition that (s)he will look good in their 70s.
I also question the use of any equity or debt data prior to 1972. If you don't know why, ask Stefan. **That's one of the great things about the list; there are sources for just about everything.
Several moving functions you should consider:
Moving linear (i.e., regression) fits and their slopes.
Moving parabolic fits and their slopes. Since most economic and price data are parabolic, this is the better of the two. There is also something to be gained in the difference between a parabolic fit and a linear fit. Fitting parabolas is quite tricky, and it took us a while to code it. If you try to do so and want a check on your efforts, try fitting a parabola to a straight line. If the result is ludicrous, try a different method.
Moving correlations are particularly interesting between markets that might be alternatives to one another. Moving correlations between stocks and bonds (levels to levels) are something we have used for years and continue to do so. I thank Gibbons for his comment that Colby & Myers recommended them, as I had not been aware of that. (I'm not a fan of C&M.)
Gyve Bones responds:
Colby and Myers didn't recommend the linear regression study per se… the empirical analysis simply showed that study to perform best with a fixed loopback parameter over NYSE index returns data over a long period of time compared to other trend following signal generators. This book was an early attempt to quantify different approaches to see how they performed trying as best as can be done to compare apples to apples. In the mid-to-late 80s, it was the best thing that had been done like that since Dunn & Hargitt's study using punch card futures data in the late 1960s (which found that the Donchian Four Week system was best, the system which launched a thousand CTA, including the Dennis Turtles and their spawn.) Another similar study was done in the 90s by Jack Schwager and another fellow whose name escapes me at the moment which was well done.
Larry Williams adds:
A question: when was the regression line fit? Today? 20 years ago? 50 years ago? The slope will change based on your starting and end points. How overbought or sold is a function of this. A more careful analysis would either apply this same "method" every year with a set of rules (i.e sell above x% overbought) or would do the same thing on a rolling window basis. It's an interesting chart nonetheless and gives one pause, but I would suggest it lacks a certain amount of rigor.
Gibbons Burke writes:
It seems to me that this is a flawed chart to look at historically to make rules from because the trend line drawn into the past contains information about the future. The line is drawn using the linear regression of the entire data set so, for example, the line segment covering 1998-1999 "knows" about what happened in 2014. Very deceptive and misleading to make a rule based on the relationship of the data to the trend line.
Victor Niederhoffer comments:
The disturbing chart is a case study of why charting is so misleading because of the regression bias and also at the variance of a sum is the sum of the variances.
Steve Ellison says:
Here is the way to solve the problem of the regression line incorporating future data. Attached is a graph of a "moving regression", as Dr. Rafter calls it. For each date, the red point is the last point of a 30-year regression of the S&P 500 as of that date (the graph is from 2010).
Whatever became of Martin Armstrong? Now a movie! Egads!
X= "I touched the stove"
Y= "I got third degree burns"
then I would NOT disregard the data set as "too small to be indicative".
Larry Williams writes:
Yes, yes…one sample size is adequate–there seems to be a connection readily seen.
Leo Jia replies:
My issue is when X is not exclusive to Y, I have not much clue on how X happens and whether Y has a rational reason to be linked with X. This can be possibly because if it is too clear it might very well be different later.
It is quite like a situation where I am a monkey in the zoo. Most of the time I am kept very hungry. The zoologists play a lot of games with me, delivering food here and there at times. During the last five years, I discovered this thing (which I have no idea what is but you humans call it "stove") 10 times in my play field. The 9 times I touched it, it was warm but not harmful and dispensed quite a lot of food, although one time it had no food and was hot so I got burnt then. I am not sure if this is part of the game, but I am clever enough to remember this.
I keep in mind that the stove was not the only thing I encountered that dispensed food. There have been a lot of other situations, one of which is that, quite frequently, you human spectators throw in bananas andvcandies though also often times with garbage which causes some real pain.
So in this case, how do I take into account the stove case?
This is the first hard core data I have seen that supports what only seem natural:
Long-term use of both mobile and cordless phones is associated with an increased risk for glioma, the most common type of brain tumor, the latest research on the subject concludes. The new study shows that the risk for glioma was tripled among those using a wireless phone for more than 25 years and that the risk was also greater for those who had started using mobile or cordless phones before age 20 years. "Doctors should be very concerned by this and discuss precautions with their patients," study author Lennart Hardell, MD, PhD, professor, Department of Oncology, University Hospital, Örebro, Sweden, told Medscape Medical News. Such precautions, he said, include using hands-free phones with the "loud speaker" feature and text messaging instead of phoning.
It's the optimism of the electorate that causes the rally right around election time, not the actual election.
This goes with the chair's admonishment to never play poker with someone named doc. Never play poker with a magician either, you will always lose.
Larry Williams writes:
There is this trick I've seen which is perhaps the all time best card trick. I call you on the phone
a million miles away, you cut a deck and choose a card. I tell you what the card is.
Seems impossible, until you know how it's done and that's a lot like trading.
Jeff Watson writes:
Larry, bravo, but do you think
you should let the masses in on this trick? That trick is the bomb and a
variation of that is my current bet of the moment at the 13th hole of
my club. I try to be hush-hush about profit centers like these nice
little bar wagers, and any other prop bet, including wheat, which seems
to be at the center of the board on the craps table lately. The wheat
market's been asking how may ways does it take for the pass to be made 4
times in row with the dice?(I know the answer, just trying to provoke
some good insight from the readers).
I have a signal for when you should short companies. It was a fascinating insight from my female friend.
Do it the minute they choose a woman as CEO. This isn't because the woman won't be competent. It will be because the boards don't think the company can be saved.
They shove a woman out in front so she will take the fall!
This is after decades of inbred cultures of irresponsibility, denial, arrogance, lack of innovation, and crappy service have brought a once good company to its knees.
I have spent a great deal of time of the last 50 some years looking at volume and found nothing of great value. That may reveal my intellectual weakness or is volume the Emperor's New Clothes kind of story?
My fading memory of the opening breakout is that it was created by Jake Bernstein and the break out was of the first 15 minutes and required a close > or < than the 15 minute zone; that too I am certain on balance would not be very profitable it still needs discretion or another tool to make it valuable.
I'm reading one of the best training books I've ever read for training for endurance sports, which they define as almost any sport lasting more than two minutes. Training for the New Alpinism: A Manual for the Climber as Athlete House, Steve, Johnston, Scott. They draw on many studies from high level Olympic athletic training and physiology.
Technical physiological detail supports their theory. In a nutshell to train for endurance sport, duration as opposed to intensity is key. Building up an aerobic base where you can exert yourself without hard breathing is key to to building mitochondrial mass, capillaries and appropriate ST muscle fiber which builds endurance. High intensity is not a short cut, and can lead to a decrease in endurance and performance. Cross fit is an example of high intensity.
There is no shortcut. It takes long hours building a base for endurance. The effect builds over years.
Larry Williams writes:
I would add to this discussion that endurance does not win races. The winners are the fastest runners, skater's bikers, etc.
When the marathon running aspect of my life began I was doing 100 miles a week, ran 50 milers and all that but could never qualify for The Great Marathon; Boston, as I had to post a 3:25 at a sanctioned race to qualify. I was then running 4 hour marathons, and while I could run all day that was not enough.
Once we began doing speed work on the advice of a Kenyan runner who, while running with I asked, "What do I have to do", was given the simple answer, "run faster".
So off to the track we went for speed work and that on— top of endurance— got us to 4 Bostons, one with Ralph V.
There is a difference between completing a race, triathalon, etc and wining. Winners are fasters and work very hard to gain speed.
Seems like this applies to the markets in some fashion but I'm too slow to put that all together.
Anatoly Veltman writes:
We're always taught that staying in the game is the key, because that's your prerequisite to catch the once-in-a-lifetime move. But then again, ascribed to palindrome: it's not whether you're right or wrong; it's how much you have on when you're really right!
Larry Williams adds:
It's that delicate balance between spend and endurance– above average performance and staying in the game— in our game it seems. At times I have had speed in trading, competition, and like all in this list we have endured, but getting both at the same time still eludes me.
Buffet only has endurance.
Anatoly Veltman writes:
I don't think Buffet only has endurance. He'd been given valuable chunks on silver platter.
Gary Rogan writes:
It seems like being given valuable chunks came after 1990, when he was already a billionaire. He made his first million in 1962, and a million was worth a little more back then. Perhaps someone has the goods, but it doesn't seem like he built up his fortune early on on anything but taking advantage of available opportunities. Early on the opportunities were not flexionic, but later on they got to be that way more and more. He will do or say anything to make a buck, but was he given or did he take what he saw?
As for only having endurance, it would appear based on his objective net worth that in acquiring wealth endurance matters more than speed, unlike marathons.
Rocky Humbert comments:
Mr. Rogan makes a key point which should be underscored. The tortoise beats the hare in investing because of the law of compounding.
In a marathon, the objective incremental value of the runner's speed at mile #2 is the same as at mile #22. That is, the marathon result is a simple sum of the time used for each mile.
In a lifetime of investing, the incremental value is different at year #2 versus year #22 … because net worth is a geometric series due to compounding.
There are many subtle aspects to this — the effects of volatility on the compounding, and the effect of a bankruptcy in year #1 versus year #22, etc.
Lastly, to the extent that one believes that there is a random/luck/chance is a factor, the turtoise will do even better than the hare.
Ralph Vince writes:
Good points Rocky (ever-prescient, except in matters matrimonial and matriarchal, in my humble opinion). In reading what you wrote though, the following question comes to mind (and I am unable to answer it, perhaps you or someone with a more sports-physiology knowledge can — my interest here in in the mathematical function pertaining to…).
There is not difference in benefit accruing to the marathoner by a given speed at mile 2 versus mile 22. However, is there a tradeoff a cost, involved between running wither of these faster that would indicate a particular strategy as being more preferable than another? I know individual marathoners may have a different take on this, I'm more concerned with the actual physiological function however.
Overall fitness requires strength, speed/agility, and flexibility. The mental component is extremely important as it is the brain that gives the signals to the muscles to act. If there is no deep reserve, or lack of strength, the brain senses this and pulls back autonomic functions. Motivation however allows the brain to tap the reserves of strength and endurance in times of need.
Each individual has different training requirements. Many a sport trainer or coach has found this out the hard way. Each individual reacts to training in different ways at different times in the training regime.
Training actual changes the body and brain functions. Mitochondrial cellular mass actually increases, as does enzyme production and along with muscle mass and function.
Recently I started logging my training efforts in a quantitative manner. Very helpful.
Overtraining is a common problem. A typical cure is to increase training, but it is counterproductive. When you feel tired, cut back, or rest. Your body is telling you something.
There is another aspect of winning races beyond speed and endurance.
I saw that today in our Memorial Day 2 Mile race. Teddy Seymour, a 71 year old trader and the first black man to circumnavigate the world by himself, knocked 2 minutes off last years time. For non runners that's huge.
I asked Teddy, "What have you been doing in training that was an amazing performance today?"
His reply was, "I'm resting more now, I run 5 days and take off 2, what I've found is that rest helps me get faster. All my life (he's a former marine) I have pushed it, it's taken a long time to learn stop, to rest."
Happy Memorial Day Trails to all.
Scott Brooks writes:
Larry makes an excellent point. Rest is vitally important.
I was taught how to lift weights by Clif Koons. We used to work together at Executive Fitness in St. Louis (which went of business over 20 years ago).
One of the things Clif emphasized was rest.
We used to have guys coming into our gym that would work out long and hard……and do it 7 days a week. Those guys would hit plateaus that would last seemingly forever. Yet, other guys would work out just as hard, but take several days per week off to let their bodies rest and recuperate. They got better results than those that would work out everyday.
I think trading can be the same way. Yes, we need to immerse ourselves in the business and become students of trading, but at the same time, we need time off from trading to let our minds recuperate. Sitting around, doing nothing, hiking, spending time with the family playing games the kids enjoy (I HATE Mexican Train……but my kids love it…..so I play it……their laughter makes it all worthwhile, though).
Although Clif and haven't worked together in 30 years, we have run into each other around town a few times and have kept in contact via Facebook. However, all that aside, if anyone would ever be interested in working with a true master of his craft, CLIF IS THE MAN to contact. He is a truly skilled student of his business, and he's a gentleman. I highly recommend Clif
For those that may be interested, here is Clif's website.
(Even if you're not interested check it out. Clif is one in-shape dude…..and he's in his mid-50s.)
April 24, 2014 | Leave a Comment
Just 52 cards (weeks) with 4 suits (seasons) with 13 cards (weeks) in each season can be shuffled into 400000000000000000000000000000000000000000000000000000000000000000000 combinations. That's 4 and 69 0's.
The Pips (spots on a card) = the number of days in the year for trivia buffs (jacks count 11, Queens 12 and Kings 13).
Oh and here is a mind-fuddling bit of math that I perform with all the time and I'm still shocked that it always works: The Gilbreath Shuffle.
Easan Katir writes:
Years ago one learned from professional card men how to shuffle overhand, riffle, even spread cards on a table and mess them up — and end up with every card in the exact place as before. Market application? There are those who know how to shuffle the news, analysts' ratings, technical patterns, so their capital increases — while the audience of investors looks on, amazed.
I often look at the amount of past price action used to attempt to predict future price action.
Some things that are useful to ponder, in my opinion, are:
1. Is more past data really going to help to make the future prediction more accurate?
2. Should there be a balance between look-back period and forecast horizon?
3. How important is data accuracy (tick level to daily range)?
4. Should reference points & times be changed every second, minute and hour of a day?
5. Should the definition of 'big move' and 'small move' be a fixed thing or relative to the market's current level?
For me, it's NO, NO, VERY, YES & RELATIVE.
Leo Jia writes:
In Schwager's book "Hedge Fund Market Wizards", Jaffray Woodriff addressed this in the following way. Any comments? It does have to do with what one is trying to get, doesn't it?
"Do you give the same weight to data from the 1980s as data from the 2000s?
Sometimes we give a little more weight to more recent data, but it is amazing how valuable older data still is. The stationarity of the patterns we have uncovered is amazing to me, as I would have expected predictive patterns in markets to change more over the longer term."
Larry Williams writes:
As I see it we certainly cannot compare data from the old pit sessions to today's electronic markets.
And how do we handle Saturday trading in the real old days or that markets were close on election day…or in 1967 the markets were close on Wednesday… or there used to be a massively important bond report the goosed bonds on Thursday??
We need to understand what the data represents.
March 28, 2014 | Leave a Comment
Here is some Friday fun. Stimulated by the S-Man's comments about electrical utilities combined with the government report on corporate profit margins being near/at record highs while company hiring is sluggish….
Hypothesis: The S-Man has hypothesized that companies with fewer employees make better investments. This is consistent with the view that capital is cheaper than labor. And that government regulations have made hiring more expensive. Rocky accepts that companies with only one employee have no sex discrimination lawsuits. And companies with less than 50 employees have no ObamaCare issues. But do companies that produce more revenues per employee perform better per se (as measured by the stock price)? Rocky did a 5 minute back-of-the-envelope study on the 150 largest (by market cap) US companies.
He asked: what is the correlation between the 5 year total return of the stock price and the revenues/employee?
The answer: -.21 . That is, ceteris paribus, higher 5 year total returns are INVERSELY CORRELATED with higher revenues/employee. Or put another way, lower revenues/employee are correlated with better 5 year stock performance.
And it is suggestive that MORE employees producing FEWER revenues produce BETTER stock prices. Very counterintuitive.
This is a very sloppy analysis. But it's food for thought. Now it's pizza time. And that's even better food for thought.
(Rocky is sure that others have done this analysis in a more sophisticated way and he welcomes critical comments.)
Larry Williams writes:
I tested revenue per employee a few years back so this is reaching into old rusty memory banks but the recollection is it was just "ok" as a value measure and did not fit all companies. Service industries have to have employees, some high tech and mortgage companies can get by with less.
Memory is clear that this was not anything special to use to find value in stocks.
The market today is like a pretty girl. It is very attractive from the long side in many markets, but it gives you no opportunities to buy on the cheap. Where is Anatoly with his bargains outside of Hawaii and Disney today?
Larry Williams writes:
One of our members has not had a losing trade in many years now; not a one in, I'm not sure, maybe 10 years. Ironically one of his clients, a large bank, closed out a few years back thinking something was wrong because of the excellent performance. Such is the life of a trader.
Alston Mabry writes:
You could argue that you don't need a transaction tax when it has already been levied by HFT firms.
Ed Stewart writes:
Virtu's dividend paying history seems very aggressive to me relative to what looks like its operating earnings and net assets. Last year $250m of the $430m dividend payment was financed. $250 being very close to the firms book value equity while earnings were 180m.
2011 and 2012 had earnings near 90M and dividend payments of 120M and 130M. Around 1.5B total paid since 2006.
Is this the magic of steady returns + finance + limited liability?
One of my favorite short term patterns here today; outside day in up close in tight trend markets are bullish and daily a/d line at new high…
Gary Phillips writes:
Beans in the teens already, but seasonally ripe for a rally…also my only worry is post-taper/risk-on has seen a stronger dollar, although it did decouple today after the european close…
You young'uns might not know who Tim McCoy was. He was one of the first cowboy stars in movies.
I have been lapping up every page of his book, "Tim McCoy Remembers The West". It is a dandy, because he was no drug store/dandy cowboy, he was a real deal cowboy (and Arapaho) who just so happened to turn up in Hollywood, which is just a little part of his larger-than-life life.
It has lots of wonderful history (insights to Custer's Battle I had never read before), movie making, living in Wyoming in the 1880s…and on and on.
Western, Hollywood (you will learn why Hollywood was the start of it all), and history buffs will find this a gem.
There are several reasons the Oregon Ducks have become a powerhouse in football. I listen intently for life and trading lessons from this team.
Here are a few pointers:
Was it about what Kelly expected, based on the practice week Oregon had?
"We don't put a point total on it or think about it that way," he said. "But it's almost like, you studied really hard for the exam, and you can't wait for the professor to put it on your desk."
"Obviously it's not the same as a game-time situation, but throughout my career we've gone through practice scenarios, what we need to do if we are down six or down four, three," he said. "We try to make us feel uncomfortable. Whatever situation we end up being in, I think the team is prepared and is confident in our ability to be successful in those situations."
Coach Mark Helfrich echoed Mariota's sentiments, with his own catchphrase being making sure his players were "comfortable being uncomfortable."
There are lots of treasures to be found in My Lunches With Orson conversations over lunch with Orson Welles by Peter Biskind. It's a very good book.
Aversion to losses or aversion to risk? Which of the two is addressed by willingness and ability to close out losing trades?
Well, without invoking mathematics where it is not necessary, it is common and logical to place on the table that when a losing trade is closed one has the willingness and aversion to the risk of the persistence of loss becoming into a bigger one and one does not have aversion to the present level of loss in being accepted.
Now on the other hand, unwillingness to stop out a losing trade is indeed loss aversion.
The computations that show that having utilized some sort of mechanical rules for stopping out adverse incursions actually increased the probability of meeting with adverse incursions is totally flawed abuse of statistics.
1) Historical data analysis does not undertake the "uncertainty at a given moment to decide upon" into account and is definitely incorporating hindsight 20:20 vision mind-set.
2) Any measurements of uncertainty and thus risk are never definite, since measurement of uncertainty too will be having an uncertainty of its own. So a trader in the middle of a losing trade has to decide that the level of uncertainty in his method, mind or cognition regarding the calculation of the "value of uncertainty" in his trade has become too high for him to handle. That's where humility, the currency that prevents others from profiting more from your mistake, can come into play and allow the willingness to hit the stop.
3) However, when either with or without the illusions of statistical computations of stop losses increasing the probability of meeting with more losing trades, one fails to control the human weakness of loss aversion, to somehow and anyhow turn that loss into a profit, one is becoming totally risk-insensitive. From skill, the turf changes to the power of prayer. The game begins to change from action to hope. Inconsistency of thoughts thus turns one into a trader who is continuing to hold on to risk without a mental apparatus to assess it or react to it. As the loss continues to grow not only the lack of willingness to take it hurts, the ability to accept the increasingly bigger loss also dwindles rapidly.
I am ready to be thrown before any firing squads of mathematical minds and ideas on this list if they can with or without numbers help me learn how come this list celebrates and cherishes a human value of humility and yet indulges in an idea that staying on in a trade that has incurred a level of loss greater than anticipated when the trade was opened are mutually consistent.
I would close my submission for now with one thought:
When loss aversion creeps in it makes a decision system (mind) risk-insensitive and with no respect for risk, returns are impossible. Yet, if a mind continues to be risk-averse it does not have loss-insensitivity and in humility such a mind closes out risk that has turned out to be less than comprehensible.
Phil McDonnell responds:
Since I am the well known culprit I shall give Mr. Kedia a reply. If the probability of a decline art the end of a period of time equal to your stop is p then the probability of losing the stop amount with a stop loss strategy is 2 * p. It is simply a derived relationship. It is what it is.
It is not a misuse of statistics but rather a description of how a stop loss exit strategy will change the distribution of returns. Larry Connors studied over 200,000 trades from a winning system and compared the results with and without stops. He found the use of stops increased the probability of loss and reduced the expected gain.
In my opinion the best way to trade is to reduce position size so that no one loss hurts your account too badly. That means many small positions to me.
Larry Williams adds:
Ahhh here I go off on a rant; please excuse a tired old mans bitterness at system vendors who claim stops hurt performance.
Yes, they are correct in that the statistics of your system will look better if one) you don't use a stop and two) your use a market with a perpetual upward bias like the stock indexes have been, usually.
They are absolutely totally incorrect in terms of living the life of a trader. So what if I am long in a position that eventually shows a profit but because I did not have a stop loss that one trade moved against be 20,000 or $30,000 and it took a year or so to get out of? Yeah, the numbers look good (high accuracy) with no stops but it's one hell of a lifestyle.
High accuracy is a false God.
Consistency and never being in a place where you can get killed is more critical. Perhaps Mr. Connors has never sat through the reality of a large loss, especially in a large position. I have; I would rather battle the devil at midnight on a new moon with both hands tied behind my back.
It's one thing to have a system with "good numbers" it is quite another thing to be a trader and have to deal with reality.
It only takes one bullet in the chamber to kill you when playing Russian roulette. As near as I can tell trading without any stops, in any way whatsoever, is just the American version of this form of spinning the wheel.
Play the game as you wish but please heed the warnings of an old man.
Leo Jia adds:
I have been studying the use of stops. Due to loss aversion I guess, I would like to use narrow stops. But among the various strategies I have yet found one working well with narrow stops. Good stops have to be relatively wide in my cases, but having no stops or stops that are too wide clearly hurts results (my trades are time limited). So a good choice for me is to size the position according to the stop size.
Sushil Kedia writes:
If you reduce position size can it be argued that a position of Size N reduces to N-n implies that you took a stop loss on n lots out of N you held. Then too, it validates the fact that you do take stops.
Anatoly Veltman writes:
Larry covered main bases (different markets, different position sizes, different lifestyles) pretty well. I just want to be sure that reader doesn't end up with wrong impression. I think the best conclusion is "it depends".
And because my act follows Larry's (who is certainly biased in favor of stops), let me try this. If you enter based on value (which is certainly against trend), then there is no justification available for a stop. Unless you argue that this stop proves you were an idiot on the entry. But if you are an idiot on value entries, then why play value…
Anton Johnson writes:
The problem with using Conners' simulation as evidence that placing a trade stop-loss reduces returns is that he tested a winning system that likely had never experienced any 5-sigma negative excursions prior to the test date. And of course there are no guarantees that his strategy, or any unbounded trading strategy, will perpetually avoid massive drawdowns.
When implementing a strategic trade, a good compromise between profit maximization and loss mitigation can be achieved by balancing trade size along with a stop-loss, which when placed at a level that only an extreme event will trigger, will likely contain losses to a predetermined range, and also prevent getting stopped-out of a potential winner. If one is disciplined, maintaining a mental stop-loss level is preferable to an order pre-placed in the book, and available for all the bots to scan.
Larry Williams adds:
But speaking of stops, I go back to my litany, my preaching the essential reason for never putting stops on an exchange server, or even your brokers server. Putting stops on servers means that your stop becomes part of the market. And not in a positive sort of way either. Pick a price, hit the button, and take the hit. Discipline is key here.
Ed Stewart writes:
A trader needs a decision process for managing the expectation or expected value of the trade as well as the equity position. The problems occur when these two things are in conflict.
The thing with stops is that at times it makes no sense to get out of a trade when the expected value is still good. What is the difference between exiting at a small stop-loss point 4X in a row vs. one loss of that same size? Well, if at each "stop out" point the expected value was favorable, it makes no sense, one is just locking in losses. At times the best "next trade" is simply staying in the current trade.
However, I see Larry's point and it is a good one. Yet, the example of letting a loss get huge or holding an underwater position for a year is to me something of a false alternative. No exit strategy but hoping for a profit at some point is not a reasonable alternative.
What maters, I think, is the expected value of the trade at each moment, and balancing that against equity and a margin or error to ensure, "staying in the game".
Given this I always trade with mental stops, if not on individual positions, on total account equity. Having that "self-preservation" discipline is useful.
Jeff Watson writes:
I learned very early on in the pit on how to go for the stops, and that weaned me off of stops completely (except in my head).
May 14, 2013 | Leave a Comment
I first saw the 'dead eyes' look of a poker player/loser when I was 13 or so. Still gives me restless nights and I know I cannot become that way.
My dad took me into the "stockman's bar" in Billings, Montana to impress upon me what degenerate, greedy people turn into.
Probably another sleepless tonight tormented by that devil.
Gary Rogan asks:
What is the real difference between gambling and speculation (if you take drinking out of the equation)? Is it having a theory about the odds being better than even and avoiding ruin along the way?
Tim Melvin writes:
I will leave the math side of that answer to those better qualified than I, but one real variable is the lifestyle and people with whom one associates. A speculator can choose his associates. If you have ever been a guest of the Chair you know he surrounds himself with intelligent cultured people from whom he can learn and whom he can teach. There is good music, old books, chess and fresh fruit. The same holds true for many specs I have been fortunate to know.
Contrast that to the casinos and racetracks where your companions out of necessity are drunks, desperates, pimps, thieves, shylocks, charlatans and tourists from the suburbs. Even if you found a way to beat the big, the world of a professional gambler just is not a pleasant place.
Gibbons Burke writes:
Here is something I posted here before on this distinction…
Being called a gambler shouldn't bother a speculator one iota. He is not a gambler; being so called merely establishes the ignorance of the caller. A gambler is one who willingly places his capital at risk in a game where the odds are ineluctably, mathematically or mechanically, set against the player by his counter-party, known as the 'house'. The house sets the odds to its own advantage, and, if, by some wrinkle of skill or fate the gambler wins consistently, the house will summarily eject him from the game as a cheat.
The payoff for gamblers is not necessarily the win, because they inevitably lose, but the play - the rush of the occasional win, the diversion, the community of like minded others. For some, it is a desire to dispose of money in a socially acceptable way without incurring the obligations and responsibilities incurred by giving the money away to others. For some, having some "skin in the game" increases their enjoyment of the event. Sadly, for many, the variable reward on a variable schedule is a form of operant conditioning which reinforces a compulsive addiction to the game.
That said, there are many 'gamblers' who are really speculators, because they participate in games where they develop real edges based on skill, or inside knowledge, and they are not booted for winning. I would include in this number blackjack counters who get away with it, or poker games, where the pot is returned to the players in full, minus a fee to the house for its hospitality*.
Speculators risk their capital in bets with other speculators in a marketplace. The odds are not foreordained by formula or design—for the most part the speculator is in full control of his own destiny, and takes full responsibility for the inevitable losses and misfortunes which he may incur. Speculators pay a 'vig' to the market; real work always involves friction. Someone must pay the light bill. However the market, unlike the casino, does not, often, kick him out of the game for winning, though others may attempt to adapt to or adopt his winning strategies, and the game may change over time requiring the speculator to suss out new rules and regimes.
That said, there are many who are engaged in the pursuit of speculative profits who, by their own lack of skill are really gambling; they are knowingly trading without an identifiable edge. Like gamblers, their utility function is not necessarily to based on growth of their capital. They willingly lose their capital for many reasons, among them: they enjoy the diversion of trading, or the society of other traders, or perhaps they have a psychological need to get rid of lucre obtained by disreputable means.
Reduced to the bare elements: Gamblers are willing losers who occasionally win; speculators are willing winners who occasionally lose.
There is no shame in being called a gambler, either, unless one has succumbed to the play as a compulsion which becomes a destructive vice. Gambling serves a worthwhile function in society: it provides an efficient means to separate valuable capital from those who have no desire to steward it into the hands of those who do, and it often provides the player excellent entertainment and fun in exchange. It's a fair and voluntary trade.
Kim Zussman writes:
One gambles that Ralph and/or Rocky will comment.
Leo Jia adds:
From the perspective of entering trades, I wonder if one should think in this way:
speculators are willing losers who often win; gamblers are willing winners who often lose.
David Hillman adds:
It is rare to find a successful drug lord who is also a junkie.
Craig Mee writes:
One possible definition might be "a gambler chases fast fixed returns based on luck, while a speculator has time on his side to let the market decide how much his edge is worth."
Bill Rafter comments:
Perhaps the true Speculator — one who is on the front lines day after day — knows that to win big for his backers, he HAS to gamble. His only advantage is that he can choose when to play.
Anton Johnson writes:
A speculator strives to be professional, honorable, intellectual, serious, analytical, calm, selective and focused.
Whereas the gambler is corrupt, distracted, moody, impulsive, excitable, desperate and superstitious.
Jeff Watson writes:
I know quite a few gamblers who took their losses like men, gambled in a controlled (but net losing manner), paid their gambling debts before anything else, were first rate sports, family guys, and all around good characters. They just had a monkey on their back. One cannot paint with a broad brush because I have run into some sleazy speculators who make the degenerates that frequent the Jai-Alai Frontons, Dog Tracks, OTB's, etc look like choir boys.
Guys — this is serious, not platitudinous, and I can say it from having suffered the tragic outcomes of compulsive gambling of another — the difference between gambling and speculating is not the game, the company kept, the location, the desperation or the amounts. The only difference is that a gambler, when asked of his criterion, when asked why he is doing this, will respond with "To make money."
That's how a compulsive gambler responds.
Proper money management, at its foundation, requires the question of criteria be answered appropriately, and in doing so, a plan, a road map to achieving that criteria can be approached.
Anton Johnson writes:
It's not the market that defines whether a participant is a Gambler or a Speculator, it's his behavior.
Gibbons Burke writes:
That's the essence of my distinction:
"gamblers are willing losers who occasionally win"
That is, gamblers risk their capital on propositions where the odds are either:
- unknown to them
- cannot be known
- which actual experience has shown to have negative expectation
- or which they know with mathematical precision to be negative
They are rewarded for doing so on a random schedule and a random reward size, which is a pattern of stimulus-response which behavioral scientists have established as one which induces the subject to engage in the behavior the longest without a reward, and creates superstitious as well as compulsive behavior patterns. Because they have traded reason for emotion, they tend not to follow reasonable and disciplined approach to sizing their bets, and often over bet, leading to ruin.
"speculators are willing winners who occasionally lose." That is, speculators risk their capital on propositions where the odds are:
- known to have positive expectation, from (in increasing order of significance) theory, empirical testing, or actual trading experience
They occasionally get unlucky, and have losing streaks, but these players incorporate that risk into the determination of the expectation. Because their approach is reason-based rather than driven by emotion, they usually have disciplined programs for sizing their bets to get the maximum geometric growth of their capital given the characteristics of the return stream, their tolerance for drawdown.
If a player has positive expected value on a bet, then it is not a gamble at all. The house does not gamble. It builds positive expectation into its games. It is a willing winner, although it occasionally loses.
There are positive aspects of gambling, which I have pointed out earlier in the thread and won't belabor. To say that "all gambling is bad" is to take the narrowest view. Gamblers who are willing losers (by my definition all are) provide the opportunities for willing winners (i.e., speculators) to relieve gamblers of the burden of capital they clearly have no desire to hold onto, or are willing to trade in a fair exchange for the excitement of the play, to enable their alcoholic habit, to pass the time, to relieve their boredom, to indulge delusions of grandeur at the hoped-for big win, after which they will quit playing, or combinations of all of the above.
Duncan Coker writes:
I found Trading & Exchanges by Larry Harris a good book on this topic and he defines all the participants in the exchanges and both gambler and speculators have a role to play. Here is something taken from page 6 that make sense to me: "Gamblers trade to entertain". Speculators to "trade to profit from information they have about future prices."
He divides speculators into those that are well informed versus those that are not. One profits at the expense of the other. Investors "use the markets to move money from the present into the future". Borrowers do the opposite.
April 12, 2013 | 3 Comments
I have always followed the Baltimore Orioles because in high school I caught for Dave McNally one of their star pictures.
But this story is not about Dave. It is about Cal Ripken, the guy who always played and set the record for the most consecutive games. There probably should be an asterisk in the record book from a story I was told here by a former well-connected resident of Baltimore.
It seems Cal came home to find that his wife and the gardener, or tennis player, I forget which, were having an affair. He beat one or both of them up, got arrested and was tossed in jail overnight. Which meant he couldn't show up in time for a game.
So what happened? He was loved so much by the local people, as being really one of them, one of the working class, that the electricians went to bat for him.
At game time there were problems with the lighting, they flickered on and off. I actually remember seeing this because of following Dave Nalley's Orioles. So that part of the story I know it's true. My friend here said once the electricians heard that Cal had some problems getting to the game they began to see what they could do. There was a rumor he had been released and was on his way to the stadium so the lights came back on. Then they found out he had not been released. So the lights flickered a little bit more, then turned off with the electricians saying there was a major electrical outage problem and they would not be able to get the lights back on for that game.
Cal was released the following morning and able to play the next day's game, keeping his streak intact. That ladies and gentlemen of the dailyspec, and Orioles fans, is the story of "The night the lights went out in Baltimore.
We have had numerous discussions on this venue regarding stop losses. Part of the surprise from those discussions is that using a stop loss will double your odds of having a loss in the amount of the stop loss.
However the same is true for a profit target. Using a profit target will double your probability of having a gain equal to the target gain. The reason for both phenomena is that in a random walk half of all such trades will get reversed after hitting the target or the stop. The fancy name for this is the Reflection Principle.
Larry Williams writes:
In a random walk, half of all stops/targets get hit, so if that is not true in several trading systems, does it suggest the market is not random?
Anatoly Veltman writes:
Electronic markets are far from random. Your broker's HFT frontruns your orders, and non-broker largest HFTs parallel run your orders. Thus your limit (profit-taking?) order is played against by unabling, and your stop-loss order is played against by triggering. Random? Not to your account.
Ralph Vince asks:
But can non-random ticks, sampled on a bigger time frame, degenerate into randomness?
Anatoly Veltman replies:
In the sense that all those orders, magnified by HFT mechanism, will carry markets somewhere - sure. The other question is: OK, so 70% of executed trades resulted in robbing the outsider spec - but the HFTs and the brokers have not fully benefited by your loss, because of their high overhead (the arms race, et al). So ok, the wall street salaries, the IT salaries get financed out of your pocket. Then the only way to keep you in the game is to inflate your remaining funds…So the mechanism will continue on…but to what end, if the economy is not picking up? So the result may well be non-random: all prices will go up.
Gary Rogan writes:
Clearly the natural drift and/or inflation-driven accelerated drift will result in an upward bias that will make a random walk impossible. In addition, if there is an HFT-induced tendency to hit stops and not hit limit orders (by the way are there any objective statistics that prove that?) the question becomes: would an independent observer looking at the data tick by tick, but who is not himself placing limit/stop orders be able to tell that the statistical nature of the tick distribution has changed?
Jeff Rollert says:
No, HFT is attacking your behavioral biases. Not the academic ones ones. Your bids show your hands.
These are modeled after high yield bond trading patterns.
How would you trade if the book was open and public? That is the point. Trading systems are rational, and your systems are easy prey…seriously, inject the random. To borrow a sports analogy, you can't bore a machine into an error.
When gold was money, its price was measured not in currency but in what it could buy. From the adoption of the Constitution to WWI, except during the Civil War and Reconstruction, the price of gold as currency remained the same — 1 ounce was $20; measured by what it could buy during crashes and depressions, gold's "price" went up. That was equally true during the periods when the dollar was not redeemable in gold at the Constitutional standard — 1873, after 1932; in all these panics what gold could buy in the world's markets has increased. That is to be expected; it is a tautology that, during panics, the prices of things other than money go down and the price of money goes up, and gold has been the world near-money even when it has not been legal tender in the U.S.
George Parkanyi writes:
I think that gold is difficult to read. It's not a slam-dunk by any stretch. In a rapid deflationary scenario where credit markets seize up and no-one trusts counterparties it could get hammered with everything else - people having to liquidate to cover margin calls, just needing cash to meet other obligations and so-on. Also you wouldn't be able to finance it. There would be a complex interplay between flight to "quality" and scrambling to raise cash. And inflation seems to be mixed bag; specific pockets of it - there seems to be plenty in food and energy, but little wage inflation in a globalized economy with a lot of cheap labour still around. Consumer electronics still seem to be trending down.
On the other hand you do have seemingly unstoppable currency debasement underway (the main gold bug meme), and interest rates practically at zero. And bank runs could be good for gold since the issue will be about parking cash somewhere other than in a bank - gold would sop up some of that. Equities at the moment I believe are benefiting from a shift in the perception away from the "safety" of fixed income. Sovereign debt everywhere really does look like crap because of the unsustainable amount of it - why would you tie up your wealth in it? You see it with companies starting and/or increasing dividends because of investor demand for income they're not getting from debt. Debt markets dwarf equity markets, so if this is a real trend, equities, and commodities in the mix somewhere, could go a lot higher. But at some point all this selling of debt will increase interest rates, which will then work against the economy, equities, and commodities - including gold.
Then there are other dynamics. They say central banks are buying gold right now. Bullish or bearish? That's taking production off the market (bullish), but then they have all that more to turn around and dump on the market if the agenda changes again (bearish).
Gold itself hasn't been hit that hard recently - 16 or 17% perhaps from its high - but the miners have been massively hammered, mainly - and I find this ironic - because of high operating cost inflation. I'm thinking its overdone and a fairly old story, setting the stage for at least a bear market rally (the rationale for my current trade), but the market's not agreeing with me so far.
Larry Williams writes:
You make some nice points. I see a strong 4 year cycle operating in gold that called the top real well 2 years ago and suggests a low yet to come.
February 6, 2013 | 2 Comments
Have any of you been to Argentina lately. What is the situation on the street. I am thinking of going there on the way to Antarctica later this year. Is it safe?
Vince Fulco writes:
One thing to seriously consider even though it sounds like you are going thru mostly modern areas, Amex has an extremely affordable medical expense insurance while traveling. For a few hundred bucks, as I recall < $250 for my wife and I when we traveled to South Africa, they'll airlift you out of spots and take care of many extraordinary medical expenses. Considering you'll probably never use it but if you do, fees can run tens of thousands of $ depending on what your normal health insurance covers, brought us piece of mind. Everything can be done online.
Larry Williams writes:
Careful on med-evac policies
I just lost my best buddy here; heart issues. We tried med-evac but the reality is 1) you need a local doctor to agree to release, which they will not do unless stable and 2) before the plane/jet wheels up they must have admitting dr and hospital at the other end.
That takes a long time to arrange
Best is to charter. Don't tell anyone of medical issues-get aboard and then go to closest emergency room to airport.
Carder Dimitroff writes:
I'm sorry for your loss.
I would like to add to your thoughts. Once in the air and within US control, request the pilot use the LIFEGUARD call sign. This will notify the FAA to give the plane priority handling, direct routing and airport priority.
In addition, here may be a helpful link.
A commenter adds:
First: LIFEGUARD has recently been changed to MEDEVAC to conform with ICAO international standards.
Second, and most important: If you are in ANY WAY fearful that your medical situation may be life threatening, communicate this CLEARLY to the crew and they will declare an EMERGENCY.
MEDEVAC flights get priority handling when they request it. But EMERGENCIES — well, let's just say that controllers will move heaven and earth and every airplane in the way to get that aircraft on the ground at the airport of the pilots choosing.
Most domestic airlines subcontract to a company that has professional medical staff on call 24/7 that assists them in determining the best course of action for the patient. They will get a doctor on the radio directly with the flight crew to assess each situation.
Not all right wing radicals are forecasting inflation. Many of us are of the other side of that coin. The conservatives are back to the old argument that money supply creates inflation.that argument was destroyed since the Big O took office. M1,2,3,4 increases did not produce inflation. Gee whillikers, what does that mean?
They should go back to the drawing boards, but instead beat the same old drums, preach the same old mantras.
Alston Mabry writes:
In the present regime, the Fed is increasing the money supply only by the amount of interest they are paying the banks to park at the Fed the very money the Fed shovels at them.
Mr. Allen writes:
The mistake is to think the inflation must show up in rates like in the 1970s. in the 1940s, the last time we had a significantly managed economy rates averaged 2.5% but inflation average 5.5% for the decade. That can occur when there is anchoring or when people think the inflation is transitory in nature. Under a gold standard, which is what inflation targeting is - short rates were volatile but long rates flat as a board because there was no systemmic inflation. Also, you can get stagflation which are higher prices and lower output which is more of what we have experienced as taxes, insurance costs, etc. are up but unit output for many industries flat. Also, do not fail to realize how empty the bucket was in 2008 v. now, Continentals did not immediately loose their value but there were $350 mil of those which in today's dollars is $3.5 trillion, so the Fed may in fact just now be crosses the Rubicon.
A commenter writes:
And 30-100% increases y-o-y in health insurance premiums for independent contractors and other small biz types. why? because they can under the guise of obamacare. really putting the squeeze on some average joes I know.
I think the VIX is greatly over-rated. Here is my synthetic vix (7page PDF ). It follows the actual vix closely and the actual vix is just the market upside down.
A short squeeze is really more of a commodity market function and serves a valid economic purpose.
I really like Atul Gawande's writing and ideas. Failure is an interesting subject that's not looked at enough. The success stories always get the spotlight, the books, the movies, but part of it is just survivorship and luck. Tragic failure comes from making a whole series of mistakes in a row, and making a minor problem in to a major one. Your judgment is not as good under pressure and mistakes lead to others. Sometime multiple small mistakes lead to death as we saw in our previous discussions on the book Deep Survival.
Older people are cautious because they have failed. Many young people have not experienced failure, partly from being protected, partly just less exposure to random bad luck. It's as good as most major ideas, accomplishments are done by the brave, often young people willing to take the risk.
George Coyle comments:
I'm reading a book called Great Failures of the Extremely Successful by Steve Young. It goes through perhaps 100 or more stories (a few pages each) about failure and how it helped various people from different walks of life. Some are great, others not so great, but the book is littered with facts (I am skeptical about some but regardless) about people who overcame failure and adversity. Whenever you have a bad day look up Abe Lincoln's road to the white house.
Larry Williams writes:
Charlie Sheen is my new hero this guy bounces back from disasters better than anyone I have ever seen how can he go through that much humiliation and still go on stage…pretty amazing or great drugs… one or the other I suppose
Dailyspecs who like a good Indiana Jones adventure might want to check out the book The Mountain of Moses: The Discovery of Mount Sinai.
The book is a true story expedition to find the Mountain of Moses in the Arabian peninsula not Sinai.
It does not star Harrison Ford but rather features the site's own Larry Williams.
It is a great read.
Anyone who has dined in Singapore's fabulous cheap eateries may be interested in these numbers.
How much is a recipe worth? About $1.8 million, according to the owner of Kay Lee Roast Meat Joint , who boosted the sale price of her Singapore eatery by that amount when she put it on the market this year.Betty Kong and her husband want S$3.5 million ($2.8 million) for their 60-plastic-stool establishment, a premium over the S$1.25 million assessed value of the site. The price includes the property, their recipe for roasting duck, pork ribs and crispy pig skin as well as other Cantonese-style classics, plus three months of cooking lessons
The Roast Meat Joint generates sales of around S$2,000 a day, she said, or S$620,000 annually, assuming it’s shut one day a week and three days for Lunar New Year holidays. Profit margin is 60 percent, according to her broker Raymond Lo at Knight Frank LLP. The asking price is 5.6 times annual sales, compared with the 1.1 multiple for the Singapore benchmark
Straits Times Index. (FSSTI) It would take six years to recoup the recipe premium.
Larry Williams responds:
Food costs are 20% in the best run restaurants so the 60% profit cannot include labor, overhead etc. No way
Black pepper crab in Singapore is one of the worlds greatest dishes.
The other day I heard somebody say:
"Assuming the future behaves the same as the past, I reason that this way makes my funds efficiently used".
I wanted to say, my experience is that the past is never like the future so we waste valuable time and skills on a false postulate.
As I see it, it is better to have a core strategy to deal with equity drawdowns, etc –based on logic–as opposed to a strategy based on the past real results or back tested as that is for the most part a make believe world since it never happens quite that way again.
Gary Rogan writes:
Larry's statement seems to be exceptionally profound in what it's saying and in the unambiguous nature of what it's saying. Speculation seems to be about predicting the future. Is there anything but the past, in some sense, that can guide us towards correctly predicting the future? If so, and if it's not similarity, what is it about the past that can help predict the future?
John Netto comments:
There are ample proverbs espousing the merits of both deriving information on events which have taken place before us, as well as the the complexities in attempting to accurately predict the future due to the inherent uniqueness of the time we are living. As a speculator in the financial markets, sports arena, and poker, it's my experience the answer lies somewhere in between. For me, the ability to extract alpha is how well I can ascertain what qualitative aspects are unique and execute a strategy from there.
Two sayings which are both contradictory and complimentary:
"Past is not prologue" "Those who do not learn from history are doomed to repeat it"
Gary Rogan writes:
There are many ways to use the past, such as:
1. Under similar circumstances, Security A behaved a certain way during a statistically significant percentage of the time. I will therefore bet that Security A will do it again under similar circumstances.
2. In the past, a certain class of securities had a certain trajectory under similar circumstances. I will therefore bet that this new Security B, which seems fit to be a member of this class, is statistically likely to follow this trajectory close enough to bet on.
3. In the past, when people were this excited/depressed/confused you could bet with them/against them and make money. Let's do it again.
4. The past rarely repeats under these circumstances. Let's bet against the past.
I'm sure there is an infinite variety of similar observations. Yet in every case the past was used SOMEHOW. There is nothing but the past as the basis for human knowledge, and that's why I was so fascinated by Larry's statement, especially because he is a master of his game.
Craig Mee writes:
Running a stop with any position, regardless of the backtest, is both logical and prudent.
Stefan Jovanovich adds:
When the British and French were forced to give up their remaining military strength in the Arabian Peninsula and the eastern Mediterranean - abandoning the base in Aden, being forced to withdraw from their assault on the Suez Canal, the U.S. did not replace them on the ground. The great fear was that "the loss of the Canal" would result in "the oil weapon" being used against "the West". The actual result was the development of supertankers that by-passed the Canal entirely and increased by an order of magnitude the ability of the oil exporters to ship their crude to Europe and Asia. At the height of the Suez crisis the inflation-adjusted price of crude (using the 1947 nominal price of $15 as the baseline) rose to $18 a barrel - higher than it had been during the Korean War. A decade and a half later - even as U.S. supplies went from 40% of world production to 10% - the inflation-adjusted price fell by nearly a third, hitting a low of $13 in 1972 after production began flowing from the North Sea discoveries.
I find myself wondering if the U.S. eventual withdrawal from Afghanistan and the withdrawal from Iraq already largely completed will not have the same paradoxical effects as the Anglo-French withdrawals did. I realize that this question is completely irrelevant to the questions that anyone trading in commodities has to answer; but those of us in the bleachers are interested in what the professionals on the field think will be the effects of the closing of America's 25-year military misadventures in Southwest Asia.
Larry's maxim: "it never happens quite that way again" - certainly applies to political history. This is the second time in my lifetime that the American public has lost its belief in the virtue of our allies. Last time they were wrong; this time they are right.
Best book of the year was "Myth of the Robber Barrons", second best one, about finished, is "Destiny of the Republic" its about much more than crazy Charley Guiteau…a historical thriller; learned a great deal.
The men's marathon record was broken this week end at the Berlin Marathon, pending ratification, by Patrick Makau with time of 2:03:38. Beating the old record Sept. 2008 record by Haile Gebrselassie by 21 seconds. The record has been broken with some regularity since the mid 1930s when the depression caused men to look for any way to make a living or bolster their reputation to get and keep a job.
It has occurred to me that the record tends to be broken when the economy is in long term trouble and when the young are having hard time finding opportunity elsewhere. The record broken by decade is as follows: 1930, 3; 1940, 1; 1950 6; 1960 9;1970 3; 1980, 5; 1990 2, 2000 4, 2010 1st last weekend. Marathoning is a tough way to make a living and the competition, at least in my time, has appeared to get stronger as the stock market tanks and gets weaker as it is about to take off. I missed my opportunity by not taking up serious marathoning in 1992 but waiting till 1996, 1992 was when I was at my peak and the competition was weakest.
Below is a comma delineated file with the month the record was broken, the % change in the Dow Index 12 months prior (counting month record broke), and the 12 month after % change in Dow index.
Here are some key statistics:
12 month prior avg. 3.22% stdev 13.64% count 34 negative count 15
12 month after avg 13.59% stdev 21.06% count 33 negative count 7
Student T test 2 different stdev one tail, 1.14%
It would appear that the marathon tends to be broke at the turning points in the Dow, however with a wider deviation than prior and the last 2 times this did not work. A similar result occur taking out all but the first occurrence when the record is broke within 12 months of the last time, except the student T test is higher due to fewer occurrences.
Month Record Broke,Prior 12 month % change in Dow,Next 12 month % change in Dow, New Record
9/1/2011 -0.15%, ?, 2:03:38
Kim Zussman queries:
Russ, in your opinion, will the 2 hour mark be broken in our lifetime (you and I are about the same age)?
The 12 records since 1980 have dropped in a roughly linear fashion, which if continued extrapolates to about 2035
Russ Sears replies:
If you would have asked me in the 90s about this, I would have said no way, they are beginning to form an asymptote. However, I believe sport science has made considerable progress in 2 areas critical to lowering this record below 2 hours.
1. There is a much better understanding of the altitude effect has in endurance training and how to maximize this effect.
2. There are now much more creative ways to exercise more with less detrimental stress but maintaining the positive stress and exercise specifics. See zero gravity treadmills and water running treadmills for example.
And perhaps it simply is competition creates its own opportunity and it is clear to me we are in a new competitive era for distance running.
Hopefully it will continue for the rest of our lives. But not because the economy will continue to flounder. Especially for the young, ambitious and talented.
Larry Williams comments:
The math is all a runner has to do is knock 8 seconds a mile of his/her pace. Much easier said than done, but it's a goal within reach. 2:00:00 will be broken in next 10 years is my forecast. Mostly because of what Russ said–better training habits and knowledge. With science, a runner with lots of personal angst will break through. Must have personal angst (see current frank shorter article in Runners World) to get through the pain.
I just finished The Mighty Atom. It was a delightful read as well as educational on several levels.
Chris Tucker adds:
This reminds me of a passage in The Psychology of Risk: Mastering Market Uncertainty, by Ari Kiev, M.D., (John Wiley & Sons, 2002) pp. 39, 40:
Commitment is an example of what Joe Greenstein, a circus strongman in the early years of the twentieth century, believed was necessary to overcome what he called "impossibility thinking." He believed in a Life Force that we all have but fail to activate because we are constantly thinking, "I can't do that. I'll hurt myself."
According to Greenstein, the little voice in you - that instinct for preservation - does not give us an accurate picture of our capabilities. We all have mental and physical abilities beyond our own estimation, but to realize them the mind must be deconditioned from impossibility thinking. Only after this deconditioning does it become possible to do what you will.Greenstein believed you could do almost anything if you applied your mind and body to the task with enough diligence. The critical step to take is to overcome the instinct for self-preservation, which inhibits action. To do this, he believed, it is only necessary to become totally focused on the event at hand, with no reservations and fears of anything untoward happening. In traders, this instinct shows up in the form of such things as mental accounting and loss aversion.
Nothing could more vividly illustrate the importance of belief in a favorable outcome without reservations or fear than the large number of people who were able to run sub-four-minute miles *after *Roger Bannister broke the magic four-minute mile barrier on May 6, 1954. Until that time, the four-minute mile was no longer a vain, exaggerated dream but an attainable goal that could be reached by any runner who was capable of overcoming the pain, adversity, and anxiety involved in reaching it. Once the barrier had been broken by Bannister, the event itself suddenly became relatively easy. By the end of 1978, as many as 274 runners had broken through the "magic, impenetrable barrier."
Kiev then progresses into an interesting discussion about the nature of commitment. Highly recommended book along with his Trading to Win: The Psychology of Mastering the Markets, John Wiley & Sons, 1998
A rather shocking study which shows that investors in mutual funds receive a much worse return than the actual return shown by the mutual fund through putting most of their investment in before the mutual fund goes down and least of their investments when mutual funds before the mutual fund goes up. The actual underperformance seems to be of the order of 3 percentage points of return a year, i.e, 6% versus 9%, with sector funds and specialty funds and growth funds showing much greater underperfomance. This must be a pretty good indicator of when to go against a particular sector.
Steve Ellison writes:
I read Mr. Swedroe's book Rational Investing in Irrational Times. This is very interesting data, but it undermines his main message, that the market is efficient and even professional managers can't beat it, so you should diversify and keep costs low by buying index funds. If mutual funds favored by the public underperform by a wide margin, something else must be overperforming. One might be able to beat the market by simply avoiding the hot funds and their favored stocks, like Bacon's technique of betting on all the other horses besides the overpriced favorite.
Mick St. Amour writes:
A great contrarian indicator. The retail investor is often guilty of chasing returns and as you can see this is a performance killer.
Larry Williams writes:
Aha, mutual funds are for the masses, while the elite managers of money: Cohen, PTJ, Dalio– are the winners for their clients.
But hold on a moment here. Lots of professional managers have beat the market for many, many years. It can be done, and it is being done. But not all can do it. Just like not all teams will be in the final four, and while luck is part of the game, in the end skill carries the day.
Copper finally succumbed last night, and whether it leads the equity market, or equities lead it, they tend to mimic each other pretty well over the March/April reversal period for the next several months. If they hold this relationship, copper will now have a bit to make up to give this duo strength.
Anatoly Veltman writes:
1. it happened quite abruptly in North American session.
2. mass media explanation: potential premium fuel costs dim prospects for industrial demand.
3. over the years, I've often seen other metals follow next day - although there is no fundamental link. It always was: like players were too busy knocking one market down today; and they switch to the next market next day. Which will be interesting to note tomorrow, as one important nouveau element (cross-market algorithms) should have kicked in already today (?).
Larry Williams adds:
Dominoes is the next game?
The Ditch Digger's Daughter is the best book I have read in many, many years. I swelled up, reading it, choked up and cried.
London & Hong Kong Traders:
Open interest has fallen almost 30%, but gold has only dropped 6%. Normally if you are a short in a market and you start to have an asset correct because of significant liquidation, you will see a precipitous drop in price. Given the sheer volume of contracts that has been liquidated, we should have seen a massive correction in gold. Instead it has stayed incredibly strong.
Larry Williams writes:
Open interest rallies or declines must be put into perspective of who is causing the OI move, which group of traders…to just mention OI w.out background is not a full meal.
Rocky Humbert writes:
I would add the following to Larry's comment — which is something that makes today's markets quite different from decades past — and which changes the character of OI and CFTC commitment of traders data.
In a futures market, the long positions must EQUAL the short positions. Hence it's a zero sum game. This is not true in the physical gold ETF!!!
For example, when a long liquidates his gold futures, the short is also liquidating his short position. And the open interest declines. Larry and Anatoly believe that there is predictive information in this.
However, in the gold etf, there is no genuine reduction in open interest. Instead, when an ETF long sells his gold ETF position, the gold leaves the ETF system — and the physical gold finds its way to another holder outside of the ETF system. Because gold is (mostly) not consumed by commercial end users, the gold remains in existence in perpetuity. In theory, in a futures market, if the open interest is zero (and remains at zero), the price of a commodity will not change. The price will just sit there. However, in the case of physical gold, the open interest can never be zero — since the physical bullion will continue to exist (whether inside or outside of the ETF system.) This means that gold (whether entering or leaving the ETF system) has a quasi-permanent open interest.
None of this is necessarily predictive of the gold price– however, it's important to understand that the CFTC data on gold futures open interest misses this nuance.
Great points, thanks for making them.
Who are the players in the ETFs? Relatively small specs, I assume.
And on gold consumption it is consumed, not like wheat, but the physical inventory is turned into rings and things so the inventory needs to be replaced my commercial users. Commercials do take delivery.
Trading is hard with small kids around. I remember when I was trading copper. I'm long, and my actress daughter gets a bloody nose (of course she dramatizes it, she's a born actress). I forget the trade in copper, stop the bleeding….and….go back to see a 45,000 loss in copper. It would have been cheaper to med-evac her to the hospital.
Victor Niederhoffer reminisces:
Twenty five years ago I lost half my wealth between the first and second games of a racquetball game with Reuben Gonzales.
Craig Mee writes:
Shocking…It reminds me of a local interest rate trader, heavily
short, who went for a "quick" haircut, next door to the exchange in
Sydney…. BOOM. Reserve bank unexpected rate move …house wiped out.
An anonymous contributor writes:
They [ed.: i.e. the French] had then learned how easy it is to issue it; how
difficult it is to check its over issue; how seductively it leads to the
absorption of the means of the workingmen and men of small fortunes;
how heavily it falls on all those living on fixed incomes, salaries or
wages; how securely it creates on the ruins of the prosperity of all men
of meager means a class of debauched speculators, the most injurious
class that a nation can harbor,—more injurious, indeed, than
professional criminals whom the law recognizes and can throttle; how it
stimulates overproduction at first and leaves every industry flaccid
afterward; how it breaks down thrift and develops political and social
immorality. All this France had been thoroughly taught by experience.
Everything was enormously inflated in price except the wages of labor.
As manufacturers had closed, wages had fallen, until all that kept them
up seemed to be the fact that so many laborers were drafted off into the
army. From this state of things came grievous wrong and gross fraud.
*- Andrew Dickson White, “Fiat Money Inflation in France”, How it Came, What it Brought and How it Ended*
Stopped off to see a buddy in a large trading room here…apparently no one can speak above a whisper. It seems a strange way of doing any trading…it is imperative that one learns to curse at himself (often and with gusto) to do well in this business.
Mr. Krisrock writes:
They use buttons…one beep ok, two beeps get the fades out of my way, three beeps with one following get me a single coffee, two beats two coffees…
Jeff Watson writes:
Interesting about the character of many trading rooms. While they might play music as background, the players are generally as quiet as church mice, concentrating very hard. On the other hand, at my tiny room, because of my floor based background, anything goes. Rabelaisian jokes, potty humor, pranks, lots of noise, telephone calls, nothing bothers me and I encourage discussion, stories, jokes, etc. I guess it all depends on how one was brought up. Floor guys are just different, much more animated and aren't as cerebral as screen guys. Just because there is a lot of money involved doesn't mean that one can't have a sense of humor, or gallows humor;Plus, it might be better if one can trade with distraction, much like floor traders had to deal with,, but this should be tested.
I don't think one becomes good at trading until we have been beaten so much that we no longer fear the beast…once you learn how to take any shot the market give you, success comes so much easier.
Jay Pasch replies:
There is wisdom in this post; it also emphasizes the importance of having enough skin in the game to experience its sensitivities especially when it comes to turning points– turning points start to hurt, they frustrate you, they wear you down, they rub you raw to a point where you think you can't take it anymore, to a point where you question your methods, why you trade for a living, to a point of throwing in the towel– it is then that the trader needs his perseverance the most and to stay awake.
Victor Niederhoffer asks:
What are the turning points and how can they be predicted? That's a good way of
trading I think. A turning point and run are pretty much the same with
proper definitions as a start.
Jim Sogi writes:
There are enough niches and styles in markets that a person can find one in which his own weaknesses create the least problems.
Rocky Humbert writes:
Craig wrote about Cyclone Yasi a few days ago. This is a monster storm, and may hit Queensland sugar (and other ag) production. It will be a couple of days before the markets "digest" the results.
Spot sugar is already in tight supply. If the Queensland crop is damaged, it could push up out-month sugar prices, and this might even feed into higher corn prices (i.e. corn syrup). Conversely, the ag markets are already extremely "hot," and we've not seen a bearish headline for ages.
Earlier this morning, the chair asked a most relevant question: "what are turning points and how can they be predicted?" The chair has also previously written that "reversals are more lucrative than trends." Over the past 12 months, sugar is up 65%, coffee is up 76%, cotton is up 125%. If reversals are indeed more lucrative than trends, I'd love to figure out when I should reverse these positions, since I keep wasting money on my hedges. Sadly, the only turning points that I ever see are with 20:20 hindsight.
Vince Fulco writes:
There seems to be a prevailing reasoning in the trading world that "reversals" or "turning points" are something which must be predicted– while trading "trends" is something which is not predicted, but merely, reacted to. The latter, not requiring "prediction."
I think that prevailing reasoning is false. Being a trend follower still requires one to predict in the sense that he is predicting the trend will continue. Both approaches require prediction. (Similarly, a non-directional approach, a market-neutral approach, say, writing butterflies, is, by the same reasoning, requiring prediction in that one is predicting the market will stay sideways, or at least not go into a protracted trend).
So my question to the site is this: Is it possible therefore to trade and not predict?
Gibbons Burke comments:
Method one: Book your profits in your mind, don't treat it as "house money" and decide right now, for each market, how much of your money you are willing to give back to the markets. Draw your line in the sand and let the market take you out at that point. If it takes you out and then goes back to make new highs, consider maybe getting back in.
Method two, which I prefer: take half of your positions off the table, cash in the chips and reward your self for being right. Let the rest ride with a stop set at the point determined by method one. If you keep being right, and start feeling like you want to reward yourself for being right again, take half off again. Keep raising your stop on the remaining positions to lock in your profits, and let the market take you out when it feels like doing so. And given the magnitude of the trends, the likelihood is that when it decides to take you out, it will keep going in lobogola fashion.
I've had this very argument with a well known trend follower/leader on his Facebook page a couple of times. He keeps insisting that trend followers are superior to the other species of traders because they don't make predictions. But my contention is that trend followers are simply deluding themselves if they think they aren't making predictions.
They are predicting that when they get a trend following signal that the market will continue in their direction by a magnitude that is more than twice the size of the risk they are taking on. They predict that this will happen maybe 20% of the time, and that when they catch those big moves they will make up for all the psyche-destroying losses of which they predict their method will keep small.
It is a different sort of prediction, but it is nonetheless a prediction.
Farm Journal had a good article on the tightening wheat supplies.
Their contention that milling quality wheat is getting scarce (but not to 2007 levels) and the market reflects this, resulting by the ever expanding Chicago/Mgex wheat spread. The resulting scarcity of high protein milling wheat has caused all contracts of Minneapolis Wheat to trade at a premium to Chicago. A few months ago, Chicago Dec 2011 was trading at a 5 cent premium to Minneapolis, which was an anomaly as every other contract of Minneapolis wheat was at a premium, except Dec 2011. As Minneapolis wheat normally trades at a premium to Chicago (Quality trumps everything and transport and storage are basically a wash), Chicago trading at a premium to MGEX can mean a good trading opportunity, but one must be very careful. It can also mean ruin if the mistress of the market continues her irrational behavior as evidenced by the Dec 2007 debacle that bankrupted many traders caught on the wrong side. Right now, in the wheat market, the seven wild cards are the next crop (there is a wheat crop harvested somewhere on the planet every 3 months), China's demands, our other exports, what Russia will do, the dollar's value, and acreage yields, and numbers of acres planted.
(As a side note, the government's directives might result in a reduction of wheat acreage like 2007 in order to plant more corn for ethanol, but this is speculation and not fact as of yet) However, supplies of Chicago's lower protein wheat are not very tight as evidenced by the front month, March, trading at a 30 cent discount to May. If there was a tight supply, the front month would trade at a much higher price, possibly even a premium, to shake some wheat out of storage to accommodate immediate needs.
The milling wheat on the other hand is only trading at a 6 cent discount in the front month suggesting much tighter supplies. From a practical standpoint, I am noticing a substantial increase in the price of pasta at the grocery store, and fewer markdowns on a retail level.My milling contacts also are mentioning substantial price increases in the near future. Still, this upward drift of the entire wheat market is rather confusing as the fundamentals somewhat support the rise, but there's something else going on beyond the mere fundamentals. I'll leave it up to others above my pay grade to ascertain and explain the intricacies of the market.
Meanwhile, I will try to make it safely to port without any damage. My mea culpa here is that 8 months ago I was of the opinion that while there might be a slight upward drift in the wheat market, it would be orderly and negligable and I saw no real rally unlike other sagacious members of the list. I even reported this on Daily Speculations, on Jan 26th. Larry Williams was the hero of the day when he said, "Wheat is set up to rally." Kudos to Larry, and a hairshirt for me…. I was so wrong, but still providence was with me when I decided to not try to buck the ever rising market and keep my longs.. Thinking of all the times I have been completely wrong, (and I keep very exacting records) it's amazing that I have managed to stay afloat and am not working the overnight shift at the 7/11.
Larry Williams writes:
I hate to disagree with a trade journal, but my stuff is bearish on wheat at this time.
There has been a noticeable drop in Gold open interest. Since open interest is definitely declining, look at the players. Who is doing the buying here? The Commercials. Small Specs and Large Specs have been liquidating.
You can get as-reported earnings for the S&P 500 from 1988 on at Standard & Poor's website.
Using 12-month trailing earnings for each year's September quarter (the last that would be known by Dec. 31) to calculate an E/P ratio for the S&P 500 as of Dec. 31, I get a somewhat positive correlation of trailing E/P to year-ahead returns with t=1.10, R sq=0.057, p=0.29, and N=22.
Larry Williams writes:
My model for the DOW suggests a 12.25% growth for the year, slightly above the long term average growth.
For the S&P, I get 10.6 % barely above the long term average.
Bruno Ombreux writes:
There are two things I don't like in P/E or E/P studies.
1) Your regression is in the form:
-1 + P(t)/P(t-1) = f[P(t-n)/E(t-n)] + e we have the same variable on both sides, and even if it is lagged I am not sure standard regression is OK to handle this type of formula. Just to give you an idea, multiplying both sides by P(t-1), it is actually P(t) = P(t-1) + P(t-1)* f[P(t-n)/E(t-n)] + P(t-1)*e
This is certainly amenable to study, but not with the standard regression toolbox.
2) Price is more volatile than earnings. There is a subtle bias introduced by the fact that over the estimation sample, high P/E will be naturally followed by lower P/E, and vice-versa. This is a bit like regression to the mean but more subtle. This can lead to spurious mean-reversion.
Phil McDonnell adds:
The issue is not really the dependent variable. It is using the Shiller variable with its serial correlation. One way to use the Shiller variable would be to take every tenth month. That might work but you would have one tenth the data. You still might have the Holbrook Working flaw because of the averaging. The averaging also leads to the Slutsky-Yule effect which creates spurious sinusoidal artifacts in the adjusted variable when no such sinusoidal effect is actually present in the original data..
One notes that:
1. NY Commuter rail fares will increase by more than 11.1% on December 30th (for the Harlem and New Haven lines.)
2. NY Commuter rail fares will increase by more than 14.3% on December 30th (for the Hudson line).
Assuming a brisk walking pace, a Westchester County resident can make this round trip trek in about 12 hours. In contrast, a round-trip peak ticket costs $28.50 and train-station parking costs $6.50. Hence, a day-trip into Manhattan costs $35.00 per person. Assuming a 40% marginal tax rate (State & Federal Income Tax), the pre-tax cost becomes $58.33. This is about $4.86/hour.
It's therefore a relief to know that the New York State Minimum Wage is now $7.25/hour. So it still pays to work.
Victor Niederhoffer writes:
One would have to adjust Mr. Humbert's calculations based on the age distribution of the population. "One senior ticket and one child," Aubrey always says when the conductor comes. That's Keely's 7 bucks for me, but my walking pace has slowed, (as witness my failure to pass the California test for the DUI). Say I am at 3 miles and hour. It would take me 17 hours to get to Manhattan for my 50 miles. (I believe Elonra Sears, the lady squash champ, would do it in 16). If my time is worth more than 50 cents an hour or so, it pays to take the train, assuming I would not make losing trades. (In the past, when asked to do chores, I could always tell Susan, that the chore cost me 1000 or 5000 an hour when I could make money with impunity, but now that doesn't work and Susan often says that I'd save money by washing the dishes or changing the light bulb, or shoveling the snow.)
Russ Sears writes:
A couple guys come to mind when you talk of going 50 miles a day to work.
Legend has it that Bill Rodgers headed for nowhere, working in a morgue delivering bodies, when his motorcycle was stolen. He started to run everywhere. This helped him to start running again after stopping after college track. He also was smoking before this. And he is the only guy I have heard of that doing more than 150 miles per week actually strengthen him. He topped out at 200 miles per week 16 in the morning 13 in the evening.
The other guy is Dr. Horton, who was a Phys Ed Professor at Liberty. He set the record for running the Appalachian Trail. He averaged I believe, near 50 miles per day. He had line up Churches to help him throughout the course. He would meet them at points most nights, so he could eat hardy and sleep and then next morning drop him off at the same point. It was getting to the meeting points that added to the distance to the 2,200 mile course. (now I hear 2 other guys have broken his record of 52 days) This was very tough on him and I heard from my CC coach that it took him over 2 years to shake the mental depression such distances placed on his mind and body.
At 50 miles: plans would have to be made to have plenty of liquids along the way some light food. then latter eat and eat hardy and well. A mile burns roughly 100 calories, for average weight guy. Plus the normal 2000 calories, would require about 7000 calories a day. Phelps is said to eat 12,000 calories a day.
One summer in college, I lived on a nickel to save for the next years tuition and road a bike near 30miles a day for a couple months to work. 100 mile days are normal for serious bicyclist.
Henry Gifford writes:
I used to compete in and win, 24 hour bicycle races– ride as much as you like, rest as much as you like. Some wimps even took naps.
At the level we were at, consuming enough food was a deciding factor during a race, and buying it was a major expense, for a race and at all other times. One year someone handed me up candied pineapple, which I had never eaten. I barfed, but still rode as hard as I could, but I was like the car in the Indy 500 with the torn gasoline fill pipe from a sloppy pit stop exit. I was able to keep up, but couldn't refuel, surely coudn't have finished or won.
Someone helping run the team knew to feed me boiled potatoes, after which I was good to go. I ate everything on the next lap around.
Larry Williams wrote:
I also found boiled potatoes to be the key, with salt, to correct food for ultra marathons.
Keynes was interested in markets, and did pretty well. What about Hayek?:
"Keynes was another Kelly-type bettor. His record running Kings College Cambridges Chest Fund is shown in Figure 2 versus the British market index for 1927 to 1945, data from Chua and Woodward (1983). Notice how much Keynes lost the first few years; obviously his academic brilliance and the recognition that he was facing a rather tough market kept him in this job. In total his geometric mean return beat the index by 10.01 per cent. Keynes was an aggressive investor with a beta of 1.78 versus the bench- mark United Kingdom market return, a Sharpe ratio of 0.385, geometric mean returns of 9.12 per cent per year versus Ð0.89 per cent for the benchmark. Keynes had a yearly standard deviation of 29.28 per cent versus 12.55 per cent for the benchmark. These returns do not include Keynes (or the benchmarks) dividends and interest, which he used to pay the college expenses. These were 3 per cent per year. Kelly cowboys have their great returns and losses and embarrassments. Not covering a grain contract in time led to Keynes taking delivery and filling up the famous chapel. Fortunately it was big enough to fit in the grain and store it safely until it could be sold. Keynes emphasized three principles of successful investments in his 1933 report:
1. A careful selection of a few investments (or a few types of investment) having regard to their cheapness in relation to their probable actual and potential intrinsic value over a period of years ahead and in relation to alternative investments at the time; 2. A steadfast holding of these in fairly large units through thick and thin, perhaps for several years until either they have fulfilled their promise or it is evident that they were purchased on a mistake; and 3. A balanced investment position, i.e., a variety of risks in spite of individual holdings being large, and if possible, opposed risks.
Jeff Watson writes:
I could not find much about Hayek's investment performance and speculate that his work in getting a Nobel Prize and publishing seminal works probably attenuated any desire to actively play in the market. Granted, Keynes was a genius……completely wrong about everything, but a genius nonetheless. I notice no comment from his fans on the left about his legendary Anti-semitism, his frequent use of the N-word when describing American Blacks, and his dismissive attitude towards Russians, and other Eastern Europeans who he thought to be the unwashed masses and very ignorant. Still, in his complete wrongness, he provided a very bright beacon for those of us who wish to pursue the correct course. Keynes is our own perfect fade factor, a Douglas "Wrong Way" Corrigan of economics.
Larry Williams writes:
What an article on this that does not mention Ralph Vince. Oh, I get it…much of his comments are lifted from Ralph, so why let people know he exists? Trade kelly and you are doomed to die.
Ralph Vince responds:
Thank YOU Larry. A couple of things on this.
1. Whenever people start talking "half Kelly," or other ad-hoc locations on a dynamic curve (with respect to the number of plays) I realize they don;t know what they are talking about. It doesn't mean they aren't good mathematicians, they just don;t understand their material well enough. Ziemba has been doing that for years.
How can a man look at the curve and not begin to discern the nature of it beyond that???
2. The "Kelly" Criterion answer is NOT what any of these guys thought it was. It is NOT the optimal fraction to invest. It is a leverage factor — a number not bound between 0 and 1 but 0 and + infinity. Thus, if you treat it as a fraction, you will inadvertently be using a fraction that is way beyond optimal in trading.
3. Once you discern what the real optimal fraction is to invest (and you won't get there with the Kelly Criterion) then you can make intelligent decisions on what value to use as a prediction of where the optimal fraction will be in the forthcoming periods.
All of that if you want to be growth optimal. Go ahead, have at it slugger. The REAL benefit to understanding the nature of the curve of the optimal fraction (not to be confused with Kelly's misguided criterion) is that you can use it to satisfy OTHER objectives aside from the incredibly aggressive growth optimal one.
I don't claim to be the mathematician any of these guys are. But I know I understand this material better than all of them combined.
And I have the real-time track record to prove it.
December 20, 2010 | Leave a Comment
The mind set of a fighter is not much different than that of a trader. A great read: The Fighters Mind: Inside the Mental Game by Sam Sheridan.
Does anyone have good thoughts or conventions on how to factor entry/exit slippage in futures markets? I am trying to find a general rule and google searches of academic papers result in not a lot and what I do find is a bit in excess for my needs.
My thought was to assign a rule such as when my entry/exit sizes combined total <0.50% of the average daily volume it would be safe to assume 1 tick worth of slippage per side. Ex. bid/ask 500/500.25 then assume sell/buy price at 499.75/500.50 (assuming a market that trades in 0.25 increments). Granted this would not happen in all instances but very likely not all fills would be at disadvantageous levels so the average slippage over time of this magnitude seemed fair. The scale could be increased as the percentage of average daily volume increased (i.e. <1.5% assume 2 ticks, <3% 3 ticks, etc.). A time of day factor might also be useful (i.e. pit vs electronic hours) as the liquidity is certainly better at peak trading times. Without very robust data this is difficult to test/determine so I wanted to see what the specs think.
Larry Williams comments:
It depends on the market and also on news of the day. Sometimes there's no slip at all other times it's massive so a decent trader would factor such things into his or her trading style; the problem is mechanical systems have trouble doing that which an individual can do.
Phil McDonnell writes:
There is no slippage with limit orders. Either you get your price or do not trade. If your testing can make a decision based on previous time periods. Then a simulated limit order can be placed. If the low for the next period is below the limit then you got filled on a buy. If the high is above the limit then you know you got filled. The tricky case comes in where the limit is exactly equal to the respective extrema. In that case I would suggest two ways of testing.
1. Assume 50:50 chance of a fill, take them at random.
2. Assume no fills at the exact bottom tick or top tick.
I recommend doing both tests. The 50:50 is probably realistic, but the no fills scenario tells you if the system might depend heavily on getting these top and bottom tick trades.
Larry Williams writes:
Phil's point is why I buy a small partial position on stop, then rest on a limit. This has helped my short term trading a great deal.
Jim Lackey comments:
One should always be a gentleman here. How to trade is the easy part. How much to trade, when to trade and when to exit a trade are the difficult questions.
You can get partial fills, sub penny or fractional orders. Your limit is like a big round number for order sniffers HFT's other traders once made public. Everyone knows we should scale into positions.
When volatility is very low and liquidity is very high it's much easier to enter with out slippage.
The original mechanical question should be adjusted for volatility. I remember 2 times in year 2000 and again in the fall crash of 2008 where I used basically market orders with a price limit. Simply bidding through the current price to start + X ticks to start a scale or to end one. Back in those days the markets might rally 3% in an afternoon so you had to get some on.
Opens and closes are best to use at the market orders. However for the mechanics market at open or closes can be the best or the worst price of the day then your in either deep trouble all in on the high tick, a trend down or lucky you caught a trend day up.
The problem has been that since the markets have become more mechanical its the big up opens that continue to go up and the down opens that have gone straight down. These are order ladders and the moves are over by 10am. It was confusing to see the markets gap big and the entire day over by 10 or 11am. Now we are used to it…perhaps time for a change.
What is wild is to see the govie bond markets move a couple percent a day and the SNP move 1% and the ebbs and flows between stocks bonds and those indexing commodity etf's for investment over the next 100 years are guaranteed to lose.
Chris Cooper writes:
I often have the problem called "adverse selection" with my limit orders, in forex or futures. This is not a problem with small orders, but when your order is larger, you may (frequently) see a partial fill, then the market turns around in your favor and you make a profit. But the profit is not as large as it would be if your order had been filled to completion. The times when your order is completely filled, the market has moved through your price point and past it, usually. You are sitting on a loss if marked to market, and that loss is for your entire size. In other words, you get the bad fills when you really want to be filled, and the good fills when you don't want them (in retrospect). It's a big problem for short term trading.
Similarly, in markets with a long order queue such as ES, by the time your order is filled the market has pretty much moved past you and you are sitting on a loss. Unless you can play games with pre-positioning your order in the queue, you will always see this slippage with limit orders.
Jonathan Bower writes:
You "could" avoid posting your limit order for bots, hfts, etal especially if it is of consequence (size) by working it semi-"silently" in an attempt to not get "screwed". One of the earliest execution algorithms was the "iceberg" where a limit order could be placed in (user defined) pieces, once taken out another one will replace it until your order is filled. Of course this is easy for those who want to to sniff it out. More recently you can add variance to your lot size firing in somewhat random orders as part of an iceberg. Or you could set a timed order to release a limit order (or combination of orders) based on a print, bid, or ask with what/if OCO ammendments, etc etc etc.
The bottom line if you need to be and can be creative with order execution especially in illiquid markets. Of course being right about the trade idea is still the hardest part…
Chris Cooper adds:
GLOBEX does not have liquidity that is as good as the forex interbank market, especially in the Asian hours. If you are trading smaller quantities, in New York hours, then futures are a better choice than forex.
If you are not getting filled when you should be, then your forex broker probably has what is called a "last look" provision in their system. This allows the liquidity providers a chance, say for 30 seconds, to decide not to fill your limit order. This has been an annoying problem for me in the past. The solution is to switch to a broker without this provision. It could also happen if your broker is not an ECN, and in that case you should find a new broker.
Jeff Watson adds:
There were (and still are) some people in the pit who made an excellent living with just the ability to tell when the market is going from quarter bid to quarter sellers. That, in itself is a huge edge.
November 21, 2010 | Leave a Comment
It has been three months since the confirmation of the Hindenburg Omen, and the S&P 500 is up 12% since then.
Larry Williams writes:
True on H Omen sell but….there was what I call a H Omen buy on 8/27 and 8/30. I am writing a paper about this.
Jim Sogi writes:
Steve's point is why I believe that quantitative price analysis must be augmented with game theory such as Chair's infrastructural and and natural observations, but also with Mr. E's macro observation and political gamesmanship. This helps with the cycle analysis.
Trend is the basis of profits; no trend, no profits.
Trend is a function of time. 3-5 hours does not allow for trend moves of any real magnitude; day traders play with a stacked deck. The dream of daytrading is in fact a nightmare for most everyone. It can be done, but not for massive profits.
Nick White writes:
There is an excellent discussion of the dynamics of this point in the gentleman from Amioun's first book.
I quote / summarize / paraphrase at length:
Take a regular dentist. A priori, we know he is an excellent investor and has an expected annual return of 15% over t-bills with a vol of 10%. Dentist builds a trading room in his attic, deciding to spend every day watching the market and drink cappuccino. He buys a bloomberg and lots of expensive PC's to automate his trading etc etc. His 15% return with 10% vol per yr translates to a 93% probability of success in a given year. But seen at a narrow time scale, this translates into a mere 50.02% probability of success over any given second. OVER THE VERY NARROW TIME INCREMENT, THE OBSERVATION WILL REVEAL CLOSE TO NOTHING,. Yet the dentist's heart will not tell him that. Being emotional, he feels a pang with every loss, as it shows red on his screen.At the end of each day, the dentist is knackered and emotionally drained Minute by minute examination shows that each day (given 8hrs / day) he will have 241 pleasurable minutes against 239 unpleasant ones. These amount to 60,688 and 60721 per year. Given that an unpleasurable minute is worse in reverse pleasure than the pleasurable minute is in pleasure terms, then the dentist incurs a large defecit when EXAMINING HIS PERFORMANCE AT HIGH FREQUENCY. In contradistinction, imagine the situation where the dentist examines his portfolio only upon receiving the monthly account from the brokerage house….67% of the months will be positive, he gets only four negative pangs of pain per year, and eight uplifting ones. THe efffect is magnified at the annual level where, for the next 19/20 years, he will have a pleasant experience looking at his annual statements.
1 qtr 77%
1 mth 67%
1 day 54%
The net net of all this is as follows:
If we look at the ratio of noise to non-noise then you get the following. Over one year we gets .7 parts noise for every part of performance. Over one month we get 2.32 parts noise per part of performance Over one hour, 30 parts noise for every part performance Over one second. 1796 parts noise per part of performance. Therefore:
Over short time increments, one observes the variability of the portfolio, not the returns…you just see variance and little else. This is emotionally difficult to parse…even though at any moment you see a combination of both variance and return, your brain can't tell the difference. This explains the burn-out rate amongst people who constantly expose themselves to randomness (especially given greater effect of negative experiences than positive experiences).
Kim Zussman writes:
The analysis is flawed:
1. A regular dentist should not day trade because of opportunity cost and squandering what he paid for his profession's barrier to entry
2. One can only know their returns ex post. If you could know them ex ante there would be no psychological issues - fear comes from uncertainty.
3. Irregular dentists are another story
Rocky Humbert writes:
Kim writes: "If you could know them ex ante there would be no psychological issues– fear comes from uncertainty."
This is not correct.
Fear is a pre-wired response in most people, and can have little to do with the rational analysis of a situation or uncertainty.
Example 1: While waiting to mount an extreme amusement park ride (roller coaster etc), the symptoms of fear (sweating, elevated pulse, etc.) are a genuine physiological response regardless of the fact that the odds of injury on the ride are miniscule.
Example 2: If you are engrossed while watching a horror movie, your limbic system kicks in … just as if you were being chased by the Blair Witch.
Example 3: Phobias are real, and irrational. A person who is afraid of flying is not interested in the fact that his plane will almost certainly not crash. Yet, people are not afraid of driving to work — even though the odds of dying during a commute are much higher than on a cross-country flight.
The emotional fear/greed response to P&L mark-to-market are real– and I believe genetically pre-disposed. They are analogous to phobias and obsessive-compulsive disorders. Hence, I believe Nick's post about timeframes is extraordinarly profound and accurate.
As far as I have ever been able to ascertain, Larry Williams was the first to attempt to apply the Kelly Criterion to outright position trading, and the first to openly discuss it. His pursuit in this regard not only was my initial immersion to the ideas, but he funded those attempts. Whatever I've uncovered along the way is a product of that — Larry's unquenchable curiosity, fearlessness regarding risk, and willingness to fund pursuits others would never touch.
A couple of points further in the post worth mentioning here because I think the other interested members deserve to have light shed on some misconceptions, some of which are a little dangerous to ascribe to, but are widely held.
"One "plays" forever, or practically forever."
But no one does and no one can, and it is this very notion of there being a finite "horizon," that changes not only the calculation of a growth optimal fraction, but every other metric related to it, giving rise to an entirely new discipline in and of itself.
"If one is somewhat risk averse, one can establish a half Kelly criterion, essentially betting half one's full Kelly bet. This results in a lower probability of one's bankroll halving."
But why "half?" Why this arbitrary number? (Or any other arbitrary dilution for that matter?) Remember, we're dealing with a function that has an optimal point, implying a curve, and it is the nature of this curve that is important to us. Being at different points on the curve has vastly different implications to us. Further, the various and important watershed points almost all are a function of that "horizon" mentioned earlier, i.e. the points migrate about this curve as a function of that horizon. Advocates of a "Half Kelly," or other arbitrary point along this chronomorphic curve (with respect to the horizon and events transpired) are seemingly unaware of the implications of their arbitrarily-chosen points.
"The criterion is to maximize the expected value of the logarithm of one's bankroll."
Yes, that is the Kelly Criterion which, in trading, does NOT result in the growth optimal fraction but a far more aggressive (and dangerous, without growth-commensurate benefit) number. No one seems to understand this.. The number returned in determining the value that satisfies the Kelly Criterion can be converted into a growth optimal number (which I call the optimal fraction. or optimal f) but in and of itself, the value that satisfies the Kelly Criterion is NOT the growth optimal fraction in trading. Incidentally, the so-called Kelly Formulas (put forth by Thorp I believe, and market applications attempted by Larry Williams in the mid-1980s) do NOT satisfy the Kelly Criterion in trading applications, but DO in gambling ones (that is, in trading applications they will not yield the same results as the value which satisfies the Kelly Criterion. The Kelly Formulas do, for dual-outcome situations, return the growth optimal fraction). For more on this I can only refer those interested to the most recent Journal of the International Federation of Technical Analysts 11 (available at admin at ifta dot org) or the 2-day course on Risk-Opportunity Analysis I am having in Tampa Nov 13 & 14 see http://ralphvince.com)
"The biggest issue of application is that one makes many assumptions about statistical distributions, correlations, returns, etc. that are all wrong."
I agree. In a strange, ironic twist to my modest participation to this story, it was (again, but some decades later) Larry Williams (rather recent) insistence of a way to apply what I know of growth maximization in a robust way. As a result of the pollenization of these ideas by Larry, I can state unequivocally that there are clear, simple, mathematical solutions to these impediments — in short, if someone wishes to apply a growth optimal approach to their future trading, these impediments ARE readily surmountable. But be certain your criterion is growth optimality, and be sure you really want to get into the cage and fight the gorilla. Most just want to sit and watch Dancing with the Stars.
Nick White comments:
Dancing with the stars….brilliant and well said.
We're all fortunate beneficiaries of Mr. Vince's investigations into the intricacies of these issues.
Phil McDonnell writes:
Kelly originally wrote his paper based on race track examples with binary outcome. You won or lost with assumed probabilities and you knew the wager size and payoff. So strictly speaking his formula only applies to wagers with two outcomes. Even a blackjack hand has at least five possible outcomes (win, lose, blackjack, double down, split) and not just two so strictly speaking Kelly's formula does not apply. Some people have erroneously tried to modify the binary Kelly formula by using average win size and average loss size to compute. All such formulas are dead wrong. The reason is that, in general, the average log does not equal the log of the average.
As Larry Williams pointed out most people do not feel comfortable using the optimum log approach even if the math is done correctly. I believe there is a simple reason for this. Most people do not have a simple logarithmic utility function. Rather they seek to maximize ln( ln w), where w is wealth. This is an iterated log function and results in a much more conservative ride. I talk about this distinction toward the end of my book. Ralph Vince also has written extensively on this subject using his term optimal f.
There is another issue with simply maximizing returns and that is it may not really take into account risk in a proper manner. It is true that the log function weights the largest loss the most in a non-linear manner and reduces the weights of gains so that the largest gains are weighted sub-linearly. But that may still not be enough to satisfy one's real risk aversion. That is part of my argument for the iterated log form but it may be that an explicit metric such as standard deviation is still needed.
Larry Williams writes:
Optimal or Maximum Wealth (possible gain) only comes with Maximum Risk; therein lies the problem. Not loosing…risk…is more important than gain in the art of speculation business.
Chris Cooper writes:
More important, as far as my practical experience goes, is that one's estimate of the edge is always subject to uncertainty. The reasons have been discussed on this list before, but certainly include changing regimes, limited history for the models, curve fitting, flexionic machinations, scaling nonlinearity, etc. I relied on the Kelly formula extensively in the mid-'70s when gambling, and uncertainty in your edge was no less important then. The problem arises because overestimating your edge is so destructive to your terminal wealth.
It might be interesting academically to consider an approach, such as Bayesian, where your estimate of the edge is not stationary, but in fact must decrease when you hit a losing streak.
James Arveson writes:
I am a newbie on this site, but I can assure y'all that any finite amounts of outcomes can easily be handled by maximizing the expected value of the logarithm of one's fortune. I have also executed these theoretical outcomes for many years in AC and LV in BJ, and yes, in Bethlehem, PA in Texas Hold 'Em. See Mathematics of Poker for a better exposition of these issues than I could ever present.
Remember that each bet is a single bet, and one can bet forever. Leo Breiman has actually proved that (in the most general cases) that this approach DOMINATES all other strategies.
Now, IMHO, this approach is irrelevant to the market. NO ONE can get all the statistical assumptions correct-statistical distribution, EV, correlation, return, s.d., etc.
Have fun until we get to the next level. Same goes for Markowitz. Check out www.styleadvisor.com. I have no piece of their puzzle but wish I did (I might be able to get a write-off ski trip to Lake Tahoe where they are located).
Actualizing all of this crap may be the next Nobel Prize in Econ, but it will probably not help schlepers make money in the markets.
Ralph Vince replies:
Pursuing awards is for schlepers like Krugman or other academic dweebs –it's an award voted upon by dweebs for dweebs, and its pursuit bridles and constrains the mind (as *any* political pursuit will. Usually, the truth lies with things that - people off). To-wit, the lack of challenge to the notion that Kelly presents on p925 in the conclusion of his now-famous paper wherein he asserts that geometric growth is maximized by the gambler betting a fraction such that "at every bet he maximizes the the expected value of the logarithm of his capital."
This is accepted by the gambling community, and, by extension (falsely, mistakenly) accepted by the trading community. HOWEVER, a critical analysis of this notion reveals that it does NOT result in the growth optimal fraction, but rather in a multiplier of one's account to risk (the two are different indeed, the latter being less than or equal to the former, resulting often in over-wagering). In fact, the multiplier on one's stake equals the optimal fraction to risk only in certain, specific instances which manifest in gambling, but are rare still in trading (e.g. only on long positions, etc.). I would gladly go into this in depth put I cannot publicly do so as the paper on this has been publish in a current issue of a journal, and I have agreed to refer those interested to the article instead. The upshot is, that the Kelly Criterion, as specified above, is not what Kelly and others thought it was except in the special case I just mentioned — it is NOT the growth-optimal fraction, but something different, equal to the growth-optimal fraction only in the special case — a case that manifests in gambling with ubiquity, and oddly, in trading very rarely.
Again, the gambling community has accepted it for reasons mentioned –because it does give you the same answer for the optimal fraction to bet as the formulations for the optimal fraction in the gambling situations. But just because it gives you he same answer as the optimal fraction in special situations does not mean it is the formulation for the optimal fraction –it isn't.
Secondly, even the "optimal fraction"it is never optimal. Suppose you are playing a game with a 50% probability and odds of 2 to 1. Your optimal fraction is .25 (if you were to play forever). However, after the first pay, the phone rings, it;s your wife, and she informs you of an emergency and you have to bolt the game (with your winnings from the one play, make you a popular guy). If you knoew beforehand you were going to only play for 1 play, you should have bet 100% of your stake to maximize your gain. If the call came after two plays in this game, you should have bet .5.
Tomorrow, you come back to this game — and you bet .25, reconciling yourself that yesterday you bet .25. (so….the game possesses memory?)
Wait, it gets worse in trading, where we see that each individual bet is, in fact, NOT a bet. Let's say you trade only XYZ stock, and you put on 300 shares. Let's say you have a stop below your buy price but it;s a different level for each of 100 shares, so you have three stops below the proce for 100 shares each at different levels. Now, let's say onyl the closest stop, for 100 shares, gets hit, resulting in a loss on 100 of the three hundred shares. Weeks later, you sell out 100 shares at a profit, and, a few weeks after that, another 100 shares at a price higher than that.
But these are NOT three separate trades. This is ONE trade, one wager for the purposes of growth-optimal calculations. And the reason is because you are ONLY trading XYZ — there has been NO recalculation of positions to put on until the entire thing has been closed out. IF, on the other hand, you were having other trades throughout the course of your aggregate position in XYZ, then you WOULD consider each of these a separate trade.
Trading is not the same as gambling. There are similarities, but don't make the assumption that because you risk something and gain something that it is the same. There are things which are proxies for truth, that asymptotically appear to be truth, but they are only proxies (such as the Kelly Criterion) as well as the widely-held (in the gambling community, and hence the trading one as well) but incorrect notion that a wager should be assessed based on it's asymptotic mathematical expectation. This too is a mere proxy and an incorrect one that can, in extreme cases, lead one to accept bad wagers and reject favorable ones.
Again, critical thinking has been absent and trumped by the acceptance of industry catechism.
Finally, you speak of SD's and EV (mean-variance is dead incidentally, as dead as dead can be everywhere BUT academia) correlations, and Chris mentions the (valid) problems of assessing the edge in the future and the problem of non-stiationarity.
The solution to growth optimality in the markets, lies in NOT accepting the Kelly Criterion, but instead accepting what IS the growth optimal fraction– because that then reveals (in the simplest of ways!) how to address the problem of non-stationarity in the future and it doesn't require any of these parameters, or even a computer, it's really THAT simple if you want to attempt growth optimality in the future.
Phil McDonnell comments:
Ralph raises a lot of interesting philosophical questions. On some points I disagree, so let me elaborate. For the purposes of this piece I will assume one is entirely risk averse and seeks only to maximize expected wealth on a compounded growth basis.
First he raises the point that there is no guarantee that a game or investment opportunity will continue. Certainly a true statement. However it is also true that there will be a succession of such opportunities available in one's lifetime. Thus some rational basis for choosing bet size each time should include consideration of expected logs of the outcomes.
Philosophically I disagree with Ralph's analysis of bet it all on the last bet. His math is correct, in that it will maximize the expected dollar outcome. But there will always be other bets, so one's lifetime objective should still be to maximize the expected log not simply the arithmetic expected value. I believe Kahneman and Tversky made the same error in their Nobel winning papers.
I have an alternate take on Ralph's argument that it is hard to define a trade because you can put on 300 shares and exit 3 times at different prices, unknowable in advance. Rather than look on each trade as the basic metric one should look on the portfolio as the metric and a basic unit of time as the portfolio re balancing decision point. For example if you invested .25 of your wealth in a trade that doubled you know have .50 of your 1.25 wealth in the trade. That is too much if you want to maintain the .25 ratio so you need to sell .1875 to get back to your optimal ratio. But the simplest way to look at it is to look at the investment portfolio in each time period, be it a day, week or whatever.
One of the reasons the mean covariance model is in disfavor is that it seems to fail when everything hits the fan. In fact the model is incomplete in the sense that EV and COV are stochastic variables and vary over time. (I am implicitly including VAR here.) You need to explicitly include the correlations somehow in order to take into account how an entire portfolio will vary together. Using the formulas for optimal bet size on a trade level will always lead to serious over trading if there are multiple trades put on at the same time except in the case of a negative correlation between the trades. So it is misleading to calculate an optimal trade size for one system or one trade without consideration of any others that might be on at the same time. At best it is a dangerous upper bound for any single trade size. But it will almost always be an estimate too high. Optimization of expected log of wealth can only be done at the portfolio level.
Ralph Vince responds:
I am not raising ANY "philosophical questions." Just because people may have to think about them doesn't make them philosophical questions as opposed to facts:
1. The value that satisfies the Kelly Criterion is NOT the (growth) optimal fraction of ones stake to risk (although, in special circumstances which we find ubiquitously in gambling and not in trading, it is an equivalent value to the value that IS the optimal fraction). And the pervasive mistake by those attempting growth maximization in the marketplace of using the Kelly Criterion result puts then OVER exposed, to their unwitting peril. They are NOT growth optimal. In fact, the value that satisfies the Kelly Criterion NEVER returns the growth optimal fraction. This was a mistake on the part of Kelly and Shannon. The very fact that it is still accepted by others is testimony to the absence of critical thinking in this matter.
2. Further, what IS the growth optimal fraction is a function of the horizon of the game — and all games have a horizon, including the game of evolution on earth. Further, all metrics, including the analysis of drawdowns (including VAR where a horizon of 1 is implicit), even the analysis of whether a wager should be accepted or not, are a function of horizon. Disregarding the horizon leads us to incorrect conclusions at every turn in risk-opportunity analysis. In fact, it is the necessary introduction of "horizon" that gives rise to this entire burgeoning discipline.
3. Once we accept points 1 and 2 above, the obvious solution to solving for the non-stationarity of the distribution of outcomes we are dealing with becomes obvious. Growth-maximization, unlike attempts at it in the past, now CAN be performed with informed assessments of what the best growth optimal fraction value to use in the future will be.
The leading historian says that he'll buy me a $ 8 cup of coffee under certain considerations. And I don't know much about coffee. But I've had occasion to have coffee at Stumptown Coffee, an Oregon firm with branches in New York now, and it's far and away the best coffee i've ever had. Next in line is the coffee at Kaffe that Mr. Florida surfer has recommended. The web mistress is a vegan, and I don't pay her that much to do all the editing and picturing so she usually doesn't put our stuff on barbecue up unless I get her mother on the case, which isn't that effective since she doesn't believe in coercion. Let us expand our mandate from bar b que to good beverages like coffee and tea.
Vince Fulco comments:
I wouldn't say THE top tier but for solid, day-in, day-out coffee, a NYC mail order institution which we order from is portorico. It's been around for over 100 years and we especially like their couple times a year sale with numerous versions of beans $5.99-7.99/lb, a veritable bargain when retail goes for similar prices for 10 ounces. They also have a weekly sale of one kind or another.
Jeff Sasmor writes:
For NJ suburbanites, the local roasting of primo beans and a nice college town quasi-hipster atmosphere is provided by Small World Coffee in Princeton. In spite of a Starbucks opening around the corner, Small World has actually grown larger.
David Hillman writes:
Stumptown is among best ever drunk here, too. We have a pound or two shipped in regularly. They ship the same day they roast and deliver in about 2-3 days, so coffee is very fresh. Currently in the cabinet is Indonesia Sulawsi Toarco and the African's are exceptional this year. An admirable direct trade business model worthy of support.
Also, when in Portland, breakfast at Mother's. They serve Stumptown varieties in a french press at the table. That and the wild salmon hash is more than worth the long weekend a.m. waits.
Boom Bros. in Milwaukee is also happily recommended. Excellent roastmaster, their Velvet Hammer is the 'every morning' coffee at Cafe DGH.
Another favorite is this coffee from the D.R. Very cheap, very good. Best drunk in a cafe on the beach in Sosua. Maybe there's a Caribbean store of some sort in NYC?, but if not, there's always Bonanza:
"…..Always the most fresh production guaranteed! Manufacturer send my orders 3 times a week…..Thanks for looking!!!"
Chris Cooper writes:
Coincidentally, I have recently embarked on a quest to brew (consistently) the best cup of coffee. I have started roasting my own beans, and now it is evolving to importing my own green beans. Next month on the container arrives 300 kg of single-origin green beans from Indonesia from five farms. We call them Bali Kintamani, Java Jampit, Aceh Gayo, Sumatra Lintong, and Torajah Kalosi. I guess this may become more than just a hobby.
While Mr. Surfer and family visited not so long ago, we served some Kopi Luwak, famous due to the journey of the fresh beans through the digestive tract of a civet. It turns out that there are various grades of Kopi Luwak, and since that time I've found a verifiably authentic version, which is rarer because often the growers will mix in other beans. I may try to import that as well, but it's very, very expensive, and I can probably only get 10 kg per year. The taste is really different, much earthier.
Larry Williams comments:
My cup runneth over with coffee from these guys, but thanks for the tips. I will begin my journey again for greatest java.
By the way, Overstock.com seems to have the best deals on espresso machine.
T.K Marks writes:
All this talk of coffee has gotten me nostalgic for one of my life's more squandered opportunities.
There was this little coffee spot on the Upper West Side, just a stone's throw from Lincoln Center, called Cafe Mozart. I used to spend much time there.
I would get a pot of coffee. Once even this thick Turkish stuff that perhaps made one look of Left Bank sensibilities, but tasted like tar. Would while away the hours there with reading, backgammon, or chess. It was a peaceful place.
So one night I'm sitting alone at my table reading when walks in and approaches, a woman.
A woman with a very fetching smile.
Bob?…she asked hesitatingly, as one would when meeting a blind date.
I stood up politely, smiled at her for a few seconds, and, No, was all I said.
Till this day I regret not lying through my teeth.
Had nothing to lose.
Jeff Watson writes:
Many of my friends are coffee experts but I am sadly lacking in that department. One thing I do know is how to make is one of the better pots of coffee on the planet. The following recipe will even make even the most mediocre coffee taste good, and good coffee taste……delicious.
1. Wash an egg then break it into the bottom of an old fashioned metal campfire coffee pot, beating the egg slightly, leaving egg, shells and all in bottom of the pot..
2. Add a cup of very cold water to the pot, covering the egg and then add a pinch of salt.
3. Pour in a whole cup of course ground coffee to the water and egg mixture, and stir it up.
4. Pour enough boiling water over the coffee, egg, mixture to almost fill the pot up, and stir until mixed.
5. Cover the pot and plug the spout with a dish towel.
6. Put the coffee pot over a fire, heat it up to a gentle boil, back off, then let it simmer for a couple of minutes.
7. Take the pot off of the fire, let the coffee settle for a couple of minutes then add a cup of very cold water to precipitate the coffee grounds/egg mixture. Let the coffee settle for another minute, then serve.
My grandfather was taught to make coffee this way from some real cowboys when he went to the Arizona Territory for a trip sometime before 1910. He taught me how to make coffee when I was around 7 or 8, and put me in charge of the coffee every time there was a family picnic or outing. The secret to wonderful coffee is the egg, the pinch of salt, and good water. Coffee prepared in this manner evokes many good memories, and the good smell alone will attract any friends or neighbors in the near vicinity. Once in a great while, I will make this coffee on the stove and it's almost as good as on a campfire.
I have often wondered what a Kona coffee would taste like if prepared in this manner.
J.T Holley writes:
I'm not a professional roaster or barista, but the keys that I learned in the 8-9 years that I mentored to roast, grind, and brew coffee are the following:
1) The time between roast and grind needs to be minimal (oils of the roast and storage important)
2) Method of brewing important to your individual tastes (percolate, press, or electric drip)
3) Water is 99% of a cup of coffee! Good tasting waters need to be used and free of chlorines, flourides, and impurities
4) Filtration choice and cleanliness of the brewer of choice imperative for consistent cups of good flavor
5) Once pot is brewed then stirring the pot and stirring the cup is important regardless of cream and sugar for consistency of coffee.
That's the basics!
All good shops should know this regardless if its a private house, private shop, franchise or friend.
Kim Zussman queries:
How can coffee gourmets taste fluoride but not civet excrement?
Jim Sogi writes:
Chris's special Java java was distinctive and earthy. A treat especially in the palatial surroundings.
The key to brewing good coffee from whatever origin, is:
1. Be sure the parchment is sun dried, not machine dried. It has a much mellower smooth flavor.
2. Roast your own coffee. My favorite roast is 462 degrees, 11 minutes give or take based on humidity and ambient. Roast until the oil just starts to show, but is not oily. The oily roast is more for show. Roast only what you can use in 3 days.
3. Grind your own fresh roast. This is the most important of all. Don't try to freeze coffee beans.
When brewing in filter, only pour a little, not boiling, water through at a time.
Oh yes, Kona Coffee is without doubt the best in the world.
How would the speed up stuff (see below) work in trading?
Trading while standing up?
Trading with a gun rather than a mouse?
Taking a fast 4 ticks? (guaranteed to lose money unless you have the infrastructure of a flexion)
Trading 3 markets in succession???
Larry Williams adds:
Going from yesteryear's 200 day moving average to a shorter one? Trading instant spreads?
Jim Sogi writes:
It's a whole new skill set, both different motor and mental with a learning curve. Years of practice with certain tools cannot be discounted. Like switching from squash to tennis to ping pong. Or longboard to shortboard.
Ralph Vince writes:
Great questions. Based on my own, limited, life experience, I would add that there is an element of a certain mental "groove," to all of this, necessary to success, not altogether very different than that of an athlete on the top of his game (we have discussed this at length in this forum– some great discussions on it I think) or when you are thinking a problem through– a very difficult, elusive one, threatening to drip off the edge of your consciousness…….and I'm not so sure that is even timeframe-specific, so long as you find your groove.
When I put on a trade, I KNOW I'm going to make money on it, I'm not worried about it one jot. You get into certain habits, which are a function of your cadence, and "settling in' to that, whereas I think it IS timeframe-specific, seems to be timeframe specific to the individual and how he trades.
I very much believe that the kind of "hurry up" trading you are describing here may fit certain individuals and may sabotage others. Even if on a purely mechanical basis. What comes to mind for me on this is trying to play simple, basic strategy blackjack at a table with a fast cadence– I can't handle it, and am certain to fumble it.
Ken Dreees writes:
It would be interesting to create a dynamic trading skills test in which you had mutliple positions open in multiple markets and were then given simulated info in a real time sense that caused market disruptions. You would be graded under criteria such as:
1. exiting safety
2. capital protection
3. Finding and exploiting panic etc.
Like a trading version of star fleet's test.
Jeff Watson adds:
Here's an interesting site with info on CBOT full seat prices from 1898-2004. There's a handy little excel download in the site with the high/low of CBOT seat prices on a yearly basis. 1942 was the year to go long the CBOT.
Russ Sears comments:
My opinion is that building up the endurance to concentrate for long periods of time is not like riding a bike. If you've been away from it a while train yourself back into it.
Taking scheduled stress relief breaks should be required to be on your best defensively, especially in volatile markets.
October 29, 2010 | Leave a Comment
Oregon football: Ducks' success on offense is just a matter of time
Oregon's fast offensive tempo has baffled opponents this season, and USC will try to solve it Saturday
By Rob Moseley
Appeared in print: Wednesday, Oct 27, 2010
In college football, it's best to take things one week at a time - unless Oregon looms on the schedule.
Because of the Ducks' withering offensive pace, USC has increased its tempo in recent practices. And not just this week or during the preceding bye week, Trojans coach Lane Kiffin said, but even earlier than that.
"Over the last couple of weeks, even going into the Cal game, we've picked up our tempo in practice whenever we were going against the defense," Kiffin said Tuesday. "I just think that if you try to all of a sudden do it in the week you're playing Oregon, it's not going to help a whole lot. …
"We've been doing this for a few weeks and took a different approach to the bye week than if we were probably playing someone else, by the speed we practiced at and the way we approached it."
The list of top-ranked Oregon's scoring drives this season, entering Saturday's 5 p.m. game at No. 24 USC, includes 16 shorter than a minute, and only three longer than four minutes. To some extent that reflects the big-play ability of such stars as LaMichael James, Jeff Maehl and Josh Huff, and also some favorable field position owing to the 25 turnovers forced by the UO defense and special teams.
But it also illustrates the tempo at which Oregon's offense plays, and the minimal time the Ducks take between snaps. Mostly, that's determined by how long it takes officials to spot the ball after the previous play.
"We're playing at a pretty good clip right now," UO coach Chip Kelly said. "I think it's because our players have a really good understanding of what we're trying to do. We just try to eliminate that time between plays, and then just go play."
Snapping the ball just a handful of seconds after the previous play minimizes the time available for the defense to call plays, substitute or react to Oregon's offensive formation. It's a common occurrence to see defenders spend those moments with their hands on their hips, a sure sign of fatigue.
Oregon doesn't have such trouble signalling plays to the offense. The Ducks streamlined their terminology in order to play faster on offense when Kelly was promoted to head coach in 2009, he said, and they employ a complicated system of signboards and hand signals to indicate plays during offensive possessions.
The signs each include four images - faces of ESPN personalities are among the most recognizable.
Kelly said the signs indicate a package of plays, and hinted that they can sometimes be used as a decoy. Sometimes, he said, players aren't asked to "go to the board" to get the call.
"It's just another way to play fast," Kelly said. "The analogy I can give you is, iif you go to McDonald's and order a No. 2, that's all you have to say and you get a Quarter Pounder and a drink and fries, and you just say, 'No. 2.' If we send them to the board, one picture can mean the formation and the play and the snap-count. That's all it is. It's just another way to play faster."
The Ducks' tempo seems particularly suited to their spread-option offense, which doesn't feature excessive pre-snap shifting and motion to fool the defense, as in the scheme employed by Boise State, to use just one example. But Kelly said he'd look to push the tempo even if Oregon's personnel was best suited to packages requiring three tight ends and a fullback.
"You could play up-tempo no matter what you do," Kelly said. "You watch (Indianapolis Colts quarterback) Peyton Manning, they're not running any option but they play as fast in the NFL, in terms of him being able to get their plays in and the speed they want to play at. So it's not married to the system."
Kelly also quipped that, "The byproduct is, as the play-caller, you can call a lot of really bad plays, and people forget about them quickly because we're on to the next one." But clearly Kelly, whose offense leads the nation in points and yardage, hasn't had many missteps in that regard.
The potential drawback to playing so fast on offense is that Oregon's defense is on the field so long. The Ducks are 114th out of 120 teams nationally in time of possession, at 26:40 per game; the 516 plays defended by Oregon are more than anybody else in the Pac-10 but Washington State.
The Ducks try to mitigate that with a regular rotation of about 25 players at the 11 positions on defense.
But Oregon also uses about 20 players regularly on offense, another challenge for opponents.
"The different ways they mix and match that personnel in terms of personnel groupings and formations, you're trying to identify what their top runs are and their top passes are, and when there's more of them it's hard to really pinpoint what their favorite ones to do are and defend them," Stanford coach Jim Harbaugh said.
Kiffin and his staff at USC are trying to tackle that challenge this week.
"You don't know who's going to be in when," Kiffin said. "They do rotate guys in. I'm sure it's because they know they're going to play a lot of plays because their offense scores so fast and doesn't use very much time.
"The 14 touchdown drives under (54) seconds, that's unheard of for four years of games, let alone half a season. I'm sure that's why they do that. I don't think until this week I realized how deep they were. For them to play so many people on offense and defense, they've got great depth."
All of it speedy, as Oregon tries to keep pushing the pace of this historic season.
"We just try to eliminate that time between plays, and then just go play."
- Chip Kelly, Oregon Football Coach
Pitt T. Maner III shares:
On speed in practice:
Bill Walton on Wooden and UCLA basketball practice, from (With Steve Jamison) Wooden: A Lifetime of Observations On and Off Court, Contemporary Books (Lincolnwood, IL), 1997.
For us, it all started with our practices at UCLA, which were nonstop action and absolutely electric, super-charged, on edge, crisp, and incredibly demanding, with Coach Wooden pacing up and down the sidelines like a caged tiger, barking out instructions, positive reinforcement, and appropriate maxims: "Be quick, but don't hurry." "Failing to prepare is preparing to fail." "Never mistake activity for achievement." "Discipline yourself and others won't need to.
"At the same time he constantly moved us into and out of minutely detailed drills, scrimmages, and patterns while exorting us to "Move…quickly…hurry up!" It was wonderfully exhilarating and absolutely intense.
In fact, games actually seemed like they happened in a slower gear because of the pace at which we practiced. We'd run a play prefectly in scrimmage and Coach would say, "OK, fine. Now re-set. Do it again, faster." We'd do it again. Faster. And again. Faster. And again.
I'd often think during UCLA games, "Why is this taking so long?" because we had done everything that happened during a game thousands of times at a faster pace in practice.
Ralph Vince writes:
It's a game of evolution, the hurry up, with a mobile quarterback, an absence of putting a man in motion (and hence, plays that aren't contingent on defensive reads) and spread out receivers.
The counter to it is the part I am trying to study, and seems to be confusing blitzes (which are difficult to coordinate when faced with a no-huddle), middle zones and man for man on the outside (and, surprisingly, it appears the outside coverage man should take the INNER of to wideouts on the same side when that occurs).
Russ Sears writes:
Football is a sport requiring quick short burst of speeds. Speeding up the normal pace of the game cause the recovery time to shorten drastically. An offense that knows before the position of attack can rotate the burst of speeds, where as the defense must all be ready and attacking. The stamina to endure these burst of speeds with short recovery takes a good month of training. This is, I believe, the Indy Colts advantage also.
In basketball, it takes even longer because of the aerobic base needed. Wooden and several other great basketball coaches practiced running stairs and full court press. As full court press is the basketball equivalent to no huddle offense.
In distance running the equivalent is cycles surging the letting up the pace. If you practiced this you could beat better runners who are not prepared for this.
Prechter says sell this rally off of yahoo finance headlines–no need to link, that's probably all you need to know about this move.
But if it is a market bluff, yesterday the market bet before the flop and today you should see the continuation bet on the turn and then a big bet to come on the river. If it's a bluff, then they gotta sell it.
Anatoly Veltman comments:
He's often quoted out of context, just like everyone else– thus everyone's track record may appear roughly same.
Prechter does certain analysis well. Those who understand his writings can benefit by incorporating some of his effort into own analysis. Those few who would actually enter trade on his conclusions– risk not knowing how/why to exit.
Ralph Vince writes:
Entirely true, Anatoly. I may not agree with his prognostications, but he does his work very well. What's more, he is often quoted in overly simplistic terms– such as to be a seller on this rally. I am certain he has a point where he would flip and go long, an alternate count or something. I am also sure he has a downside target– is it Dow 5000 ? Dow 10,500 ? These quotes of his floating around don't really tell you want his strategy is, and that's key. He's a guy who, if/when he is wrong, I have found he has not been wrong by much, often able to adapt to changing market conditions as well as any I have seen.
Larry Williams observes:
Prechter go long? Has he ever? His bearish book riding the wave came out the low the 2002, at the recent market low the clarion call was to sell. Be alert to broken watch correctness.
Dylan Distasio asks:
I'm genuinely curious as to why you lump Livermore in with the rest of the financial ne-er-do-wells. I'm not an expert on the man by any stretch of the imagination, but I've read assorted stuff on him, and while he was far from perfect in both trading and life (but then again who is?), I've never seen fit to paint him with that brush based on what I've read. Why do you have such a low opinion of him?
Larry Williams attempts an answer:
Livermore and the Reminiscences are two different stories. The Saturday Evening Post serial that became the book is oh-so well written but it is not just about Livermore it is/was a novel with a fictional character that paralleled Jesse but was also a collage.
In real life once Joe Kennedy took over the SEC, Jesse seems to have never made another penny; in other words he was most likely a runner of stocks not some brilliant trader like Steve Cohen, etc.
This article on income mobility will put in perspective the malaise affecting our economy. It's the 40 % from each of the lower 2 quintiles who moves to the top 2 quintile that has made us beautiful and created the jobs and responded to the past incentives, and dolorously "prefers not to" create jobs and value now.
Australian Nick White comments:
This is a great country. Being back here the last few weeks just reinforces to me how lucky America is– even if you're in a perceived funk right now. This is the country where anything can get done…that's not the case in any other western nation. You have freedoms that you take for granted every day (even post legislative amendments that may have eroded them more than trivially). You have every type of geography and lifestyle. You have 36 different choices of one brand of orange juice fer crissakes! (which you can drink while watching one of thousands of tv channels).
I don't know much, but I know that if America continues to focus on the the things that got them to here– without trying to reinvent the wheel– you will all be just fine. The only danger I see is increasing reliance on form rather than substance– but this is a malaise of the world in general, not just the US.
Rudolf Hauser writes:
This data on income mobility does not give us a complete picture. Large gains or losses from realized capital gains/losses, special bonuses payments, decisions to take long breaks from work, etc. can all influence results for any one year. One would also expect income from most careers to advance with experience and age. What would be interesting to see but probably very difficult data to obtain would be an average of five years of data say at age 50 with those relative income positions of those households income compared with that in the same period in the lives of their parents. I suspect that there would be a good deal of upward income mobility demonstrated by such an analysis, but it would nonetheless be most interesting to have that evidence.
Russ Sears writes:
Isn't this the premise of the sitcom "The Big Bang Theory"?
A group of nerdy physicists meet their neighbor, a beautiful blond girl waiting tables at the cheesecake shop… but even she is hoping to become an actress.
But you miss the point– from the Will Smiths to the nerds in physics to the marathon runners to the Saints QB, they are all incredibly talented, even the WS geeks, not just the WS geeks.
And as someone seen how letting a small business owners put the money back into a sport can revitalize: it can change how everybody developed talent. In 1992 The US marathon trials were a joke, but these guys changed it.
The world will never know the talents that were not developed for lack of a few dollars, but I have seen first hand how thin the pie can be sliced at the top, and how a few centimeters thicker can change everything.
Jordan Neuman comments:
It is interesting that you mention the varieties of orange juice. I just read The Paradox of Choice which argues that our lives would be better if we did not have so many choices. The varieties of grocery items was the author's starting point.
It would not matter unless such ideas had support in this Administration. The references to health insurance in the book are illustrative. And I found the interview with the author in the afterword absolutely chilling. This professor was sure he and his "expert" friends knew better.
Larry Williams writes:
Living in the US Virgin Islands means giving up many choices in foods, clothes, cars, etc. I have found that a wonderful thing; it causes one to focus on what is really desired (that can be ordered from off island). It makes for a simpler life style and turns ones attention from man made consumables to the ocean, the trade winds, local markets and such.
Sam Marx comments:
I remember one of the escaped English spies then living in Moscow, when asked what he missed most about England, he replied Lea & Perrins Steak Sauce.
Being a trader and not a math guy like our host I may not be able to prove there is or is not a seasonal pattern for gold to rally in August, but I am impressed when I create a seasonal chart for Au from inception to 2000 and see what looks like such a general pattern. I then run the same tool on data from 2000 to date and see about the same thing and, then again, I am very prejudiced on this seasonal concept as I wrote the first book ever on seasonal studies and even way back then we saw this pattern in gold. So to me it seems there is–at times–a trading advantage or bias here. (plus I am long as this is written from 8/11). My views on seasonals have altered since that 1973 book quite a bit; seasonals are not a mandatory thing but often offer confirmation and a suggestion of what is in store.
I have tried to attach charts to illustrate this point but size is to large (will send if you just have to have them), it is different opinions that make for horse races.
Happy Trails to all,
Victor Niederhoffer says:
The handball senator said about seasonality, the chair agrees with
the senator. I was too hasty involved in a position. With humble mien.
The small chart below is trade by trade buying on the open of the 13th, exit first profitable open or 3,600 stop. In the S&P 500 since 1982 there have been 46 trades. 80% were profitable and not stopped out– but, always a but…risk reward is .48 and drawdown was equal to a little less than 1/2 the net profits. Bottom line: looks like an advantage that might be traded by the nimble but a filter would help. Happy trails to all.
The perpetual edge comes from understanding the human condition.
We all have an interest in not suffering through another day like May 6. And without violating our rule of never disseminating anything that is a meal for a day, i.e. a recurring regularity, perhaps you will forgive me if I attempt to open a discussion of how a day like May 6 where the market was at a minimum to start, open down and then went up and then dropped 110 points or more, a nice 10% to wipe out point– how such could have been predicted.
Ralph Vince comments:
What cost? If someone has stop orders in (fear of loss or missing a move to the downside), there is cost. If someone was, say, buying on a limit, it was a boon. If someone got shaken out of a position (fear) because they couldn't ride it to 0, I posit they were in too heavily. They were clearly people who were trading with money they could not afford to lose. (In my book, that makes them losers before they even put on the trade!)
Jonathan Bower writes:
I saw many parallels to "that" day on Wednesday. I'm curious if you all enjoyed Wednesday too…
Larry Williams replies:
I have enjoyed my luck which is soon to fade I suspect.
My grandfather grubstaked miners. I have grubstaked many many systems. Only one made money and then ok but no great shakes.
March 17, 2010 | 1 Comment
VIX Doesn’t Work as Signal for U.S. Stock Returns, Birinyi Says
March 17 (Bloomberg) — Investors looking for clues about the U.S. stock market should probably ignore the Chicago Board Options Exchange Volatility Index, according to a study of the VIX by Birinyi Associates Inc.
Speculation that equity returns will be positive after the volatility gauge decreases and negative when it climbs has little basis in fact, Birinyi said. "The VIX is alleged to be an indicative indicator and has become a staple of analysts and journalists alike," Laszlo Birinyi and analyst Kevin Pleines wrote in a report to clients.
The following is a table of the S&P 500's average gain or loss during periods after implied volatility climbed above or fell below the 50-day average: (since September 2003)
1 Month 2 Months 3 Months 6 Months
VIX 20% Below 0.09% -0.49% 3.33% 5.84%
VIX 20% Above 1.25% 0.50% 0.95% -4.51%
Source: Birinyi Associates
Larry Williams writes:
As I have always postulated, the VIX is just the Dow/S&P upside down. It's hard to predict A with A.
Jason Goepfert comments:
I'm not a VIX fanboy by any means, but that article was ridiculous. It only looked at returns since September 2003. And it only tested a strategy of crossing 20% above or below the 50-day average. Why 20%? Why the 50-day average? Why just since September 2003? Did they test anything else? Or is that the one they found that supports their (so far very correct) bullish view?
The ridiculous part is taking such a weak study and then proclaiming "the VIX doesn't work."
Allen Gillespie adds:
He doesn't have enough bins — bins of 5 show something different.
Kim Zussman writes:
- Volatility was extinguished by fiat liquidity
- The only double-dippers left are Jibao, Roubini, and Michael Moore
- Nothing to fear above moving averages
Marlowe Cassetti responds:
I have always doubted the assertion that VIX is a measure of market fear and greed. Years ago I read Whaley's academic paper and I was not satisfied with the author's fear/greed connection. To me VIX is simply the volatility number you plug in to make the Black-Scholes option equation work.
Bud Conrad answers:
My detailed review of VIX concluded that the VIX followed stocks (inversely) a day later. It was not predictive. Longer term charts seemed to indicate opposite movements, but the data could not be used as expected.
January 8, 2010 | Leave a Comment
Trading hours upset the circadian rhythms of millions of people and we need to learn the side affects of sleep deprivation and how to deal with them.
Jim Sogi agrees:
When you don't sleep enough you get grumpy, uncoordinated, depressed, groggy. It's bad news.
Nigel Davies comments:
If trading does that to someone and he can't find a way around it — smaller position size, some kind of healthy stress relief — he should quit. It just isn't worth it.
Phil McDonnell explains:
A considerable amount of research has been performed into our nightly dream cycles. The typical cycle lasts about 90 minutes. So in six hours of sleep we get four completed cycles. In 7.5 hours we get fvie cycles, in nine hours we get get six. Note that eight hours does not give us an even number of cycles.
The importance of a full cycle was established a long time ago. When sleep researchers monitored test subjects and woke them at their deepest part of the dream cycle they were shown to be mentally impaired on simple cognitive tests. Awakened subjects could remember their dreams.
Subjects awakened after a full cycle performed much better on the same tests. The full cycle subjects could not remember their dreams.
Ideally sleep should be an even multiple of 90 minutes and one should awaken with no memory of the last dream. A corollary to this is that if one is awakened after, say, six hours and cannot remember a dream then it my be wise to get up. This is especially true if one feels refreshed and cannot stay in bed for another full hour and a half.
Dr. McDonnell is the author of Optimal Portfolio Modeling, Wiley, 2008
It is rather interesting that the ubiquitous 200 day moving average (actually 40 week ma) still works at all, because everyone seems to know about it.
Larry Williams replies:
Technical indicators, I think, can be widely known about and still 'work' because of the frailty of humans; as a group we have problems with following/sticking to rules. We can all know we should not drink and drive, speed, do drugs, smoke, etc… but most violate basic rules.
How can we avoid curve fitting when designing a trading strategy? Are there any solid parameters one can use as guide? It seems very easy to adjust the trading signals to the data. This leads to a perfect backtested system - and a tomorrow's crash. What is the line that tells apart perfect trading strategy optimization from curve fitting? The worry is to arrive to a model that explains everything and predicts nothing. (And a further question: What is the NATURE of the predictive value of a system? What - philosophically speaking - confer to a model it's ability to predict future market behavior?)
James Sogi writes:
KISS. Keep parameters simple and robust.
Newton Linchen replies:
You have to agree that it's easier said than done. There is always the desire to "improve" results, to avoid drawdown, to boost profitability…
Is there a "wise speculator's" to-do list on, for example, how many parameters does a system requires/accepts (can handle)?
Nigel Davies offers:
Here's an offbeat view:
Curve fitting isn't the only problem, there's also the issue of whether one takes into account contrary evidence. And there will usually be some kind of contrary evidence, unless and until a feeding frenzy occurs (i.e a segment of market participants start to lose their heads).
So for me the whole thing boils down to inner mental balance and harmony - when someone is under stress or has certain personality issues, they're going to find a way to fit some curves somehow. On the other those who are relaxed (even when the external situation is very difficult) and have stable characters will tend towards objectivity even in the most trying circumstances.
I think this way of seeing things provides a couple of important insights: a) True non randomness will tend to occur when most market participants are highly emotional. b) A good way to avoid curve fitting is to work on someone's ability to withstand stress - if they want to improve they should try green vegetables, good water and maybe some form of yoga, meditation or martial art (tai chi and yiquan are certainly good).
Newton Linchen replies:
The word that I found most important in your e-mail was "objectivity".
I kind of agree with the rest, but, I'm referring most to the curve fitting while developing trading ideas, not when trading them. That's why a scale to measure curve fitting (if it was possible at all) is in order: from what point curve fitting enters the modeling data process?
And, what would be the chess player point of view in this issue?
Nigel Davies replies:
Well what we chess players do is essentially try to destroy our own ideas because if we don't then our opponents will. In the midst of this process 'hope' is the enemy, and unless you're on top of your game he can appear in all sorts of situations. And this despite our best intentions.
Markets don't function in the same way as chess opponents; they act more as a mirror for our own flaws (mainly hope) rather than a malevolent force that's there to do you in. So the requirement to falsify doesn't seem quite so urgent, especially when one is winning game with a particular 'system'.
Out of sample testing can help simulate the process of falsification but not with the same level of paranoia, and also what's built into it is an assumption that the effect is stable.
This brings me to the other difference between chess and markets; the former offers a stable platform on which to experiment and test ones ideas, the latter only has moments of stability. How long will they last? Who knows. But I suspect that subliminal knowledge about the out of sample data may play a part in system construction, not to mention the fact that other people may be doing the same kind of thing and thus competing for the entrees.
An interesting experiment might be to see how the real time application of a system compares to the out of sample test. I hypothesize that it will be worse, much worse.
Kim Zussman adds:
Markets demonstrate repeating patterns over irregularly spaced intervals. It's one thing to find those patterns in the current regime, but how to determine when your precious pattern has failed vs. simply statistical noise?
The answers given here before include money-management and control analysis.
But if you manage your money so carefully as to not go bust when the patterns do, on the whole can you make money (beyond, say, B/H, net of vig, opportunity cost, day job)?
If control analysis and similar quantitative methods work, why aren't engineers rich? (OK some are, but more lawyers are and they don't understand this stuff)
The point will be made that systematic approaches fail, because all patterns get uncovered and you need to be alert to this, and adapt faster and bolder than other agents competing for mating rights. Which should result in certain runners at the top of the distribution (of smarts, guts, determination, etc) far out-distancing the pack.
And it seems there are such, in the infinitesimally small proportion predicted by the curve.
That is curve fitting.
Legacy Daily observes:
"I hypothesize that it will be worse, much worse." If it was so easy, I doubt this discussion would be taking place.
I think human judgment (+ the emotional balance Nigel mentions) are the elements that make multiple regression statistical analysis work. I am skeptical that past price history of a security can predict its future price action but not as skeptical that past relationships between multiple correlated markets (variables) can hold true in the future. The number of independent variables that you use to explain your dependent variable, which variables to choose, how to lag them, and interpretation of the result (why are the numbers saying what they are saying and the historical version of the same) among other decisions are based on so many human decisions that I doubt any system can accurately perpetually predict anything. Even if it could, the force (impact) of the system itself would skew the results rendering the original analysis, premises, and decisions invalid. I have heard of "learning" systems but I haven't had an opportunity to experiment with a model that is able to choose independent variables as the cycles change.
The system has two advantages over us the humans. It takes emotion out of the picture and it can perform many computations quickly. If one gives it any more credit than that, one learns some painful lessons sooner or later. The solution many people implement is "money management" techniques to cut losses short and let the winners take care of themselves (which again are based on judgment). I am sure there are studies out there that try to determine the impact of quantitative models on the markets. Perhaps fading those models by a contra model may yield more positive (dare I say predictable) results…
One last comment, check out how a system generates random numbers (if haven't already looked into this). While the number appears random to us, it is anything but random, unless the generator is based on external random phenomena.
Bill Rafter adds:
Research to identify a universal truth to be used going either forward or backward (out of sample or in-sample) is not curvefitting. An example of that might be the implications of higher levels of implied volatility to future asset price levels.
Research of past data to identify a specific value to be used going forward (out of sample) is not curvefitting, but used backward (in-sample) is curvefitting. If you think of the latter as look-ahead bias it becomes a little more clear. Optimization would clearly count as curvefitting.
Sometimes (usually because of insufficient history) you have no ability to divide your data into two tranches – one for identifying values and the second for testing. In such a case you had best limit your research to identifying universal truths rather than specific values.
Scott Brooks comments:
If the past is not a good measure of today and we only use the present data, then isn't that really just short term trend following? As has been said on this list many times, trend following works great until it doesn't. Therefore, using today's data doesn't really work either.
Phil McDonnell comments:
Curve fitting is one of those things market researchers try NOT to do. But as Mr. Linchen suggests, it is difficult to know when we are approaching the slippery slope of curve fitting. What is curve fitting and what is wrong with it?
A simple example of curve fitting may help. Suppose we had two variables that could not possibly have any predictive value. Call them x1 and x2. They are random numbers. Then let's use them to 'predict' two days worth of market changes m. We have the following table:
m x1 x2
+4 2 1
+20 8 6
Can our random numbers predict the market with a model like this? In fact they can. We know this because we can set up 2 simultaneous equations in two unknowns and solve it. The basic equation is:
m = a * x1 + b * x2
The solution is a = 1 and b = 2. You can check this by back substituting. Multiply x1 by 1 and add two times x2 and each time it appears to give you a correct answer for m. The reason is that it is almost always possible (*) to solve two equations in two unknowns.
So this gives us one rule to consider when we are fitting. The rule is: Never fit n data points with n parameters.
The reason is because you will generally get a 'too good to be true' fit as Larry Williams suggests. This rule generalizes. For example best practices include getting much more data than the number of parameters you are trying to fit. There is a statistical concept called degrees of freedom involved here.
Degrees of freedom is how much wiggle room there is in your model. Each variable you add is a chance for your model to wiggle to better fit the data. The rule of thumb is that you take the number of data points you have and subtract the number of variables. Another way to say this is the number of data points should be MUCH more than the number of fitted parameters.
It is also good to mention that the number of parameters can be tricky to understand. Looking at intraday patterns a parameter could be something like today's high was lower than yesterday's high. Even though it is a true false criteria it is still an independent variable. Choice of the length of a moving average is a parameter. Whether one is above or below is another parameter. Some people use thresholds in moving average systems. Each is a parameter. Adding a second moving average may add four more parameters and the comparison between the two
averages yet another. In a system involving a 200 day and 50 day
average that showed 10 buy sell signals it might have as many as 10 parameters and thus be nearly useless.
Steve Ellison mentioned the two sample data technique. Basically you can fit your model on one data set and then use the same parameters to test out of sample. What you cannot do is refit the model or system parameters to the new data.
Another caveat here is the data mining slippery slope. This means you need to keep track of how many other variables you tried and rejected. This is also called the multiple comparison problem. It can be as insidious as trying to know how many variables someone else tried before coming up with their idea. For example how many parameters did Welles Wilder try before coming up with his 14 day RSI index? There is no way 14 was his first and only guess.
Another bad practice is when you have a system that has picked say 20 profitable trades and you look for rules to weed out those pesky few bad trades to get the perfect system. If you find yourself adding a rule or variable to rule out one or two trades you are well into data mining territory.
Bruno's suggestion to use the BIC or AIC is a good one. If one is doing a multiple regression one should look at the individual t stats for the coefficients AND look at the F test for the overall quality of the fit. Any variables with t-stats that are not above 2 should be tossed. Also an variables which are highly correlated with each other, the weaker one should be tossed.
George Parkanyi reminds us:
Yeah but you guys are forgetting that without curve-fitting we never would have invented the bra.
Say, has anybody got any experience with vertical drop fitting? I just back-tested some oil data and …
Larry Williams writes:
If it looks like it works real well it is curve fitting.
Newton Linchen reiterates:
my point is: what is the degree of system optimization that turns into curve fitting? In other words, how one is able to recognize curve fitting while modeling data? Perhaps returns too good to believe?
What I mean is to get a general rule that would tell: "Hey, man, from THIS point on you are curve fitting, so step back!"
Steve Ellison proffers:
I learned from Dr. McDonnell to divide the data into two halves and do the curve fitting on only the first half of the data, then test a strategy that looks good on the second half of the data.
Yishen Kuik writes:
The usual out of sample testing says, take price series data, break it into 2, optimize on the 1st piece, test on the 2nd piece, see if you still get a good result.
If you get a bad result you know you've curve fitted. If you get a good result, you know you have something that works.
But what if you get a mildly good result? Then what do you "know" ?
Jim Sogi adds:
This reminds me of the three blind men each touching one part of the elephant and describing what the elephant was like. Quants are often like the blind men, each touching say the 90's bull run tranche, others sampling recent data, others sample the whole. Each has their own description of the market, which like the blind men, are all wrong.
The most important data tranche is the most recent as that is what the current cycle is. You want your trades to work there. Don't try make the reality fit the model.
Also, why not break it into 3 pieces and have 2 out of sample pieces to test it on.
We can go further. If each discreet trade is of limited length, then why not slice up the price series into 100 pieces, reassemble all the odd numbered time slices chronologically into sample A, the even ones into sample B.
Then optimize on sample A and test on sample B. This can address to some degree concerns about regime shifts that might differently characterize your two samples in a simple break of the data.
If one can predict the market then there are better techniques than Dollar Cost averaging. But DCA is a decent strategy if two conditions hold:
- One cannot predict the market.
- One has an external income source available to be invested regularly.
Leveraged ETFs can grind investors up in unexpected ways because of the daily rebalancing. I suspect he will see that these ETFs are the exact opposite of a DCA strategy. In DCA your investment buys more shares after a dip and acquires fewer shares after a market rise. Overall your average share price is below the average of the market prices.
Leveraged ETFs employ the exact opposite strategy. When the market rises they are forced to buy more shares. When it falls they are forced to sell shares to maintain their constant leverage ratio. The net result is they buy shares at an average price above the average of market prices over the period. Thus the levered ETFs use an anti-DCA and that is what causes the grind.
Dr. McDonnell is the author of Optimal Portfolio Modeling, Wiley, 2008
Larry Williams comments:
I would add a third condition: Markets make new highs.
I'm sounding alarm. Amidst signs that:
1. Trading and Investment capital is continuing to contract.
2. Political and regulatory interferences are becoming more chaotic
3. Economic downturn's impact is widening
…it is harder than ever before to pin down intermediate-term opportunities. Those who've been trading in-and-out and even reversing every few days (albeit, with year-opener bias in mind), have done well. But in the course of February, year-opener directions should fade. Sector by sector:
1. SP: Commercials have been on the right side of every twist-and-turn. Raising Net Longs late Q4, shorting early Jan, buying again mid-Jan and selling again last week. Test of Nov lows is certain - and only then the panic will reach the pitch tone. However, it's important to not lose sight of the absolute diminishing values: what will look like the ultimate break-down - in fact, will have little room to forge ahead, relative to enormous Bear coups of 2008. When smaller players finally go overly Short on new lows, en mass - they'll find little reward.
2. Treasuries: 30y futures have retraced exactly half of their straight-line Q4 sprint 111->142, thus relieving unconscionable Xmas overbought. O.I. pattern is bullish at current juncture, dropping on down-days and rising on up-days.
3. Currencies: C.O.T. display intriguing divergence vis-a-vis equity-Bear posture. Yen commitments are Bearish, while SF Bullish. I'm getting ready for substantial reversals in both dropping European currencies and rising Yen. My scenario is that such reversals will be playing out against the background of equities' panic, and will catch Specs flat-footed.
4. GC commitments got predictably stone-walled in course of super-rally. Commercials offered scale-up across the precious metals complex. While long-term outlook for Gold is unavoidably Bullish (given few viable investment alternatives), I'd much rather be a buyer on any sharp profit-taking spells, than on any "strong trend". Copper O.I. pattern remains Bullish - but it will be up-hill battle against the back-drop of equity panic.
5. Energy contracts remain in disarray, with little of new indicators in the past week. Of note CL 6-week consolidation pattern that follows vertical 147->33 move, record HO O.I. levels, RB price out-performance (not supported by O.I. pattern) and NG price under-performance, nearing very important $4.05 low of 2006. My conclusion is that the complex will struggle with equities - and that important buying opportunity will form in the process.
George Parkanyi comments:
I admit that I apply COT like a simpleton, but time and time again I've noticed that aligning with the commercials in physical commodities (in the financial indices or currencies I don't even know who a commercial is, or if they're particularly bright enough to make a difference) generally gets you going in the right direction. THAT I learned to pay attention to from Larry's excellent books on the subject. (Unfortunately, Larry, I didn't follow your money management advice and eventually skewered my commodities account).
A guy called Barry Lees runs a site called cotfutures and basically all he does is reorganize COT data into a useful format -particularly a rolling 18 month 0-100 ranking representing the range between the maximum commercial net-short position (0) and the maximum net-long position (100). I've found the 0-10's to be pretty good markers for a downward reversal and the 90-100's for an upward reversal. Speculators would be at the opposite side of the spectrum. There are other factors of course, but these are pretty good ballpark indicators.
Right now, Copper and Rough Rice are at 100-0 extremes, corn 98-4, oats 93-1, cotton 93-20, lumber 92-14. The latter makes sense - it reflects the housing market and consumer staples (leprosy). Gold and oil are mid-range and inconclusive by this interpretation. The closest thing to a short are hogs at 24-63, which is not really considered to be an extreme. (Commercials seem to have not much of a market to sell into and must be cutting back production or holding back inventory because prices suck. Anyway, that's the way I might interpret it.) Would I rush out and by copper and rice right away? Not necessarily. But to buy copper stocks right now to invest for a couple of years might not turn out too badly.
Larry Williams adds:
Not that I know it all, but a little more than most I kid myself, so I will comment what Lees is doing is 10 years behind the times and fails to take into consideration price levels–a critical point.
COT is actually entering bullish area for hogs—2 weeks ago entered bullish are for lumber but not a great buy point due to price levels. Commercials buy all the way down as they take delivery and use the stuff—they are not spec buyers—and that must be factored in….same in copper sure there's been commercial buying, but low price levels induced it.
James Goldcamp writes:
Conceptually I've always had a hard time taking COT serious in markets where the futures markets are not a significant portion of the overall business such as SP(stocks). Isn't the SP overshadowed by the cash market for stocks (where many of the big players such as mutual funds don't use futures at all or minimally)? The same would seem to be true for the currencies. Perhaps you might argue (with respect to markets where the futures are not significant portion of the overall $ traded) the structure of who is positioned in what manner is indicative of sentiment ; however, I cant believe it's driving anything.
Larry Williams replies:
currencies have a very strong commercial influence; international corps protecting sales in various currencies.— keep looking »
- March 2015
- February 2015
- January 2015
- December 2014
- November 2014
- October 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- May 2013
- April 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- October 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
- Older Archives
Resources & Links
- The Letters Prize
- Pre-2007 Victor Niederhoffer Posts
- Vic’s NYC Junto
- Reading List
- Programming in 60 Seconds
- The Objectivist Center
- Foundation for Economic Education
- Dick Sears' G.T. Index
- Pre-2007 Daily Speculations
- Laurel & Vics' Worldly Investor Articles