Bill McBride published this interesting piece on wage growth in the US.

On the one hand, one might argue that this is a surefire harbinger of inflation. On the other, some wage growth might carry with it some opportunity for increased spending (save? in this country??). Some top line growth would, I'm sure, be appreciated by one and all.

And that assumes that there really is wage growth going on. At best, the jury's still out on that one.

Bill Rafter writes: 

Wage growth has not been underestimated. Payroll tax receipts suggest otherwise. The latter do so some signs of coming back from the grave, but absolutely nothing to get excited about.

Regarding inflation, there are two forms of money growth that have to be monitored: that originated by the Fed known as the Monetary Aggregates, and that originated by the banking system known as fractional reserve lending. The aggregates are the Monetary Base, M2 and MZM. The lending data are commercial and industrial loans. The planned growth of the aggregates is designed to limit deflation. Inflation will not proceed apace until you get a growth in loans. So if you are worried about inflation, at this time all you have to watch is the loan data.

Aggregates and loan data are available on the FRED site. Payroll taxes are on the Treasury site.



 The Riddle of the Labyrinth by Margalit Fox is a great book describing the decipherment of Linear B, a Bronze Age pre-Homeric script found originally on tablets in the Palace of Minos on Crete. If that is of interest to you, this book will reward you. For me it was a quick and exciting read. If you are a Sherlock Holmes fan, chances are you will enjoy it.

The decipherment of Egyptian Hieroglyphics was solvable once the Rosetta Stone was found, which contained a translation into Greek. However Linear B looking like stick figures or the runic alphabet, had no comparable Cliff Notes.

But I also found the book an excellent guide for anyone interested in doing research on market behavior. The parallels between the two were uncanny. To decipher Linear B required pattern analysis, counting and frequency analysis before there were computers to make those tasks easier. We have computers to aid our decipherment of the markets, but the process of creating a framework to do the research is the same. A lot of setup and then lots and lots of actual work.



 "Nobel winner Fama: Active management 'never' good":

Eugene Fama, the University of Chicago investing researcher who won the Nobel Prize in economics last year, once again warned investors against the lure of active management.

"The question is when is active management good? The answer is never," Fama said to laughs Thursday at the Morningstar ETF Conference in Chicago .

"If active managers win, it has to be at the expense of other active managers. And when you add them all up, the returns of active managers have to be literally zero, before costs. Then after costs, it's a big negative sign," Fama added.

He's known as the father of the efficient-markets theory, which says that asset prices reflect all available information; investment managers can never truly get an edge.

Fama dismissed the idea that it was possible to pick the best managers.

"The good ones might be good or they might be lucky. The bad ones might be bad or they might be unlucky. We can't really tell the difference," he said. "I don't know if it would ever make sense, even if the fees were zero, I don't think you'd be better off because you'd be investing in an undiversified way."

Read More Economy weak because of 'stupid' policies: JPMorgan pro

Asked about Warren Buffett's long-term record of picking good companies, Fama said the Berkshire Hathaway (BRK-A) chief actually agreed with his index-based thesis. Buffett said recently he actually has directed much of his fortune to be placed in passive index funds after he dies.

"He's, like, my hero," Fama said. "What he says is, 'I can pick a company every couple years, but if you have to form a portfolio, you're better off going passive.'"

"All the behavioral people say the same thing," Fama added. "In the end, they realize that the game of doing something active is fraught with problems."

Fama was also asked about hedging against big crashes, like what happened to the markets in 2008. Attempting to protect against them, he said, was the unwinnable game of market-timing.

"If you sold when the market crashed, you made a big mistake, and if you saw it coming you're a genius," Fama said.

Gary Rogan writes:

Everything that The Sage deems right and proper will happen after he dies, the charities, index investing, who knows what else. I guess it's no longer politically correct to say "Après nous, le déluge".

The statement "If active managers win, it has to be at the expense of other active managers. And when you add them all up, the returns of active managers have to be literally zero, before costs." is probably mostly correct but given that some active managers are also activist managers it's not completely correct. Also imagine that every single person in the world was an index investor, that would be an absurd situation where nothing in particular but the inflow of new money would determine the price of all stocks. And still, if the average of all managers, aren't some managers better than indexing? At the very least Fama could say that no person is capable of either being or choosing a better-than-average active manager, but he isn't actually saying this.

Bill Rafter writes: 

That's a poor logical argument by the good professor. While Dr. Fama may be right that before costs the average return of all active managers must be zero, clearly it is possible (if not likely) that there will be serial winners and losers. Speaking only of the latter, several years ago we were asked to propose solutions to a shop that had managed to underperform the S&P for every one of the prior 15 years. They did not like our proposals and also rejected proposals from other research providers, continuing with their own methods. They are now 0-18 versus the S&P. Since it is possible for some to get this investment "thing" totally wrong, it is perfectly logical to assume that some others have better than average performance with consistency.

anonymous writes: 

In the case of Buffett you might ask: cui bono? His non Berkshire index assets could fill an Omaha thimble. Is it not the same press release as Betfair put out about their fixed odds versus exchange book on the Scots referendum?



Would anyone advise on how to determine backtesting periods?

I presume one should choose the most recent period because it may better correlate with the present situation. But is that really true? If it is, then how far back should one include, and how far in the future can it correlate? My experience seems to say that a short backtest period can lead to a very short future prediction or even a very poor prediction. On the other hand, a longer period often leads to poor performances during the present situation.

Shane James replies: 

At the Spec Party I had the privilege to spend a reasonable period of time one to one with the remarkable Sam Eisenstadt.

His work is likely one of the best examples of creative thought in the history of financial markets. He explained to me that there wasn't much backtesting to what he/they did. He came up with some principles that made sense to him and started applying them in real time.

Now, in our so called modern world, things may have moved on (Sam graciously stated as much to the room when he was giving his views on the modern markets). HOWEVER, maybe not so much…..

Try this:

1. If your trading idea has an average holding period of a few days (preferably less) then start from today and run it in real time for the next 90 days or so. By definition, the prices upon which you are testing your ideas did not exist when you had the idea so you have already eliminated most bias if you do this.

2. If you are happy with the structure of the returns (win, lose or draw) then consider if the results were biased by any factor during your live test phase and if related to long only stock index trading then make the requisite adjustments for drift.

3. Perhaps now consider a backtest.

The point being that I think it makes sense to test on data that did not exist BEFORE you perform the backtest.
Some like to 'exclude' certain data and 'pretend' it didn't exist so they can assume that the excluded data is 'out of sample'. For instance they may take 10 years of data and use the odd number years as test data and the even number years as 'out of sample'. This might be a reasonable idea to make yourself feel more comfortable but there is an intangible and very difficult to explain benefit to performing the kind of 'spontaneous' testing set out above on data that did not exist at the genesis of your idea before one starts seeing how well a set of heuristics performed in 1971!

Leo Jia responds: 

Hi Shane!

Thanks very much for the valuable advice.

Wow, Mr Eisenstadt! I would really love to thank him for my early success stories with referencing the Value Line. But I guess it wouldn't matter to him as he might have heard from too many!

Talking about my early experience (back in the 90's), I actually had been using your suggestion all along. There was never backtesting for me — I got an idea and went to buy the stock the next day. It actually worked well overall.

Should I go back doing the "novice" way? That becomes a question worth thinking now that you mentioned it. Perhaps this goes with the valuable lessons where having had enough struggles using complex ways, one discovered the neglected simple way being far superior. In Chinese culture, Tai Chi can be considered as that type of "simple ways".

Now, a couple questions about your suggestion.

1. By putting a new idea directly live, what problem is one trying to solve? Is it the concern that poor backtesting result may make one throw out potentially a good strategy? And is this concern because of the belief that past data are already different from the present situation?

2. In what ways can this idea that seemed to come from nowhere be better than the many ideas one gets by studying historical data? I know inspirations are invaluable, but one doesn't often get those inspirations that are not the results of study. So beyond the mistrust of the correlations between past data and present situation, are there any other reasons?

Thanks again for your thoughts.

Bill Rafter writes:

I am sorry to jump into this discussion late, but think there are a few points that can still be brought.  Looking for beta over a constant period of time (say 6 months) is somewhat meaningless and useless.  It’s a bit like describing a man with one foot in a fire and another in ice as at a tolerable temperature.  You have got fat tails with market volatility and a static window might be good for a journalist, but of limited value for a trader.

At a given time there is a time period over which the study of a market’s behavior will be significant.  And let’s say that at this time it really is 6 months, or 126 trading days.  Assuming no real changes, tomorrow that time window will be 127 trading days, and so on until you get a market change.

When the sea does change, bad things can happen in a hurry and beta value for the preceding 6+ months will be of little value.  Within the last week this happened with biotech:  it had been happily chugging along with good but not extraordinary outperformance of the indices.  Then it got clobbered with huge excessive relative volatility to the downside.  Had you been adapting your monitoring of volatility you would have been prepared, whereas if you stuck with your 6-month window you would have been clobbered along with the group.

My advice to you is to learn how to deal with the market adaptively.  I assure you that if you have a monitoring mechanism which you like, if you make it adaptive you will improve results dramatically. And it doesn’t matter which signal type (momentum, volatility, sentiment) or time frame (intra-day to weekly) you favor.



 A hundred years ago Milutin Milankovich, a Serbian scientist/engineer, didn't have much to do as he was a POW held by the Austrians. So he calculated the pre-historical temperatures of the Earth, based entirely on planetary distances to the sun. Several other scientists persuaded him to go back quite far in time and eventually he calculated the temperatures back a million years. Of course at that time there was no way to prove his work, until in the 1970s data from Antarctic ice cores became available. It turns out his calculations were very accurate, as were similar calculations for Mars and Venus.

If someone a century ago could calculate Earth's temperature a million years ago, the global warming claims of one camp seem to lack significant credibility.

Stefan Jovanovich writes: 

Milankovic's theory is this: "variations in eccentricity, axial tilt, and precession of the Earth's orbit determine climatic patterns on Earth"

The theory of the warmist researchers is that "the addition of combustion gases - most importantly, CO2 - from man-made uses of energy to the earth's atmosphere determine climatic patterns on Earth".

The reason for the falsifications of data by warmist researchers– I assume here that no one denies that these have occurred– is that the theory of man-made global warming requires a dramatic increase both in temperature and CO2 levels during the period when people have been burning stuff. If that cannot be found, then the theory has to contend with the very data that Al Gore found so persuasive– the Vostok ice core samples– and explain why CO2 level increases seem to be a result rather than a cause of the rise in the earth's surface temperature. That non-modeled data (i.e. the ice cores were actually dug out of the earth, not created in a computer model) is inconvenient and true. The Vostok data shows that changes in temperature always precede the changes in atmospheric CO2 by about 500-1500 years.

The usual rebuttal to this evidence and the fact that its data is entirely consistent with the Milankovic theory is something like this: "yes, it's true there is a delayed correlation; but that ignores the more important fact. Once the rise in CO2 levels start, they take over as the most important climate force."

But here, too, the actual non-modeled data presents a problem; the declines in earth surface temperatures that begin the "ice ages" occur precisely when CO2 levels are at their highest. If the Hansen theory's forces are so strong and can overwhelm the mere changes in the Earth's orbit, then how can the 'weak' signal can start an Ice Age when the strong Hansen signal says the opposite should be occurring?

The answer to that, of course, is the usual ad hominems that are the ever available rhetoric of the progressive mind: (1) you don't understand, (2) you haven't read our secret data and (3) you are too stupid to understand these things.

I think we have another definitional problem here, HA. "Complete(ly) unbiased description(s) of meteorology-climatology science practices" do not get written by people who write: "as a historical science, the study of climate change will always involve revisiting old data, correcting, modeling, and revising our picture of the climatic past. This does not mean we don't know anything. (We do.) And it also does not mean that climate data or climate models might turn out to be wildly wrong. (They won't.)"



 Yesterday while driving I heard a report of strong auto sales of both domestic vehicles (particularly trucks) and BMW and Audi. These would show up in the Daily Treasury Report as revenues in categories such as customs duties and excise taxes. Today I went looking for them, and sure enough the recent data is positive.

I tend to think of those categories as a good upstream surrogate for discretionary purchases. There are excise taxes on auto sales, gasoline sales and even on tanning salon sales.

In the linked chart
, SPX is shown as an historical reference. In my opinion there is not a definitive causal relationship. Historically this had been distorted by the "Cash for Clunkers Program", for example.

But maybe there is a retail recovery.



Does anyone know if there is a Predictive Value to a stock's short interest ratio?

Bill Rafter writes:

Short Interest (SI) is a good area to research. We do a lot of work with it in our shop, and use it in our trading. However, the question you posted was specifically about the SI Ratio, something we consider unworthy of attention with a very few exceptions. If that ratio is all you are going to focus on, we suggest watching a good movie instead.

Many people simply look at the SI Ratio because it is available, say on Yahoo, Google or the Nasdaq websites. The problem is that ratio is more dependent upon changes in volume than changes in SI. Volume is also an area worth your attention, but not in that ratio. We maintain that there are better SI ratios to look at rather than that one. But to do that you are going to have to spend some time getting the data, which means not only SI and volume, but outstanding shares, insider ownership and institutional ownership. Then you will find the profitable relationships, but anticipate considerable work.

We have only found the volume contributor to the SI Ratio useful when in a price explosion the volume exceeds the number of shorts. That circumstance suggests that the price explosion (of a high-SI stock) is a result of short covering, which has now been exhausted. Obviously don't buy that stock!

Phil Erlanger is the regarded expert with SI data. His approach was to find stocks that one liked (say on the basis of momentum or whatever) and then look for SI patterns that would enable a greater run-up. We took the opposite approach, looking to first find good short interest patterns, and go from there. What we found was that Erlanger's approach is the better of the two if one is taking a cursory look at SI. That's because fully half of the stocks with high SI deserve it – they are headed south. Of the remaining percentage, about half of those mill around going nowhere. That leaves about a quarter of high-SI stocks overall that benefit positively, a few of which really take off.

Despite the above warnings, we would not purchase a stock without at least making ourselves aware of the SI.



For historical reasons I manually downloaded the Daily Treasury Statement files and dumped them in a folder. Once there we go through our data mining process and extract what we want automatically. Our process could be made completely automatic, but it has not been a big enough inconvenience for us to code it. For virtually all other data our downloading and extraction is completely automatic.

Several weeks ago I noticed a change in the Treasury's website that irregularly makes me click once or twice more each time I download (which is only once daily). It has puzzled me why Treasury would take something that worked perfectly and change it such that it no longer worked perfectly. It has just occurred to me that the new little two-step process would certainly screw up an automated download and extraction procedure. Also of late the data is less and less favorable to a government that may wish to claim everything is rosy.

Am I being paranoid in thinking that there might be a connection?



One wonders if the stooges, the puppets from the centrals will be hauled out to make reassuring comments about the health of the economy and the resonance of the qe's. After all, small people in emerging markets might be hurt and the idea that has the world in its grip will come into play. Trading it from that cynical world view has not been entirely unprofitable the last two days. But it was entirely unprofitable on Monday. However, it often takes a day for the puppets to receive their marching orders.

Rocky Humbert writes: 

I note a Bloomberg news story from this morning that the INVERSE VIX ETF (XIV) had a record inflow of money last week — the largest amount since the ETF started trading in 2010. This tells me that the market has become conditioned to extrapolating the behavior of the past five years.

I believe that among the biggest challenges in investing and running one's models is figuring out when the game has changed (or "ever changing cycles").

I am not making a prediction about when the game will change. But the risk is rising substantially. Conditions precedent for the game changing are (1) "Everyone" is conditioned for the same behavior; (2) High leverage in the system; (3) Rich valuations and/or optimistic assumptions; (4) Subtle changes in monetary conditions and/or other related expectations; (5) A long period of time since things looked really scary. (FWIW NYSE December Margin levels are at records fwiw.)

Think back a few years — what were you thinking then? How many people laughed at "Green Shoots"? Why do people believe the bankers now? But they didn't back then? What is different? I'll predict that we don't have another financial calamity. But to quote the wisdom of Roseanne Roseannadanna, "If it's not one thing, it's another."

Bill Rafter writes:

For the next shoe to drop you may want to look at my post of last week.

Gary Rogan writes: 

When I said we'll see 5% down I was using every one of those reasons other than 4 that I don't understand other than slightly lower QE. The margin leverage chart is the scariest thing in the world if you are looking for scary things.



 What are the major 3 body markets that orbit around each other in our solar market system and how do their epicyclic orbits relate to each other (in the future)?

Bill Rafter writes: 

I think the most important word in the Chair's sentence is "epicyclic", specifically because it is non-linear. Stocks specifically exhibit non-linear behavior, and seeming have forever. Bonds used to behave very linearly, but now behave similarly to stocks, although contrarily so. We have yet to find the defining characteristics of currency markets, but keep trying, hoping to find useful information relating to other markets. Gold is also a tough one, making one think it is a rigged game. REITS behave like a hybrid equity-debt vehicle. We tend to think of REITS as a free market version of the variable annuity (but without the huge vig).

Shane James writes: 

Arguably, and addressing prediction, the big 3 change regularly.

Simple stuff like the listing the biggest moves in X time periods is a useful, elementary starting point for cross market prediction.

Anton Johnson writes: 

Sadly, our system is unstable with the sub-stellar central mass consisting of the collective Central Banks. Orbiting, and sometimes consumed by, the central mass are the various financial instruments periodically switching in relative predominance as they accrete/disperse assets due to the actions of the brown dwarf.



December 30, 2013 | Leave a Comment

 My apologies in advance for a seemingly strange piece of research.

Recently a Speclister posted a link to a site which inferred considerable success in trading various markets on the basis of solar and lunar events. We have all seen these for decades. There are lots of charts that seemingly draw the connection between full and new moons, sunspots, geomagnetic radiation and of course the financial markets. I myself found nothing in the way of serious data that would make me want to trade on that basis, but the site exuded so much confidence that it was hard to dismiss out of hand.

The site like many in the genre spends a lot of space arguing WHY. You know, humans are mostly water and Earth's tides are controlled by solar and lunar gravitation, so why not humans. Personally I don't care what the reason is, as long as a reason exists and the data is non-random. In this case I am going to assume that a reason exists, but is not discernible. So the answer was for me to take a look at the data with our research tools.

My period of study was from January 1, 2005 through December 27, 2013. That could always be enlarged if some worthwhile results were forthcoming. As a benchmark equity asset I used SPY, as it included dividend yield and was a real and tradable market.

Over the period SPY achieved a 7.4 percent compound annual rate of return (CAROR) while experiencing a 60.83 percent maximum drawdown (DD). Thus the return to risk ratio (R/R) was 0.12. Full statistics and a chart are here.

The site made some strong claims about the value of the full and new moon dates, so my first look was there. To look at solar influences I would need a significant number of cycles and they are approximately 11 years each. First I bracketed the half-month on either side of the full moon, and the same with regards to the new moon. With regards to the full moon, you would buy SPY at the first quarter and hold for the half-month through the full moon, selling at the third quarter. When you were out of the market you were in cash, earning nothing. Thus the following constitute programs in which you are only invested for half the possible time:

Full Moon Bracketing:           2.1% CAROR,    36% DD,     0.06 R/R
New Moon Bracketing:        5.19% CAROR,    47.98% DD,     0.11 R/R

This agreed with the site in that longs would favor the new moon. But if the full and new events corresponded to troughs and peaks, we had to look at equity growth between the events. This also constituted investing for only half the possible time.

New to Full (waxing):        9.82% CAROR,    46.08% DD,     0.21 R/R
Full to New (waning):        -2.2% CAROR,    41.17% DD,     -0.05 R/R

These results would suggest that equity prices tend to trough at the full moon and peak at the new moon, exactly as conveyed by the website.

Links to stats: 






Steve Ellison writes:

To what does the t score of 3.46 refer, and how significant is it given multiple comparisons (you tested 4 subsets of data, and one looked pretty good)?



 My first experience with "serious" fraud was in grammar school. I had advance knowledge and just sat and watched the whole thing come off.

I was either in Fifth or Sixth Grade. My next door neighbor Paul was two years older, and Harry further up the block was in high school. Harry had one of those dream jobs: he worked as an usher at the local theatre for the Saturday kid matinees. It was a dream job because he got to see all the movies for free, and got paid to boot.

This theatre occasionally had giveaways to boost the audience. Well this one time they announced they were giving away a free bicycle (a real stunner) to someone in attendance. All you had to do was be in the theatre with a paid ticket. Of course they announced it for weeks and come the appointed Saturday, the place was packed. Kids were even sitting in the aisles as there were no serious fire regulations. There must have been 400 kids there, every one of which dreamed he was going to win that bike.

I sat next to Paul who told me in advance he was going to win. After the first show (the Saturday kid event was always a double-feature), the manager got up on stage with Harry the usher holding the giant bowl with all the tickets. Harry draws the winning ticket and gives it to the manager, who read out the number. Paul jumps up shouting "I won, I won". The next day Harry was riding around the neighborhood with his new bike. I was too young to inquire about the quid pro quo between Paul and Harry, or even perhaps between the manager and Harry. And of course I was in awe.

In many ways it was beauty in its execution. Not unlike the time the former First Lady of Arkansas used the futures markets to bag a payoff. But that's another story. Here's what made me think of the bicycle giveaway long ago:

Today I saw a news item that if no one wins the current $600+ million lottery and perhaps the next upcoming one, then the jackpot could be $1 billion. With this being the Christmas season, there could not be a better time to avoid anyone winning to run the jackpot up to all-time highs. All those people hoping and praying to hit the big one. All the promoters have to do is look into their computers to find unpurchased numbers for several weeks.

Now I'm not suggesting that they give the winning ticket to one of their buddies, like Harry and Paul arranged with the bicycle. But this could all be done with the goal of redistribution of wealth from those who purchase lotto tickets to the tax coffers of the states, who of course get most of the winnings. The individual winner himself does not matter, he's just window dressing.

Just thinking out loud.



 We have gone almost a year with the two percent additional payroll tax reinstated. The results are worse than expected.

What would have been expected is an increase in employment, but not enough to offset the effective tax increase. The reason you would expect an employment increase is because Americans are a resilient lot and get bored with sitting around. Sooner or later they find a way to get back to work. That is not what we have: The growth in payroll taxes is now negative, indicating a net loss in payrolls. The data is effectively "cap-weighted" so it might mean a loss in the number of jobs or switching to lower pay, as when a nuclear engineer becomes a sanitation engineer.

Philosophically, tax rate increases for individuals generate increases in tax revenue for governments. This is exactly what is expected by government, but the problem is that government does not know where to stop. They expect further rate increases to result in commensurate increases in revenue. But government neglects that individuals have a say in this: the latter can vote with their feet by leaving the workforce. America is now on the wrong side of the Laffer Curve.

Additional amounts taxed (N.B. the PPACA has been ruled by the Supremes as a tax) will have a continued negative effect.

A fellow Spec-Lister suggested I look for structural/secular changes in the employment data. My initial thought was that humans are skilled at obtaining freebies, and the disability payments coming from Social Security seemed a perfect target. Consider, faced with a lay-off, why not see a doctor, claim clinical depression and get yourself on disability? The long-term advantage of doing so may mean that you never have to work again, which would not be the case with unemployment benefits. But is my conspiratorial claim borne out by the data?
The short answer is "No". However there is more, should you feel inclined.

Firstly, which data does one use? Social Security Administration issues a report showing claimants for disability and the average claim. Multiply the two and you get the total value of disability benefits paid. Alternatively, you can go to the Treasury website and see their ledger of what actually was paid. Although the two sources (Soc.Sec. and Treasury) mimic one another, they are decidedly not identical. Of specific concern is that they differ by an odd order of magnitude, and one which is not relatively constant. So then one might posit which source does one trust.

Chart of Disability Benefits Paid

Chart of the 12-month rates of change of benefits paid

My experience suggests that the Social Security data looks as though it has been manipulated or "cleaned up". The Treasury data looks as though it contains a degree of static, which is more realistic. My guess would be that the Treasury data is "raw", while the Social Security data is "adjusted". In general my personal preference is for raw data if I cannot reverse engineer the adjustments. Both data sources indicate a relative decline in the yearly rate of change, decidedly counter to my pre-supposed conspiracy claim.

If you look a little deeper into the Treasury data you find a profound cyclic influence:

Cyclic disability benefits

This was a surprise. I did not assume the claimant had much control over the process, but the data indicates that summer is a key time to receive benefits. Oh, the joy of it all. [Skeptics should note that the cyclicality is not related to the number of days in the various months.] The cyclicality also suggests that disabled persons do return to the workplace. (I would have lost that bet.)

What is the current trend?

trend slope in disability benefits paid

For whatever reason, the drift of disability benefits is not increasing. One might optimistically believe that because conditions are not worsening, they must get better. Such logic could cost an investor a lot of his wealth.

Rocky Humbert replies: 

There was a Washington Post story yesterday that adds some color to this discussion. It notes a fact: 1.3 Million workers will have their "emergency" unemployment benefits end on December 28, unless Congress renews this aid program. This is a big number. And I was unaware of this fact. And as I consider myself somewhat informed about stuff, I'd guess relatively few market participants are aware of this fact either.

The writer then looks at the probability that a lot of these folks will file for disability claims. The author cites a study (which I have not read) which suggests that they won't. I have no opinion except that people respond to incentives. And some number of these 1.3 Million will surely find their way back into the reported labor force. This will likely distort the tax revenue, payroll, and other data to some degree in the first months of 2014.

I am raising this point not because I have any view about the currently big number of people receiving disability or what it means. (That's HR Rogan's job.) Rather, I am raising this, because the employment and tax numbers will, I believe, look really odd in January and February. (HR=hand wringer)

The story can be found here:  "Where Will Workers Go After Their Jobless Benefits Expire? Probably Not on Disability"

Jeff Rollert adds: 

Just to add another vector to the discussion, I would also argue that, since 2000 (the benchmark year in the article), the entry into the global labor pool of hundreds of millions of smart, motivated Chinese workers (not to mention Vietnamese, etc) has had a significant impact.

From the MIT Technology Review: "How Technology Is Destroying Jobs":

Given his calm and reasoned academic demeanor, it is easy to miss just how provocative Erik Brynjolfsson's contention really is. ­Brynjolfsson, a professor at the MIT Sloan School of Management, and his collaborator and coauthor Andrew McAfee have been arguing for the last year and a half that impressive advances in computer technology—from improved industrial robotics to automated translation services—are largely behind the sluggish employment growth of the last 10 to 15 years. Even more ominous for workers, the MIT academics foresee dismal prospects for many types of jobs as these powerful new technologies are increasingly adopted not only in manufacturing, clerical, and retail work but in professions such as law, financial services, education, and medicine.

That robots, automation, and software can replace people might seem obvious to anyone who's worked in automotive manufacturing or as a travel agent. But Brynjolfsson and McAfee's claim is more troubling and controversial. They believe that rapid technological change has been destroying jobs faster than it is creating them, contributing to the stagnation of median income and the growth of inequality in the United States. And, they suspect, something similar is happening in other technologically advanced countries.

Perhaps the most damning piece of evidence, according to Brynjolfsson, is a chart that only an economist could love. In economics, productivity—the amount of economic value created for a given unit of input, such as an hour of labor—is a crucial indicator of growth and wealth creation. It is a measure of progress. On the chart Brynjolfsson likes to show, separate lines represent productivity and total employment in the United States. For years after World War II, the two lines closely tracked each other, with increases in jobs corresponding to increases in productivity. The pattern is clear: as businesses generated more value from their workers, the country as a whole became richer, which fueled more economic activity and created even more jobs. Then, beginning in 2000, the lines diverge; productivity continues to rise robustly, but employment suddenly wilts. By 2011, a significant gap appears between the two lines, showing economic growth with no parallel increase in job creation. Brynjolfsson and McAfee call it the "great decoupling." And Brynjolfsson says he is confident that technology is behind both the healthy growth in productivity and the weak growth in jobs.



Some preliminary thoughts on the running median 2, 3, 4, 1, 7, 8, 9, 3.

A moving median of the first 5 is 3, of the next 5 is 4, of the next 5 is 7, of the next 5 is 8– it's a good indicator of trend. First recommended to me 53 years ago by Fred Mosteller, Chairman of Harvard's first statistics dept.

It is more stable than the moving average as outliers are removed from sample. It is easy to compute fast with computers for small running numbers like 5 or 100 by repeated sorts. For higher numbers, you can form two groups, those below the median and those above. As a new number comes up you place it in one of the two groups if higher or lower and take away the oldest number. Then adjust to make the two groups equal again. It is not used as much as the moving average so it shouldn't be hurt by front running or spikes when cross over occur. It has a defined distribution when the underlying distribution has inordinate extreme values as frequently occurs with Cauchy or similar distributions with infinite variance.

It's probably a good thing to use when using nearest neighbors as predictors, i.e using the median and running median to compute your predictors. It deserves testing in real life markets for real life applications.

Ralph Vince writes:

It is the indicator of "expectation," as evidenced by human behavior itself, and not the probability-weighted mean.

Bill Rafter adds: 

Moving medians have some distinct advantages.

They represent real values that occur. For example, taking the average of 1, 2 and 5 gives you 4, which never occurred, whereas the median 2 did occur. Continuing with the same series, should subsequent values in the series be less than 5, the value of 5 will not occur as a moving median. Hence, the moving median eliminates outliers.

One of my appliances has three thermometers to measure temperature. The value displayed is the median (and hence a series of moving medians). Should one of the thermometers be broken, or distorted by being in a particularly hot or cold spot, the median will still give me the best estimate. This elimination of outliers is very useful.

Should you have data whose importance relies upon only crediting occurring values and need to eliminate outliers, then you should test moving medians. We ourselves had experimented with them regarding price series and written extensively about them, but do not use them in our current work. Our reason is that we consider the outliers in a price series to be particularly important.

Kim Zussman adds:

The following is a plot ratio of SP500 (10 week moving average) / (10 week moving median) for the recent 5 years (SP500 weekly close data).



 For those of you interested in jobs data, this chart might be of interest.

The red line is very important, showing a 2 percent increase. Ceteris paribus the 2013 payroll tax receipts should average 2 percent above 2012. However as time has progressed, the government has received less and less of this increase, and the current receipts growths are running negative to the prior year despite the increase in rate. This is the Laffer Curve at work.

As of January 2014 the YOY growth will use as its base the tax-increased 2013 data, which should be interesting.



I have a model which at its root is theoretically (but not operationally) similar to the Fed Model, and its job it to tell me where to allocate assets among equities, debt, gold and/or REITS. I also include a few other items as 'tracer bullets'. At this time the allocation model would have most of its money in equities, and importantly no money in REITS. However when I look at my list of 30 stocks to buy, 23 of them are REITS and 2 are utilities. So if I have to rotate out of something, my only choice is cash.

Could this suggest something ominous?



 It's funny that the jobs report is not compiled yet. The Labor Dept. must have the data they use, as that report consists of happenings through 9/12. We use Dept. of Treasury as our source and we have that information through 9/27. The Treasury data is generated electronically and we might get the 9/30 report later today unless they intervene.

Bottom Line: The YOY growth in payroll tax receipts (seasonally adjusted), which is our substitute for employment, is at the lowest level of the year, whether you mean calendar year or adjusted fiscal year. But of course, you might never see that report.

Let's say you were in charge of the Administration of a country in a similar circumstance. If you knew the jobs data was fantastic, would you release it? A good economic report might be taken to mean that the country was not as fragile as previously thought, and could therefore withstand a shutdown for a while. On the other hand, if the jobs data were bad, it might mean the country was very fragile, and that the Administration should compromise quickly, effectively forcing your hand. And of course in the latter scenario you should be embarrassed by the fact that nothing you had done economically for 5 years had been successful. Your best option might be to wait until you needed a trump card, and then pull it out of the hat. Plus (if you wanted) you would have additional time to massage the data.



Attached is a weekly chart of CSI300 index (representing 300 large stocks on Shanghai and Shenzhen exchange) from January 2007 to now.

Would anyone call an upcoming bull market from this?

Perhaps the chart is not too obvious yet. Fundamentally, it is true that many foresee a slowdown in GDP growth in the coming years. But what is important now is that people can anticipate some structurally healthy growth. And this is very different from the past 5 years when the growth seemed high but the market mainly saw it as unhealthy and stayed essentially hopeless. The new government seems to deliver a lot more confidence to the market with a new direction for the economy.

Any thoughts?

Bill Rafter writes: 

One suggestion I have is that you ask yourself two questions:

1. Consider the participants in that market; what time frame do they typically observe in terms of long term perspective (i.e. lookback period), and

2. How frequently do they watch the market?

The reason to care what others do is because they are your competition. The money you make, you get from them. Thus, know them!

Point #1 may also be related to taxation. Is there a period of time in China such that if a position is held that long it qualifies for a tax break? In the U.S. that means it qualifies as a "long term capital gain" with a significantly reduced amount going to the confiscatory government.

If there is no such period, then it's nice to see history going back to 2007, but it is irrelevant to what is happening now. However it is good to have history as you can easily see with a visual how a market behaves with the signal process you use. You should statistically test, of course, but a quick look is valuable. (Tukey said so, and he is a god in this area.)

Thus your window of observation for decision making (as opposed to history) should not go back perhaps more that 50 percent greater than the period identified in point #1. In our case (in the U.S. with equities), we do not look back farther than a year and a half. Frequently as little as four days.

Point #2 is the shorter end. If everyone watches the market every day, then by limiting your snapshots to weekly, you are discarding valuable information. Ask yourself, "Why would you ever want to eliminate valuable data?" You would not do that with a neural net, so why do it with real intelligence? Some would posit that weekly information (data or charts) eliminates some noise. However we would argue (and have demonstrated) that it is impossible to separate signal from noise. Specifically I would suggest that if someone gave me what they considered noise, I could find some signal within. It may not be the best example of signal, but it's in there.

Leo Jia adds: 

Thank you very much, Bill, for the precious advice.

There are a couple reasons for me to have attached the weekly chart starting from 2007.

1. I look for a possible multi-year bull market, and for that to me the trend looks clearer on the weekly chart.

2. One key reason for the past few years' laggard market, aside from those fundamental reasons I outlined, is the bull-run and crash in 2007-2008. The bull-run was solely due to the government reform initiative in the stock market which tried to ensure all shares (government shares and floating shares) to be equal. The crash then was mainly due to market suspicion that the resulting floatable government shares would subsequently flood the market. Now 5 years over, the flooding of the government shares, if that happened indeed, is likely to have settled down.

To answer your two questions:

1. There is no tax incentive in China encouraging people to hold longer. Holding period are generally much shorter. It can be as short as a few months for funds, and as short as a few days for individuals.

2. Most participants watch the market everyday.

Perhaps one thing different in China's market is that large market movements are all initiated by government policies. Market enthusiasm are only summoned when the imagination of a government direction as positive.

I am not a government analyst, but traditionally, each government in its 10 years tended to create at least one big upward move in the market. Looking at this government, its initial months already showed signs of its focus on finance (along with new direction on economy). The recent launch of bond futures is one such key move.

img.imageResizerActiveClass{cursor:nw-resize !important;outline:1px dashed black !important;} img.imageResizerChangedClass{z-index:300 !important;max-width:none !important;max-height:none !important;} img.imageResizerBoxClass{margin:auto; z-index:99999 !important; position:fixed; top:0; left:0; right:0; bottom:0; border:1px solid white; outline:1px solid black;}



 Voyager 1, launched back in 1977, has become the first man-made object to pass into the unknown vastness of interstellar space. News Report.

I have a serious challenge for you. Name a single man-made device that has worked continuously for 40+ years without any human physical intervention. The winner will receive Rocky's usual prize: A unique gift of dubious monetary value.

Chris Cooper has a go at it: 

There must be any number of vintage self-winding watches that still work. If it must be wound, does that still match the spirit of your inquiry? Of course, there are many watches and clocks which must be wound by hand that are still operating. You can find some self-winding watches for sale on eBay.

Kim Zussman replies:

I am man-made and have worked continuously for well over 40 years (though currently half time for the government).

Bill Rafter adds:

Without doing any looking, there are lots of low-tech human creations that have survived the test of time. Many dams have performed their functions for decades and even centuries. I'm not speaking of hydroelectric dams, but simple river control devices. The Marib dam in Yemen is still there (after two millennia) and would be working if there was enough rainfall. Many artificial harbors also have exceptional longevity. Some Roman harbor constructions are still operational; the Romans having been expert in concrete manufacture. And don't forget Roman roads.

In more recent times, I am certain there is some electrical cable that is still functioning from half a century ago, if only to ground lightning rods.



There is an issue about the employment numbers that may not be getting proper attention - Section 530 and its interaction with state unemployment benefits. Section 530 of the Revenue Act of 1978 was the Carter Administration's gift to the farm belt. Under Section 530 an individual will not be classified as an employee if the alleged employer has a reasonable basis for treating that person as an independent contractor. "Reasonable basis" can be proved by:

(1) "Judicial precedent, published rulings, or technical advice with respect to the taxpayer, or a letter ruling to the taxpayer; (2) "A past IRS audit of the taxpayer in which there was no assessment attributable to the treatment (for employment tax purposes) of the individuals holding positions substantially similar to the position held by this individual"; or (3) "Long-standing recognized practice of a significant segment of the industry in which the individual was engaged."

The IRS has a "whistle-blower" form that individuals can file to challenge their classification - the SS-8. But - and here is the kicker - on the form itself the IRS warns the taxpayer that "A Form SS-8 should not be filed for supplemental wage issues." What this means, in real terms, is that people who get "fired" from their independent contractor jobs cannot use the IRS to bully state unemployment agencies into paying them benefits.

Since the states all have incentives to cut down on the cash drain from unemployment benefits, even the deep blue ones like California do not make much effort to reclassify contractors as employees once the issue gets to unemployment benefits. the result is that "the workforce" has more and more people in it who are not now and never will be classified as "employees". "Employment" itself becomes less and less of an indicator of actual incomes because the payroll numbers cannot reflect the contractors' fortunes (both good and bad).

Bill Rafter writes: 

For the "percent unemployed" number, reclassification as to who is or is not an employee may have an impact.  However this is the beauty of simply looking at the payroll tax data, as all persons (traditional employees and individual contractors) are required to pay.

Victor Niederhoffer writes: 

But with all the seasonal adjustments and other things that enter the employment numbers, how can payroll numbers not using the census seasonal adjustments be meaningfully compared. 

Bill Rafter elaborates:

It is the seasonal adjustments by the officials that we distrust. We think the adjustments are a fudge factor to be used by an administration eager to paint a picture. I don't know who is responsible (BLS or Census), but their adjustments historically have made little sense. BTW, the Fed also could use someone better at seasonal adjustment, although their number jockeys are better than whoever plays with the payroll data.

A problem is (a) do you want the truth, or (b) do you want to make money? If you are decent at it, doing your own work will get you the truth. However if the world follows the official releases as gospel, you could be right and broke. I have been in that predicament a few times.



Is an asset up or down? How do you decide?

For a somewhat offbeat reason we need an unwavering determination as to whether or not a particular asset was up or down. We do not care how much. Obviously if something is up or down by say 2 percent, there is no argument. The problem is if the data is not definitive. After all, you occasionally have days when the Dow or S&P are one way and the Nasdaq is the other. So which one is right?

The standard is, of course, the close. That would be right in many ways. Most volume occurs at or near the close, and margin calls are determined by the close. But many [technicians] use midrange, or an average of the High, Low and Close. Institutions have been known to care about the volume-weighted average price or VWAP. A priori we thought VWAP would be best for our purposes. But we were wrong.

Ours was a very limited study. We only cared about 4 assets (all ETFs): SPY, IEF, GLD, IYR. And our definition of right vs. wrong is the amount of flip-flopping during a trend. That is, how often is it wrong? We realize this is all very subjective, but we are not writing a thesis here - we just want the quick and dirty facts. The period we considered: 2005 through the present.

It turns out that VWAP is not best. It gives a lot of false signals. This was good news for us as we will not have to acquire VWAP data.

That's all we really cared about. However the fact that institutions take care in getting VWAP price executions and the fact that VWAP (at least in the limited study) gives false information, suggests that someone (a flexion, perhaps) has something at stake to effect the false information.



This is a visual representation of non-payroll tax receipts by Uncle Sam. Now I fully know that corporations and individuals are incentivized to find accountants who will keep these numbers as low as possible, but that tendency does not change over time.



 This article shows results of experiment on the E-Coli bacteria detailing the survival or death of the bacteria in response to the way it handles introduced exogenous stimuli. The upshot is that small changes in exogenous conditions can lead to large substantial differences in outcomes. Surely a rich field for market related phenomena looking at how small changes in one input (say rates) may lead to large movement in other markets (say currencies) when the dependent variable is already under some stress.

Pitt T. Maner III writes: 

This is a really interesting field.

It looks like bacteria have been "hedging their bets" for quite some time. And they have a type of "memory" that influences their response to current environmental conditions. On a larger scale it is interesting to note what happens to the ecology of a system when a "keystone species" is removed. The field of "synthetic ecology/biology" looks to have important findings for a wide range of fields and the bacterial algorithms already developed are being used for engineering problems.

1. "Bet-hedging in stochastically switching environments":

"We investigate the evolution of bet-hedging in a population that experiences a stochastically switching environment by means of adaptive dynamics. The aim is to extend known results to the situation at hand, and to deepen the understanding of the range of validity of these results. We find three different types of evolutionarily stable strategies (ESSs) depending on the frequency at which the environment changes: for a rapid change, a monomorphic phenotype adapted to the mean environment; for an intermediate range, a bimorphic bet-hedging phenotype; for slowly changing environments, a monomorphic phenotype adapted to the current environment. While the last result is only obtained by means of heuristic arguments and simulations, the first two results are based on the analysis of Lyapunov exponents for stochastically switching systems."

2. "Memory in Microbes: Quantifying History-Dependent Behavior in a Bacterium":

"Your average bacterium is unlikely to recite π to 15 places or compose a symphony. Yet evidence is mounting that these 'simple' cells contain complex control circuitry capable of generating multi-stable behaviors and other complex dynamics that have been conceptually linked to memory in other systems. And though few would call this phenomenon memory in the 'human' sense, it has long been known that bacterial cells that have experienced different environmental histories may respond differently to current conditions [1]–[3]. Though some of these history-dependent behavioral differences may be physically necessary consequences of the prior history, and thus some might argue insignificant, other behavioral differences may be controllable and therefore selectable and even fitness enhancing manifestations of memory."

3. The work of Professor Robert T. Paine and the concept of the "keystone species" where an organism has a big effect relative to its abundance:

"It was a ritual that began in 1963, on an 8-metre stretch of shore in Makah Bay, Washington. The bay's rocky intertidal zone normally hosts a thriving community of mussels, barnacles, limpets, anemones and algae. But it changed completely after Paine banished the starfish. The barnacles that the sea star (Pisaster ochraceus) usually ate advanced through the predator-free zone, and were later replaced by mussels. These invaders crowded out the algae and limpets, which fled for less competitive pastures. Within a year, the total number of species had halved: a diverse tidal wonderland became a black monoculture of mussels1."

anonymous adds: 

 OK, what about Slime Molds (particularly, Dictyostelium discoideum). It has the absolutely stunning biological characteristic that it spends much of its life as thousands of individual cells and other times as a single entity.

When times are good for Dictyostelium doscoideum its 'cells' wander off and enjoy themselves. However, in less hospitable environments the 'swarm' of cells coalesce and form a single entity.

Apparently the cells emit acrasion (or AMP) that contains information useful for other cells

When things are starting to look tough the cells pump out increasing amounts of AMP and the cells begin to cluster….Other cells follow these trails and increase to mass towards it completed whole.

Now, I wonder about the stock market. During the regular upward movements most of the components are doing their own thing, following their oscillations generally higher…. When 'it' hits the fan, the correlations between the stocks increase rapidly to 1.0 and they form a single bearish, growling entity.

Now without pushing the analogy too far, I wonder if stocks 'transmit' statistical information (AMP to follow the analogy) to each other (in this context they would not transmit as much as 'exhibit' some form of common statistical behaviour) that forced the behaviour of component stocks into a more correlated state.

Testing possibilities are legion.

Gary Rogan writes: 

My general objections to giving some purpose to the market have to do with incentives, or more precisely lack thereof to do anything in particular.

I read a whole chapter of a book on a slime mold presented as an altruism study. The reason it was presented like that is that when the individual slime mold cells cooperate, only the lucky few that join the growing "mushroom" at the right time get to propagate because they get to form spores only at a particular state of development of the hastily arranged colony. Nevertheless, when presented with a choice of dying for sure or maybe propagating (and the cells only cooperate when they are close to death) they chose to cooperate and propagate. There is also some amount of deception involved when the cells jokey for position, but not a lot, since any particular placement is hard to achieve.

What is the equivalent reason for stocks to cooperate?

Bill Rafter writes: 

Should what you say about stocks transmitting statistical information occur, it would mean a relative decline of idiosyncratic volatility. That is something we have studied, and found that when the going gets tough, the idiosyncratic vol grows faster than the market's vol.There are some other measures of "group think" that are good indicators of both the broad markets and individual assets.

I would posit that stocks do not transmit info, but their owners do. Consider the case of futures in which one market takes such a hit as to require significant margin calls. Human nature being what it is, the public sells its winners to finance its losers, and non-related markets dive along with the primary.



 I heard there is a new open source Python library 'PySEC' allows easy access to all of the SEC's filings.

This is interesting primarily because we are in our 11th week of programming to do essentially what this guy says he has done. Our goal is to glean all of the SEC submissions without human intervention. Many of the commercial data suppliers use the "thousand scribes" method in which they hire a thousand people in a developing nation to manually record and categorize data. And those commercial suppliers charge huge fees for that suspect data.

Does the Python programmer really have something? Have our 11 weeks (to date) been a fruitless exercise?

Prior to 2010 the SEC required submission of quarterly and annual reports to be postable on the web. However there are all manner of idiosyncratic ways in which that information can be posted. Most of the submissions can be mined by a computer, but the fact that we are still programming after 11 weeks suggests it isn't simple.

The vast majority of files are text files. However that does not make mining easy, as labeling of the data is not consistent. Many data items within a given 10-Q may be labeled "total assets" perhaps for each subsidiary. Total liabilities are frequently called something else, or not labeled at all. Then in 2010 it was required that the files be submitted in HTML. Then that requirement was changed to XML, but HTML has appeared to survive. Within submissions we occasionally see an extraneous dingbat dropped into a label, which screws up the mining operation. There is only one submission that has completely stymied us - where the company presented their financial results as an attached GIF file.

We are highly suspect of data that is difficult to mine. Maybe extraneous dingbats have been put there deliberately to foil such a search, or maybe the person responsible is merely trying to impress a boss. But it is enough for us to log the difficulties and research subsequent performance of those problematic submitters. That we will provide to the list, but we will most likely have to abstain from providing a list of the miscreants. We would be happy to hear from any lawyers on the list about that one.

The Python program appears to have made some progress in mining the XML submissions from 2010 but it is a tedious one-by-one search. And now that many of the submissions are back in HTML, the miner has much more work to do for the same effort. So we certainly aren't going to give up our work and pay homage to the Python program.



 The quote below is from Round Ireland with a Fridge by Tony Hanks. I needed something relatively fun and mindless to read and it was recommended by a friend. The book is a lot of fun and I never expected to find anything deeper.

I liked the idea of doing all you could to reduce the chances of you, as an old person, saying 'if only'.

… 'If onlys' are inevitable, an inescapable part of life. If only that plane hadn't crashed, if only that volcano hadn't erupted, if only I hadn't stepped in that dogshit. The trick is to be masters of our own destiny in so far as we have control, and take the rest on the chin with a wry smile. But we must go for it. Only a fool would squander the rich opportunities which life affords us."

Shane James writes:

When one reads about successful individuals in business fields this sort of thing always comes up.

Narrowing the field of individuals massively to include only Financial Market types one sees, to a man, that they all took massively outsized risks in their early days that just happened to pay off. All the heads of the current brand name hedge funds fall into this category. (As a quick aside, Portfolio Managers in these same funds now lose their allocations with drawdowns circa. 3%.) Quite harsh when the 'names' themselves used to swing 50%. The survivor bias is massive. It is not enough just to have the positive attitude of a Richard Branson or the taking all opportunities like a Paul Tudor Jones or the massive leveraged bets of a middle aged Palindrome…One still needs to have figured an edge.

So, I think one should take every opportunity, never let any experience pass one by in a business sense and only stop when you have developed the edge/ product or skill that no one else has. 



For as long as I can remember I have spoken negatively about the use of volume data as input for trading. I never stated that volume was worthless, only that I could not discover any value added by using volume. I would now like to say mea culpa and illuminate the good and the bad.

In the past I had always researched volume data as something that should be multiplied against either price levels or changes. The logic is that a price move when no one placed bets isn't much of a price move, and price moves with lots of money being wagered are more significant. Several of the notables on this list have published books and my recollection is that they counsel in favor of weighting price action by volume. And although I could use volume data to generate profits, no prior efforts were as profitable as using price data without volume.

That is, trading markets according to price momentum can be profitable, but my efforts at weighting that momentum by volume brought down the profitability. Similar with price volatility; weighting it by volume only dropped my expected trading profitability. Hence my bias against volume.

So why would I keep looking? Well, philosophically speaking, volume data is important simply because it isn't price. That is, it offers a respite from possible multicolinearity of looking at price data over and over. However the idea that volume alone could be interesting just does not seem logical. There is a lot of mythology associated with the markets. For example the oft stated reason for the market going up "more buyers than sellers" is impossible, as there are never more buyers than sellers. I had come to believe that the recommended use of volume was such a myth.

A little while ago a fellow Speclist poster (for whom I have considerable respect) commented about certain data (not volume) that gave me the idea to see if volume could be used without weighting it against price. That is, could it be used alone, or relatively alone? "Relatively alone" means comparing volume on up days to volume on down days, or volume on up ticks to that on down ticks, but still not being weighted by price activity other than the sign (+/-). Despite years of research, I had not done this before.

When you look at volume data it just looks like static, whether it is just total volume or distinguished by sign. You have to smooth it. For most people that means moving averages. First tip: don't waste your time with them. They are great at smoothing data when that data has a periodicity that is known and predictable, which does not apply to volume. So instead of using moving averages, go a notch or two higher: exponentials or better yet, regression trends. And once you have that, look at the slopes.

Second tip: looking at equities, your lookback periods should be relatively long, by which I mean 6 months or longer. If you have a method of dynamically determining the best lookback period (without using look-ahead bias) then use it, as its profitability will exceed most others.

 Third tip: it appears best used as a discrimination tool (buy the stock vs. sell the stock) rather than as a ranking tool. However we have not yet exhausted all the possibilities of the latter. Timing decisions are less critical than using price momentum, probably because there are many followers of price momentum and the exits and entrances get too crowded at certain times.

Results: trading constituent stocks of the Russell 3000 on the basis of volume over the last 15 years has produced profitability comparable to that of using price momentum. Similar rates of return with better (i.e. less) drawdowns. That's about 8,000 stocks when you include the dead ones to eliminate survivor bias. Results are better with items that go into portfolios. That means individual stocks rather than indices. Unlevered ETFs of equities = yes; levered ETFs = no; ETFs of currencies and volatility = no. Volatility of the asset negatively impacts success, just as with momentum. We have not tried futures (as we do not trade them).

We are now testing the incorporation of volume data into our overall decision process. Will advise how it turns out.

Lesson learned: when something is making a move, the signals tend to be writ large across the landscape. Accordingly it is no surprise that volume data works. The problem was in my preliminary bias against it and the thought (and published material) that it had to be weighted against price. Resolved: just do the research and forget the preliminary bias.

Third tip: it appears best used as a discrimination tool (buy the stock vs. sell the stock) rather than as a ranking tool. However we have not yet exhausted all the possibilities of the latter. Timing decisions are less critical than using price momentum, probably because there are many followers of price momentum and the exits and entrances get too crowded at certain times.

Results: trading constituent stocks of the Russell 3000 on the basis of volume over the last 15 years has produced profitability comparable to that of using price momentum. Similar rates of return with better (i.e. less) drawdowns. That's about 8,000 stocks when you include the dead ones to eliminate survivor bias. Results are better with items that go into portfolios. That means individual stocks rather than indices. Unlevered ETFs of equities = yes; levered ETFs = no; ETFs of currencies and volatility = no. Volatility of the asset negatively impacts success, just as with momentum. We have not tried futures (as we do not trade them).

We are now testing the incorporation of volume data into our overall decision process. Will advise how it turns out.

Lesson learned: when something is making a move, the signals tend to be writ large across the landscape. Accordingly it is no surprise that volume data works. The problem was in my preliminary bias against it and the thought (and published material) that it had to be weighted against price. Resolved: just do the research and forget the preliminary bias.



In the old days when I used to trade ag futures, about once a year I would see total unanimity of signals. Grains, meats, sugar, cocoa, etc. would all give the same signal at the same time. For example, they would all give a buy signal, such that I would think to myself, "holy mackerel, these markets are going to explode". But the total unanimity was a fake as the markets would stutter and then all drop. I was never bright enough to conjure up a reason why it happened, but it did.

Recently all of my financial market indicators gave sell signals. Stocks, Bonds, REITs, Gold. And the sell signals were everywhere. Momentum, behavioral economics, actual volatility, actual volatility of implied volatility, and some bizarre stuff you would never think of. You name it and it was bearish. Even the fundamental stuff I watch like surrogates for employment and retail sales are bearish.

We are very mechanized traders, and when I get a bearish signal for equities, I simply look to my overall rankings and see what to switch to. But everything was bearish, so there was really nowhere to relocate. However in looking at the rankings, equities were ranked higher than the competitors (e.g. bonds). So I had no choice but to stay in equities, being very selective and keeping every stock on a very short leash.
I have no idea why unanimity of indicators would negate the indication.

Any ideas? I don't see any flexion hands in this, but maybe others do.

One of the holy grails out there is to know how to forecast future co-movements between different assets. (As if forecasting just one isn't hard enough.) As it all starts to hit the fan, the correlations between all assets approach 1.0 at something much greater than an exponential rate…

My qualitative take on it is that the growth rate of the cross correlations as they inexorably accelerate towards parity approaches a certain velocity 'x' at which point, mathematically, we are as close to the asymptote as the 'system' can stand.

This is the 'going to the cliff and back again' phenomena that The Palindrome speaks of as a result of 'reflexive' interactions of market participants' expectations with the price and the price's effect upon the market participants' expectations. Arguably this is the ideal time for stabilising 'flexionic' behaviour (as opposed to shenanigans in TY around auctions et al.)

How they might do it, and more importantly time it, is a very deep question. For 'them' to have it figured out I think they would have to have figured out the actual underlying price generating process (what really moves prices).

Now, I guess only Renaissance Technologies' Medallion Fund has gotten anywhere near identifying the answers to that series of non linear questions. The most that one can say at this stage of the game is that the occurrence of substantial downwards co movements of assets tends to cluster (which is a 'warning sign' in itself) and for short periods after this clustering risk assets often make substantial minima. 

Steve Ellison writes: 

My first guess to Mr. Rafter's question is that, like a Higgs boson, unanimity in any market is very volatile, unstable, and unsustainable. As Richard Band wrote in a book about contrarian investing (doesn't everybody profess to be contrarian?), "If everybody is bullish, who is left to buy?"

To test this proposition, my first idea was to find instances in which the Investors' Intelligence survey of advisors had a 4-to-1 preponderance of bulls over bears or vice versa. There have been no such instances in the 2 years I have subscribed. I settled for instances in which either the bullish or bearish percentage was below 20%. There is typically a sizable group of fence-sitters predicting "correction", so the sum of the bullish and bearish advisors is much lower than 100%.

There were 10 recent weekly reports in which the percentage of bearish advisors was less than 20%. I get the reports on Wednesdays, so I tabulated the change in the S&P 500 futures from the Wednesday close to the Wednesday close of the following week.

Report             One week         Net
Date     Close     later    Close   change
3/13/2013 1550.00  3/20/2013 1549.00   -1.00
3/20/2013 1549.00  3/27/2013 1556.75    7.75
3/27/2013 1556.75   4/3/2013 1548.50   -8.25
 4/3/2013 1548.50  4/10/2013 1582.75   34.25
4/24/2013 1574.00   5/1/2013 1577.25    3.25
 5/1/2013 1577.25   5/8/2013 1628.75   51.50
 5/8/2013 1628.75  5/15/2013 1654.25   25.50
5/15/2013 1654.25  5/22/2013 1655.50    1.25
5/22/2013 1655.50  5/29/2013 1647.00   -8.50
5/29/2013 1647.00   6/5/2013 1608.00  -39.00

Average                                 6.68
Standard deviation                 25.31

Considering that the average net change during my subscription has been a gain of 3 points per week, I get a t score of 0.46, which is not only insignificant, but has the opposite sign of what my conjecture implied, i.e., that low bearishness is bearish.



 I first saw the 'dead eyes' look of a poker player/loser when I was 13 or so. Still gives me restless nights and I know I cannot become that way.

My dad took me into the "stockman's bar" in Billings, Montana to impress upon me what degenerate, greedy people turn into.

Probably another sleepless tonight tormented by that devil.

Gary Rogan asks: 

What is the real difference between gambling and speculation (if you take drinking out of the equation)? Is it having a theory about the odds being better than even and avoiding ruin along the way?

Tim Melvin writes: 

I will leave the math side of that answer to those better qualified than I, but one real variable is the lifestyle and people with whom one associates. A speculator can choose his associates. If you have ever been a guest of the Chair you know he surrounds himself with intelligent cultured people from whom he can learn and whom he can teach. There is good music, old books, chess and fresh fruit. The same holds true for many specs I have been fortunate to know.

Contrast that to the casinos and racetracks where your companions out of necessity are drunks, desperates, pimps, thieves, shylocks, charlatans and tourists from the suburbs. Even if you found a way to beat the big, the world of a professional gambler just is not a pleasant place.

Gibbons Burke writes: 

 Here is something I posted here before on this distinction…

Being called a gambler shouldn't bother a speculator one iota. He is not a gambler; being so called merely establishes the ignorance of the caller. A gambler is one who willingly places his capital at risk in a game where the odds are ineluctably, mathematically or mechanically, set against the player by his counter-party, known as the 'house'. The house sets the odds to its own advantage, and, if, by some wrinkle of skill or fate the gambler wins consistently, the house will summarily eject him from the game as a cheat.

The payoff for gamblers is not necessarily the win, because they inevitably lose, but the play - the rush of the occasional win, the diversion, the community of like minded others. For some, it is a desire to dispose of money in a socially acceptable way without incurring the obligations and responsibilities incurred by giving the money away to others. For some, having some "skin in the game" increases their enjoyment of the event. Sadly, for many, the variable reward on a variable schedule is a form of operant conditioning which reinforces a compulsive addiction to the game.

That said, there are many 'gamblers' who are really speculators, because they participate in games where they develop real edges based on skill, or inside knowledge, and they are not booted for winning. I would include in this number blackjack counters who get away with it, or poker games, where the pot is returned to the players in full, minus a fee to the house for its hospitality*.

Speculators risk their capital in bets with other speculators in a marketplace. The odds are not foreordained by formula or design—for the most part the speculator is in full control of his own destiny, and takes full responsibility for the inevitable losses and misfortunes which he may incur. Speculators pay a 'vig' to the market; real work always involves friction. Someone must pay the light bill. However the market, unlike the casino, does not, often, kick him out of the game for winning, though others may attempt to adapt to or adopt his winning strategies, and the game may change over time requiring the speculator to suss out new rules and regimes.

That said, there are many who are engaged in the pursuit of speculative profits who, by their own lack of skill are really gambling; they are knowingly trading without an identifiable edge. Like gamblers, their utility function is not necessarily to based on growth of their capital. They willingly lose their capital for many reasons, among them: they enjoy the diversion of trading, or the society of other traders, or perhaps they have a psychological need to get rid of lucre obtained by disreputable means.

Reduced to the bare elements: Gamblers are willing losers who occasionally win; speculators are willing winners who occasionally lose.

There is no shame in being called a gambler, either, unless one has succumbed to the play as a compulsion which becomes a destructive vice. Gambling serves a worthwhile function in society: it provides an efficient means to separate valuable capital from those who have no desire to steward it into the hands of those who do, and it often provides the player excellent entertainment and fun in exchange. It's a fair and voluntary trade.

Kim Zussman writes:

One gambles that Ralph and/or Rocky will comment.

Leo Jia adds: 

From the perspective of entering trades, I wonder if one should think in this way:

speculators are willing losers who often win; gamblers are willing winners who often lose.

David Hillman adds: 

It is rare to find a successful drug lord who is also a junkie. 

Craig Mee writes: 

One possible definition might be "a gambler chases fast fixed returns based on luck, while a speculator has time on his side to let the market decide how much his edge is worth."

Bill Rafter comments: 

Perhaps the true Speculator — one who is on the front lines day after day — knows that to win big for his backers, he HAS to gamble. His only advantage is that he can choose when to play. 

 Anton Johnson writes: 

A speculator strives to be professional, honorable, intellectual, serious, analytical, calm, selective and focused.

Whereas the gambler is corrupt, distracted, moody, impulsive, excitable, desperate and superstitious.

Jeff Watson writes: 

I know quite a few gamblers who took their losses like men, gambled in a controlled (but net losing manner), paid their gambling debts before anything else, were first rate sports, family guys, and all around good characters. They just had a monkey on their back. One cannot paint with a broad brush because I have run into some sleazy speculators who make the degenerates that frequent the Jai-Alai Frontons, Dog Tracks, OTB's, etc look like choir boys. 

anonymous writes: 

Guys — this is serious, not platitudinous, and I can say it from having suffered the tragic outcomes of compulsive gambling of another — the difference between gambling and speculating is not the game, the company kept, the location, the desperation or the amounts. The only difference is that a gambler, when asked of his criterion, when asked why he is doing this, will respond with "To make money."

That's how a compulsive gambler responds.

Proper money management, at its foundation, requires the question of criteria be answered appropriately, and in doing so, a plan, a road map to achieving that criteria can be approached.

Anton Johnson writes: 

It's not the market that defines whether a participant is a Gambler or a Speculator, it's his behavior.

Gibbons Burke writes: 

That's the essence of my distinction:

"gamblers are willing losers who occasionally win"

That is, gamblers risk their capital on propositions where the odds are either:

- unknown to them
- cannot be known

- which actual experience has shown to have negative expectation
- or which they know with mathematical precision to be negative

They are rewarded for doing so on a random schedule and a random reward size, which is a pattern of stimulus-response which behavioral scientists have established as one which induces the subject to engage in the behavior the longest without a reward, and creates superstitious as well as compulsive behavior patterns. Because they have traded reason for emotion, they tend not to follow reasonable and disciplined approach to sizing their bets, and often over bet, leading to ruin.

"speculators are willing winners who occasionally lose." That is, speculators risk their capital on propositions where the odds are:

- known to have positive expectation, from (in increasing order of significance) theory, empirical testing, or actual trading experience

They occasionally get unlucky, and have losing streaks, but these players incorporate that risk into the determination of the expectation. Because their approach is reason-based rather than driven by emotion, they usually have disciplined programs for sizing their bets to get the maximum geometric growth of their capital given the characteristics of the return stream, their tolerance for drawdown.

If a player has positive expected value on a bet, then it is not a gamble at all. The house does not gamble. It builds positive expectation into its games. It is a willing winner, although it occasionally loses.

There are positive aspects of gambling, which I have pointed out earlier in the thread and won't belabor. To say that "all gambling is bad" is to take the narrowest view. Gamblers who are willing losers (by my definition all are) provide the opportunities for willing winners (i.e., speculators) to relieve gamblers of the burden of capital they clearly have no desire to hold onto, or are willing to trade in a fair exchange for the excitement of the play, to enable their alcoholic habit, to pass the time, to relieve their boredom, to indulge delusions of grandeur at the hoped-for big win, after which they will quit playing, or combinations of all of the above.

Duncan Coker writes: 

I found Trading & Exchanges by Larry Harris a good book on this topic and he defines all the participants in the exchanges and both gambler and speculators have a role to play. Here is something taken from page 6 that make sense to me: "Gamblers trade to entertain". Speculators to "trade to profit from information they have about future prices."

He divides speculators into those that are well informed versus those that are not. One profits at the expense of the other. Investors "use the markets to move money from the present into the future". Borrowers do the opposite.



The Treasury Dept. puts out a Monthly Treasury Statement that breaks down a lot of interesting data. Mostly it's redundant information to anyone following the daily data, but this month (i.e. April) the payroll tax receipts from self-employed enterprises is a few sigmas to the upside. This April's self-employed number was 8.86 percent higher than that of 2012. The non-self-employed number was up 4.55 percent, which interestingly does not agree with the numbers reported in the Daily Treasury Statement. Recall that payroll taxes for everyone were increased 2 percent. Also note that the Daily Statement is done without human intervention, whereas the Monthly Statement has fingerprints all over it.



Those who choose not to read good books have no advantages over those who cannot read. (Attributed to Mark Twain.) A similar thing applies to research and data: Those who do not collect (and scrutinize) their own data, have no advantages over those who get their ideas and data from journalists or poor data suppliers. I would venture an educated guess that most of the managed investment money is handled by managers getting their information from journalists. Gentlemen, that's your competition. Go forth and prosper.

In quantitative analysis (irrespective of whether its data origins are financial statements or market prices) the guy with the best data has a definite advantage. Conversely, the best analytical mind coupled with poor quality data is at a disadvantage. Let me first deal with the problems of price data.

In equities, back-testing requires using deceased stocks to eliminate survivor bias. That means that to test the Russell 3000 over say 15 years, you need data on maybe 8,000 stocks. You cannot collect those by symbol, because symbols get recycled. So you try SEDOL or CUSIP numbers, but even those have problems. The holy of holies, CRSP has problems. And you cannot simply toss out the missing stocks without experiencing bias. Also note that the constituents of the R3k decrease monthly and are refreshed annually.

Obviously you have to adjust for dividends because you want to compare total returns. That introduces the dividend adjustment problem: do you use multiplicative adjustment or subtraction? Either one is problematic: destroying round numbers or creating negative numbers.

However the problems with data create tremendous opportunities to those who mine it. You know you are on to something when:

1. A major data provider has all of the dates of certain data off by 1 day. (systematic error) You call to ask why that is, and they don't have any idea what you are talking about. "How can you possibly know apriori that their data is wrong?" So you quickly reverse yourself and apologize for being mistaken. Everyone who uses that data has the error. They are counting things that are impossible.

2. You circumvent data suppliers and go directly to the exchange (or government website) because intermediaries screw it up. Hey, you cannot expect data replication to be perfect. (idiosyncratic error)

3. You disregard seasonally adjusted data in favor of raw data, and do your own seasonal adjustment. You cannot do this for every dataset, but certainly for the important ones.

4. A free provider (e.g. government or an exchange) provides detailed instructions on how to data mine their site. But the instructions are wrong. You call and the service people don't know what you are talking about. You eventually get to speak to the geeks and somehow learn the right way to get access. They confirm that no one had those problems before. WHY? Because no one else is looking at the data. He shoots; he scores!
These examples are like lifting back the bride's burqa, thinking that she might have a beard, and being surprised that she is absolutely beautiful.


a. When at all possible, go directly to the source. That may mean the exchanges or the government agency itself rather than your data supplier, and may appear unnecessary on the surface. But if you want to find the mistakes that most cannot find, you have to look in different places.

b. Look for site or download counters and check them out. Come back to them and recheck the numbers later to see the average daily hit rate. I was absolutely delighted to learn that I was one of only four downloaders of certain data.

c. Further check that data (with the counter) to see if it is available on Bloomberg or another major source.

d. Look for alternatives to the data you seek. The alternatives might not be the exact data, but they may be good surrogates. Real numbers for something close to what you want are better than bullsh*t numbers from a poorly conducted survey.

e. I cannot overemphasize the importance of checking the data, and checking that your data mining routine has collected it properly. Errors (either systematic or idiosyncratic) regularly occur. As renowned data cruncher John Tukey said, "There is no substitute for looking at the data." (Exploratory Data Analysis)

Typical problems you have to avoid:

- Look-ahead bias and survivor bias

- Lack of statistical significance - engineers typically require 30-50 observations, but market traders (such as technical analysts) frequently consider one event as significant. Don't do that!

- Testing on a sample of data that may not include the pattern. The solution to that of course is to always use the population, rather than a sample.

- Frequently you may test something on an index as a precursor to testing on thousands of individual stocks (or worse, options). But indices do not necessarily behave like individual stocks. ETFs might be a solution, but they are in themselves just smaller indices.

Those who challenge the validity of data mining (and also market timing) tend to cite as their proof that the first order daily changes in stock prices are random. We can concede that point, but there are lots more relationships to be studied than daily changes.

Data mining can be successful for any number of reasons but the juiciest fruit is to be found in the following ways:

- Analysis of data that is unknown or unseen by most people or, better yet, subject to systematic error (~finding buried treasure)

- Better analysis of existing data. (~having a better brain) Note that some of this will be serendipitous. Exploration by definition will lead you to discovering things you did not expect.

- Incredible persistence (hard work).

Should you seek to do fundamental analysis you will find different and more exasperating problems. We know many of them first hand.

The first problem is that the data is not easily accessed. It tends to cost quite a lot of money, and much of it has systematic errors. We have not found a commercial data supplier that did not have systematic errors.

The cost can be prohibitive. The major high-end quote provider places limits on the amount of data one can retrieve in a given period. We also know that provider uses a lot of humans in the process and has a lot of errors. Looking further afield, the fundamental data vendors we found are three in number. One replied quickly with a quote of $30,000 for the back data and a 1-year subscription going forward. Another came back a month later and wanted $72,000 for the same, and the third never came back to us. When I informed the higher priced service that they were above their competitors, they asked if their "pricing committee" could know what we were quoted by their competitors. That does not tend to make one comfortable, as what kind of business does not know what their competitors charge? Particularly if they make the point of having a pricing committee.

There is an alternative to buying fundamental data – getting it yourself. In theory this should be straightforward: the S.E.C. has all of the relevant files online. But if you are looking to get data on say the R3k for 15 years, you will have to collect it from approximately a half-million 10K and 10Q files.

Uniformity is generally not the rule, and you need some uniformity when doing computer mining. For example, sometimes a 10Q will be labeled "Ten Q" which has to be planned for. Unless you have access to a lot of people from the sub-continent, you want to do this automatically, which will also enable you to avoid things like transposition errors committed by humans. But some things are easier for humans than for computers. For example, most data constituting a company's total assets are listed as "Total Assets". Sometimes that is misspelled, and sometimes the number appears with a double underline, and other times without. Usually the next line starts with "Liabilities", but not always. It's laughable, but not fun.

We cut our teeth on a subset of the universe, REITs. The 172 that we found interesting had approximately 8,000 10K and 10Q files. After a lot of work we managed to get data cleanly from all but about 50. We consider that a major success, but even that low failure rate means we will have to go through about 3,000 files manually for the entire universe of a half-million files.

The good news is that having unrestricted access to such data provides a lot of opportunities. We are making a leap of faith that the data and our analysis will improve our existing results. Of course there isn't a guarantee, but that's the way to bet.

Phil McDonnell writes: 

Thanks to Bill for his excellent survey of data collection techniques and especially the pitfalls. There is little to add to his survey except one thing. That is when there are retroactive changes to data. To handle that case one needs to time stamp your data as to the time received. This caution applies to both fundamental data as well as price data which can be 'adjusted' a day or two later.

The worst example if this was Enron. The Enron data which showed the fraud was only released several years after the bankruptcy.



The payroll tax numbers look bullish on the economy, but any conclusions have to be weighed against the quality of the data. The upcoming Jobs report will include data for the approximate monthly period ending January 12, 2013. That is, it includes data crossing over the year end. Normally that would not be a problem, but this most recent year-end includes tax law changes.

There was a significant amount of bonuses being paid out in December 2012 rather than in the first Quarter of 2013. That shows up in the data. Then of course the tax receipts would drop in 2013 to reflect the lack of bonuses paid in 2013. That also shows up in the data. But then you would see the receipts level off at a number higher than January 2012 because the Feds are taking about 2 percent more. That also shows up in the data. (see attached chart) So the tax receipts accurately reflect policies, the fact of which counters arguments suggesting that people and businesses do not pay attention to taxes.

If taxes reflect policies in this short-run, then their ultimate effects will also reflect those policies. That is, sooner or later the increased taxes will undoubtedly have bearish effects on the economy. Like, we didn't know that?

Is there a better metric of what is going on, considering that the changing rules have the payroll tax receipts jumping up and down? Yes there is, and it is the medicare taxes, officially known as "Hospital Insurance". I have previously commented on this space about medicare taxes, so forgive me for not repeating it. This report will be released at 2 PM EST on the 8th business day of February. The bad news is that it is only monthly data, but the good news is that it will reflect receipts through January 31, and as such should give us a clean look.



 I'm sitting in a Panama City Youth Hostel next to a sailing board with departure times for Columbia. There are twenty sailings in as many days but most are filled by an assortment of backpack travelers from ten countries speaking five languages. It's musical chairs around the board for a $400 berth on a boat that takes four days to Cartagena, Columbia with a stopover at the San Blas Island. Every fifteen minutes a seat fills or opens and the standbys frown or cheer.

I arrived a week ago to help an American ex-pat on a Philosopher's stone land quest, a list of some 600 properties of which he has bought and sold five in the past year at perhaps ten times his dollar cost and two months research and surveying each to get the titles. I accompanied him a few days ago to seven hectares on the Caribbean Coast that sold for $250,000 on the spot to a Mexican developer, and viewed others on Lake Gatun within and Tobacco Island at the inlet of the Canal.

The primary reason for arriving in Panama, however, was to hike the Darien gap, a 90 mile jungle locked strip that is the only break in the Pan American Highway from the Arctic Circle to Tierra del Fuego. This is my third attempt to hike and canoe this gap and I flew in on the promise of a Darien village chief to guide me through to Columbia, but yesterday he backed out saying he couldn't be paid enough to risk his family's lives with recent increased activity of the Columbia rebel FARC throughout the region.
So, I set about alternatives and earlier today sat in Manual Noriega's Paymaster House eating a hamburger in a converted restaurant in Santa Clara, Panama that was one of his power centers with troops wearing boa constrictors around their necks to intimidate the locals and guarding the nearby airport. In December 1989 the sky filled with U.S. paratroopers who landed at the airport, asked the locals for an English speaker who pointed a Major and company of 30 Marines to where I bunked the previous night at an American ex-pat's home. He led them to the Paymaster House that was captured by the U.S.

I returned to Panama City and even as I write a space opened on the December 20th sailing of the Mars De Gato sloop and I grabbed a seat.

Tomorrow there's a 30th anniversary racquetball clinic at the Fort Clayton Gym where in the 1980's I led the racquetball invasion of Central and South America with clinics throughout Latin America, that was also the first failed attempt at the Darien Gap.
After the clinic and sailing I'll alight in Columbia and work south to climb 14,000' Ecuador mountains, and then ply rivers down to Peru where I was hired hours ago via Internet as a Butterfly hunter. I'll capture only five exotic species that an amigo collector sells on EBay for $500 to $1200 each. He has provided me with a net, glassine envelopes and mothballs, and will pay $50 for each rare species that I hope to net to finance my passage home in a few months.



My transparent, stretchable Fibonacci overlay seems to be successfully identifying price levels around which the main indexes cluster. This in itself does not predict the future, it identifies where the holes are on the bagatelle table, but not which one the ball will settle in. Moreover it may reflect a self-fulfilling prophecy rather than a rule. Nonetheless, the information can be useful in constructing multiple-leg options positions.

But my overlay is not predicting timing. All the pundits mention Fibonacci but this does not seem to be the case, has anyone tried other methods? Interested in any pointers.

Looking forward to stepping back in the water, but want to maximise the acuity of my toolset first.

I found this nice online chart for streaming SP500, also gives longer term charts.

Jim Sogi writes:

Not sure what you're doing, but I've been pondering time and time frames and relationships of time. Some systems using returns have time exits and a study of time seems like its important. Not sure exactly how, but the idea is to maximize return based on time while minimizing loss. The relationships change by cycle. It seems time itself and speed and roc volatility all have cycles in time. Perhaps survivorship times give some info.

Bill Rafter writes:

The question is whether one wants to value time or eschew it. Both can be done, so it's up to the practitioner.

Valuing time is easy, as most economics is time series processing. And most all market data comes dated. Shunning time is trickier; do you want to avoid just some time, or all of it?

Point & Figure analysis is what most subscribe to if they want to eliminate some time, and they do that by defining "box sizes" or the minimum move they consider significant. The theory is to define the noise level and throw that noise away. Sounds great, such that someone would be willing to be a tad late on a move if the signal had a higher degree of accuracy. Our extensive research says that P&F is certainly a tad late, but there is a decrease in accuracy. Here's another caution: most of the literature on P&F is written by those lacking native intellectual capacity (IMO) who have no concept of research. To them P&F is a religion akin to animism.

A more successful approach than P&F is not to create box sizes, but to drop all "inside days". I say more successful in that you eliminate insignificant data, and do not lose accuracy. However we have not been able to increase accuracy over normal data analysis. But we are still working with it, and may find something. You still get a time grid, but with lots of the days missing.

The most effective way to eliminate all time is to use Lissajoux patterns. That link will give you an animated example of such with two sine waves. There are lots to be said about this, but I don't think many have the appetite for it



 The payroll tax receipts are virtually identical to that of a month earlier. If there is any tilt to the data, it is slightly higher (i.e. bullish equities). If the media expects it to be down (maybe because of Sandy), then the released data will catch them a little off guard.

From the end of the reporting period Nov. 16th thru Nov. 28th, the data has been higher, but that should be reflected in the next month's report. Please note that I remove the seasonal effects to avoid enthusiasm over the temporary Christmas employment.



A post purporting to show that buy and hold investing does not work has appeared on our list. It is reprehensible propaganda and total mumbo. They do not take account of the distribution of returns to investing over long periods that have been enumerated by the Dimson group and Fisher and Lorie. It is sad to see this on our site. The arguments against buy and hold seem to be that the professors found that short term investing didn't work so they erroneously concluded that long term investing must be the alternative. Shiller is mentioned and cited with approval.

Alston Mabry writes: 

To explore this issue numerically, I took the monthly data for SPY (1993-present) and compared some simple fixed systems. In each system the investor is getting $1000 per month to invest. If during that month, the SPY falls a set % below the highest price set during a specific lookback period (the 3, 6, 12, 18, 24 or 36 months previous to the current month), then the investor buys SPY with all his current cash (fractional shares allowed). If the SPY does not hit the target buy point this month, then the $1000 is added to cash. Once the investor buys SPY shares, he holds them until the present.

For example, let's say the drop % is 10%, and the lookback period is 12 months. In May of year X, we look at the high for SPY from May, year X-1, thru April, year X, and find that it is 70. We're looking for a 10% drop, so our target price would be 63. If we hit it, then spend all available cash to buy SPY @ 63. Otherwise we add $1000 to cash.

Each combination of % drop and lookback period is a separate fixed system.

Over the time period studied, if the investor just socks away the cash and never buys a share (and earns no interest), he winds up with $239,000. On the other hand, if he never keeps cash but instead buys as much SPY each month as he can for $1000, then he winds up with over $446,000, which amount I use as the buy-and-hold benchmark.

If the investor uses the fixed system described, he winds up with some other amount. The table of results shows how each combination of % drop and lookback period compared to the benchmark $446,000, expressed as a decimal, e.g., 0.78 would that particular combination produced (0.78 * 446000 ) dollars.

Results in this table

The best system was { 57% drop, 18+ month lookback }, or just to wait from 1993 until March 2009 to buy in. Of course, it's hard to know that 57% ex ante. The next best system was { 7% drop, 3 month lookback } coming in at 0.99.

This study is just food for thought. It leaves out options for investing cash while not in the market. And it sticks with fixed %'s without exploring using standard deviation of realized volatility as a measure. So, there are other ways to play with it.

Charles Pennington comments: 

Thank you — that is a remarkable "nail-in-the-coffin" result.

Nothing beat buy-and-hold except for the ones with the freakish 57% threshold, and it won by a tiny margin, and it must have been dominated by a few rare events–57% declines–and therefore must have a lot of statistical uncertainty..

That's very surprising and very convincing.

(Now some wise-guy is going to ask what happens if you wait until the market is UP x% over the past N months rather than down!)

Kim Zussman writes: 

Here are the mean monthly returns of SPY (93-present) for all months, months after last month was down, and months after last month was up (compared to mean of zero):

 One-Sample T: ALL mo, aft DN mo, aft UP mo

Test of mu = 0 vs not = 0

Variable      N      Mean     StDev   SE Mean  95% CI            T
ALL mo     237  0.0073  0.0437  0.0028  ( 0.0017, 0.0129)  2.58
aft DN mo   90   0.0050  0.0515  0.0054  (-0.0057, 0.0158)  0.92
aft UP mo  146  0.0083  0.0380  0.0031  ( 0.0021, 0.0145)  2.65

 The means of all months and months after up months were significantly different from zero; months after down months were not.

Comparing months after down vs months after up, the difference is N.S.:

Two-sample T for aft DN mo vs aft UP mo

                  N    Mean   StDev  SE Mean
aft DN mo   90  0.0050  0.0515   0.0054   T=-0.53
aft UP mo  146  0.0084  0.0381   0.0032

Bill Rafter writes: 

A few years ago I published a short piece illustrating research on Buy & Hold. It contrasted a perfect knowledge B&H with a variation using less-than-perfect knowledge using more frequent turnover. Here's the method, which can easily be replicated:

Pick a period (say a year) and give yourself perfect look-ahead bias, akin to having the newspaper one year in the future. Identify those stocks (say 100) that perform best over that period, and simulate buying them. Over that year you cannot do better. That's your benchmark.

Then over that same period do the following: Buy those same 100 stocks, but sell them half-way thru the period. Replace them at the 6-month mark with the 100 stocks perfectly forecast over the next 12 months. Again sell them after holding them for just half the period. Thus the return from the stocks that you have owned and rotated are the result of less-than-perfect knowledge. Compare that return to the benchmark.

Do this every day to eliminate start-date bias, and then average all returns. The less-than-perfect knowledge results far exceeded the perfect-knowledge B&H. Actually they blew them away in every time frame. It's really obvious when you do this with monthly and quarterly periods as you have so many of them.

The funny thing about this is the barrage of hate mail that I received from dedicated B&H investment advisors, who somehow felt their future livelihoods were threatened.

If anyone wants that old article, send me a message off the list. We called it "Cassandra" after someone with perfect knowledge that was scorned.

Anton Johnson writes in: 

Here is a link to BR's excellent study "Cassandra", as it lives on in cyberspace.



 One has found that there is an electronics circuit that almost always retrospectively provides a great description of price action in markets. I wonder if there is an electronics circuit that compresses the voltage output keeping it in a range, sort of like the finger in the dike, but then after the compression is over on the negative side, e.g after the negative feedback is taken away, the voltage doesn't immediately lead to tremendous negative voltage. I seem to remember such a circuit with op amps.

Jon Longtin writes:

There are a variety of electronic circuits that perform such a role, depending on the application. One common application is a voltage regulator, which provides a (nearly) constant voltage, regardless of the load applied to it. The circuit monitors the actual voltage currently being provided and compares to a pre-set reference value. The difference between the actual and desired (setpoint) values is called the error, and is used to adjust the current provided to the circuit to bring the voltage back to the setpoint value. For example if the load increases (more electricity demand) the load voltage will drop and the voltage regulator will provide more current to bring the voltage back up. Same goes for a decrease in load.

There are some limitations and compromises in such a circuit. First is there is a finite amount of current that the power supply/voltage regulator can be provided, and if the error signal requests more than this amount, the output will not be maintained. Also of importance is the time response: a circuit with a very fast time response will respond more quickly to fluctuations in the load, but can also result in so-called parasitic oscillations, in which the output oscillates after a fast change in load is made. By contrast a longer time response provides a slower response to a variation, but tends to damp oscillations. This same behavior, of course, is seen in countless financial indicators, and is part of the art in deciding, e.g., how many prior data points to include in a signal.

A somewhat more complex version of the above, and perhaps more closely aligned with the behavior of a market signal, is an audio "compressor/limiter". This is a device that constantly monitors the volume (magnitude or voltage) of an audio signal and makes adjustments as needed. A limiter is the simpler of the two and simply sets a threshold above which a loud signal will be attenuated. The attenuation is not (usually) a brick-wall however; rather a signal that exceeds the threshold value is gently attenuated to preserve fidelity without overloading the audio or amplifier circuitry. A compressor is a more complicated animal and provides both attenuation for loud signals AND amplification for quieter ones. In essence a HI/LO range or window is established on the unit, and signals exceeding the HI limit are attenuated, while signals below the LO limit are amplified. This resulting output then (generally) falls within the HI/LO range. This is used extensively (too much!) in commercial music. Humans naturally pay attention to louder sounds (ever notice how the volume universally jumps when commercials come on TV? They are trying to grab your attention with the louder volume). Pop music attempts to achieve the same by using aggressive compression to provide the loudest average volume for program material without exceeding the maximum values set by broadcast stations or audio equipment. The result, however is that the music sounds "squished" and doesn't "breath" because the dynamic range of the content has been reduced considerably. With such devices there are a variety of adjustments to determine the thresholds, time before taking action (the attack time) and how gradually or strongly to attenuate (amplify) signals that exceed the envelop range.

Here' s a fairly decent article that describes this in more detail.

Incidentally both of the above are examples of a large branch of engineering called Controls Engineering. The idea, as Vic stated, is to monitor the output by using feedback and make adjustments accordingly. There are countless different algorithms and approaches, as well as very sophisticated mathematical models (people build careers on this) to best do the job. Like most complex things, there is no single approach that works best for every problem, but rather involves a balance of performance, cost, and reliability.

I highly suspect such algorithms have already found their way into many trading strategies, one way or another.

If interested, I can suggest some references for further reading (though I am not a Controls person myself).

Bill Rafter writes: 

 Think of your voltage regulator as a mean-reversion device. If a lot of this is being done, then your trading strategy must morph into simply following the mean.

In light of recent changes in the investment climate we suggest that one should tighten up controls in which one is long a given market. Perhaps that might also or alternatively mean (a la Ralph) tightening the size of the positions. The result will be taking less risk and incurring less return, but taking additional risk would seemingly not be rewarded in the current milieu.

Jim Sogi writes: 

Dr. Longtin's description of compressors and limiters was
fascinating.  A compressor on my guitar signal chain prolongs the
sustain on a signal in addition to smoothing out the volume spikes and has less fade as the signal weakens.  With added volume, one gets a
nice controlled feedback.

Sometimes in the markets one sees a sustained range with the spikes being attenuated reminiscent of a nice guitar sustain.

On a different note, one curious thing is that people cannot  discern differences in absolute volume.  It's very hard to hear the differences
in volume between two signals unless they are placed side by side.



 The problem with polling is because of the response rate. A generation or two ago people were honored that someone would solicit their opinion. No so today, for whatever reason. Two days ago the Pew Foundation revealed the percentage of those persons contacted who are willing to give their opinions. Take a guess as to what that percentage is: According to Pew, their "success rate" is nine percent, or one out of every eleven persons they speak with. Thus they cannot get a reasonable random sample and the Central Limit Theorem does not apply. Therefore most polls are GIGO.



Somebody is spending, but who? This chart shows YOY customs duties and excise taxes which are a fairly good surrogate for retail spending.

However, here's a question: if the federal government buys a fleet of new Chevy Volts (the ones that burn up, LOL), do they pay excise taxes?

Mr. Krisrock writes: 

The combination of Higher Energy, Higher Dollar, Higher Commodities, and the September 15 California Tax on the internet has caused a SURGE in amazon orders to beat the increase…and of course Obama's playing with the numbers. Look at the huge drop in employment a year ago in October. Obama knows how the numbers work a year ahead, and of course, where do auto parts come from…electronic components come from Asia…for starters…and foreign car assemblies do buy foreign parts.



 What are the best markets to trade? Many futures markets trade differently. Some have a lot of depth and intraday gaps are infrequent (I consider these the best to trade). Others have ample liquidity but are prone to gapping. Others still are downright scary. E-minis and 10 years seem like very "safe" trading markets. Eurodollars as well. Crude oil has a lot of liquidity but can gap. Gold seems prone to fast and erratic moves. Grains seem like they can get a bit dicey. Less trafficked softs seem rather risky. Commodities in general appear to have more erratic price risk than stock index futures or financial futures. FX is fairly liquid and seems ok. I am largely making observations based on personal experience and in some case I have none so I am curious for thoughts from seasoned specs.

Bill Rafter writes:

Ask yourself, would I rather trade an extremely efficient market in which information was digested immediately and most of the fluctuations not related to new information were due to randomness, or would I prefer a market that was less so. As you gain experience you will learn that one of these mutually exclusive choices is more profitable to trade than the other. One of these requires virtually no expertise to trade, and indeed expertise would not appear to be helpful, whereas the other requires considerable expertise. One is the frequent choice of novices, whereas the other tends to be avoided by novices. Then ask yourself, how do novices typically fare?

Jeff Watson writes: 

Grains are impossible right now. The 30 cent daily ranges make it too much of a gamble. Even trying to predict, or have a gut instinct of where the carry spreads, the corn/wheat/bean spread, the crush, are going….Oy Vey. To play the grains, to coin a surfing analogy: You better be in really good shape, you gotta see the wave (move) coming toward you, then paddle real hard, pop up and catch the wave. You better either be quick to bail or commit to the wave, make a bottom turn, then ride it until it's over. Determining when the ride(trade) is over isn't as simple as it sounds, and many dangers exist on and below the surface that can still mess you up when you bail the trade. The most important decision a grain trader can make right now is whether he wants to gamble a lot for a potentially big reward, or hunker down and reduce risk.




 We have recently learned something with regard to trading currencies; specifically that in a strategy involving switching or rotating currencies they should also be traded with debt and gold. That is, excluding gold and debt from the universe of currencies lowers rates of return and/or increases drawdowns to less than optimal.

Background: We are equity traders who occasionally run from equities when our various quant manipulations suggest we are about to get thumped. Traditionally in such a circumstance our go-to place has been treasuries, specifically the 10-year. But there are times (like now) when fleeing to bonds doesn't seem like a good idea. So we decided to reassess our strategy vis-à-vis alternatives. And our full-court-press of research shows that the best alternative is a strategy of moving between bonds, gold and the U.S. Dollar Index. This beats ALL strategies involving only one or two of those assets. More importantly for others is that it also beats ALL similar rotations among the Dollar Index and a collection of other currencies. (N.B. we are free to choose whatever time frame seems to be best suited.)

With regard to our strategy, it trades combinations of those assets rather than one alone. However there are times when the strategy will have us in only one asset, and many would express fear at being entirely in gold or the dollar. Few would fear being entirely in treasuries, although the period of greatest decline was indeed a time when all monies were employed in debt.
No one is going to get excited about the alternative rotation strategy; it does not have an exceptional rate of return. But it does have very good risk control, which is what we want in an alternative. None of this should be surprising, as we know how interrelated they are. Currency is all debt, except gold which is the traditional debt alternative vis-à-vis inflation. But then one of the costs of holding gold is the foregone interest. Since they cannot be separated fundamentally, it is logical that they not be separated in a trading program. But it took testing to convince us of such.

If you are a trader who exclusively trades currencies, you should experiment with expanding your universe to including gold and debt.



 I have found a worthy complement to the O'Brien series: Dumas' The Last Cavalier

In 1997 someone had gone through an old French newspaper and found a serialization of Dumas's last work, which had never been published as a book because it was unfinished. Finally it was published in 2005. I happened to buy a copy then, but have only just gotten around to reading it.

I have found it absolutely fantastic. For those who like historical novels, it provides great coverage of the Napoleonic era, the period after the revolution. If you are (as many on the list) a fan of the Patrick O'Brien series about Jack Aubrey, you will find this book gives you the French side of many of the events. One of Aubrey's counterparts would have been Robert Surcouf, so-called King of the [French] Corsairs. Of course Aubrey was fictional and Surcouf real. Dumas' tales of Surcouf are just as good as O'Brien's tales of Aubrey. The protagonist in Cavalier is mentored by Surcouf. Additionally there is an excellent play-by-play account of the Battle of Trafalgar and Nelson's death.

Dumas has written so much, that there are bound to be repeated scenes. An obvious one is that in which an engaged couple signs the marriage contract. That scene in Count is repeated in Cavalier including where the groom disappears immediately before signing, although the respective grooms depart for different reasons. The Tuileries Palace discussions of Louis XVIII and his staff in Count are repeated with Napoleon and his staff in Cavalier, although chronologically Cavalier precedes Count by at least a decade.

Dumas seems to want to please everyone, and refrains from taking sides, which probably accounts for his publishing success, as both Republicans and Royalists could find something to cheer about. He also provides entertainment for his female audience - lots of social gatherings.

This is a long book. My copy was 700+ pages; the action spanning six years with additional prior history.

The first 300+ pages deal with politics and troubles of a police state, somewhat on edge because of an uprising in Brittany. The parallels to the current political scene are startling. Supporters of Napoleon attributed everything good to him, while the Royalists blamed everything bad on him. One hopes we do not undergo a similar war of extermination. Finally our protagonist gets his freedom and goes on one swashbuckling episode after another, much like the Musketeers.

Dumas meant for it to cover perhaps another eight years if you consider the 14 years from the signing of the marriage contract mentioned by the soothsayer. And I truly wished it had. It's one of those books you hate to have end.

Further editing to put it into a book by Dumas would have made it even better. Still, some of ways he conveys information are extremely well done. He spoke of Surcouf (Jack Aubrey's counterpart) as a man whose one good fairy not invited to his baptism was Patience. And he chastises young men for leaving their health in brothels and their purses in taverns. Another: "While Nature may have given him a lot of excellent qualities, it had refused him like qualities of the mind."

As the Chair has said, some of the best reading is over a hundred years old.



 As a hedge fund manager you have nine assistants employed solely to give you advice. Each of the assistants has a different perspective on the markets. They are all good advisers, as any one of them improves your trading immeasurably. For example, the market has a 2 percent annual return, but with your skills you can generate a 10 percent return. If you also add the advice of any one of your assistants you can bump that return up to between 12 and 18 percent.

Over the last 12 representative years there have been times when the nine were universally bullish. But despite their unanimity the market did not always rise. Conversely, even in the protracted down moves of 2008, their bearishness was not unanimous. Put another way, there was always one or two that wanted to go long at the worst times. Yet each and every one over time provided great advice.

You would like to find a way to combine their advice to get even better results than by using any one alone. But that's not easy. Sometimes, adviser A is early, and late at other times on a move. Likewise with the other assistants. One simple solution would be to have them vote, but the performance result of the vote underperforms some of the individuals, although still better than not having any adviser.

*Note here that we are only considering return and not the risk taken to achieve that return. Risk should always be considered, but for the sake of moving along, let us assume that taking the advice of your advisors never increases risk and that their respective upside contribution to profits is directly proportional to their downside exposure to risk. That is, much of their positive return contributions come from reducing risk, which is what we have observed generally.

Now, let's suppose that these advisers are not people, but algorithms. That's actually better because as algorithms they can be combined in ways that individuals cannot. They can be viewed logically (on/off) as in the voting experiment, or they can be ranked by their actual values. If they have scalar values they should be normalized (given the same order of magnitude or scale). For example, you cannot compare the slope of the Dow Industrials with that of the S&P 500, as the former is an order of magnitude larger. But if you put them on the same scale (e.g. divided by price), you can easily compare them.

Normalization is exactly what you would do to your inputs if you were using a neural net, and you might be tempted to go the NN route. But NNs have problems; among them would be your inability to discover the actual combination of what worked best. You might say "who cares" as long as it works, but that philosophy does not have a good history. However there is a very good use for a NN, and that is as a trial. That is, if you are good at NNs (and most people fail), then you should by all means try. If the NN gives you good results, then proceed on your own to find a good combination without the NN. But if using a NN does not improve results for the experienced practitioner, then it is going to be very difficult to find a better combination.

 But how do you combine them to your best advantage? Well, there's an app for that. It's called linear algebra. It is somewhat vertigo-inducing for most traders, because most of them are comfortable with things they can chart. For your average trader that means two dimensions; options traders tend to be comfortable in three dimensions. But with our illustration we are likely progressing to higher dimensions, and they are not chartable, although the problem's solution is indeed a chart, albeit a virtual one.

Subsequent "chapters" (if the topic flies): Operations, Testing.

Jim Sogi writes: 

"But with our illustration we are likely progressing to higher dimensions, and they are not chartable, > although the problem's solution is indeed a chart, albeit a virtual one."

One of my first posts ever to the SL was Flatland, and the idea that multiple dimensionality is lost in two dimension charts which are typically used.

Easan Katir writes: 

Flatland, one of my all-time favorite books since I read it 40 years ago, offers insights in many arenas. Perhaps some enterprising ex-game coder would turn his attention to finance and provide charts where the point of view can be changed with a click. Will traders of the future be trading on an X-box-like device?



 On a recent trip to the Allerton Botanical Garden in Kauai the guide noted that the mango trees have recently gone through 3 seasonal cycles in one year when only one is normal. He had no idea why. Our mango tree did the same. The explanation here was the big unseasonable rainstorm that knocked off the flowers at the beginning of the season. Now we have ripe fruit, unripe fruit and flowers on the same tree. Chair often compares trees to markets. I wonder if unseasonable shocks (EU, Fed,) might have thrown off the seasonal tendencies in the markets, shortening cycles, or forcing cycles. The changing cycles are the hardest to understand.

Bill Rafter writes: 

That is not unique to mangoes.

Take grape vines for example. Generally the only fertilizer you should use on grapevines is a shovel-full of manure in the spring. My wife thinks we should have flowers in between the grapevines and occasionally will hit the vines with some spray fertilizer she is using on those flowers after midsummer. Relatively soon thereafter the vines put out some new flowers (although most people would not recognize them as such), which will bear fruit (grapes are mostly self-fertile). But the fruit won't have time to develop fully and is a waste.

Fig trees typically produce two crops each year. The early crop (called breva) has generally unworthy qualities, whereas the late crop is to die for. But if you wanted, you could fertilize the tree weekly and have figs all summer long. People with summer vacation homes tend to do that as they know they will not be around after Labor Day, when most of the big crop would come in. You could even fertilize yourself to a second crop of determinate tomatoes, which are "programmed" to bear one crop all at once.

I don't know how good that is for the respective plant because I don't do it. And the lessons for the market seem to relate to Quantitative Easing. Perennial plants (particularly fruit trees) need a dormancy period. If they don't get it, they produce poorly until they do. I believe the same is true with markets: you cannot preempt or "outlaw" the business cycle and expect the economy to respond favorably all the time.



 To what extent does the concept of speed ratings, popularized by Crist , have applicability to markets. One variant of the idea being which horse had the fastest quarter last race, or which one had the greatest move down from 1 quarter to the next, like from first quarter to stretch, or stretch to close. Can this concept be applied to days within weeks, or months within years? How would some of the handicappers or horses extend this and what would Bacon say?

Bill Rafter writes:

While in grad school a buddy and I used to go to Golden Gate Fields regularly. For some reason it was always on a Thursday, and we went for the last two races to avoid the entrance fee. The lack of admission was critical because it removed the necessity to bet. (relation to low fixed costs?) The Racing Form was a critical part of the exercise, and I would bet on the horse with the highest speed rating that was showing greater than 8 to 1 odds close to post time. The bets were on the minimal side.

My results were successful if you measured my wine supply, which was quite good. The accumulation of wealth from horse racing was not something I relied upon, so any winnings were spent on the way home at the liquor store on the main drag. Racing bets were not something to aspire to, for a very good reason. Going to the payout window revealed the demographics of those who typically bet on longshots, as an 8-to-1 horse was considered. We also tended to meet those same people in the liquor store, where they were buying Thunderbird.



This is a chart illustrating the S&P shaded to reflect the yearly trend of Initial Unemployment Claims (Fed St. Louis series ICSA). While the chart does not prove anything, it does illustrate a possible relationship. Note that the data relating to the claims have been inverted, such that increases in claims implicate poorer economic conditions and in-turn declining equity prices.

Editorial comments: I do not prefer the ICSA data because it is weekly and goes through a process of human intervention (?corruption?). I prefer daily data that gets recorded electronically without any possible manipulation. HOWEVER even the ICSA data is now showing bearish market indications. I could torture this data to present the current situation as bullish (by introducing significant lag), but have tried to show it similar to how most would be receiving the information.



How do multiple lead changes, and their duration in minutes very close, i.e. from up to down from close or open, or within + or - 2 for multiple minutes, affect the outcome of the day in markets? In the playoff game they had a 14 point lead. "There were six lead changes and five more ties in the final 7 minutes of the third. For the next 13 minutes, a span of 46 dizzying possessions, neither team led by more than two points." By the way, the quotes are from the AP story about the game. One of the few times that I've ever seen a good meaty story rather than boiler plate from the AP.

Bill Rafter adds: 

Sounds like a job for the Spearman Rank Correlation.



 When I was a kid, my father got the family out of blue-collar South Philadelphia to blue-collar Wildwood, NJ for the entire summer. The beach towns of New Jersey are either nice or tacky, and Wildwood was extremely tacky, with most tackiness related to its boardwalk. When you are at the seashore for the summer, collecting shells wears a little thin, so a friend Buzzy and I got a discarded window screen and would go under the boardwalk just below several pizza shops and shovel sand into the window screen. Patrons reaching into their pockets for coins would regularly drop some through the slats in the boardwalk. A few hours work would produce about two bucks each for Buzzy and I, and that was in the days when a quarter could buy you a slice of pizza.

It was dirty work, but rewarding. And of course the dirt was easily washed off in the ocean. Invariably when one of the other kids would find out about our wealth their comments were, "You guys are sooo lucky!" Luck had nothing to do with it. There was a distribution of coins that would fall through and a lot of work by the harvesters. The same is true of the markets.

Jeff Watson writes:

Bill, your experience reminds me of that failed, but magnificent musical, "Paint Your Wagon" where Clint Eastwood discovers that the gold dust gets spilled on the floor and falls through the cracks. Eastwood, Lee Marvin et al proceed to dig tunnels under the entire town and they collect all of that spilled gold dust. They do extremely well for awhile, until a black swan moment where everything collapses and the entire town caves in. There are many market lessons in this movie. About 2:50 of this video is where Eastwood has his eureka moment. 

Vince Fulco writes: 

For those who have never been to Chair's Weston office, right next to the Captain's chair is a painting depicting a similar scenario. Not sure if it is a L'Amour story but the gold miner/spec is on the verge of hitting a nice vein while the precariousness of the surroundings become increasingly more apparent. The moment on the razor's edge is caught perfectly.

Just a beautiful piece.



 I recently visited a Dr. and when I got there, the nurse asked me to fill out a computer questionnaire that took 1 hour to fill out. After I filled it out, I was asked to sign a statement that said such things as "you will not be paid for filling out this questionnaire, the contents might be used by commercial factors, there are unlimited people in the survey" and a hundred other things that gave it a false aura of legitimacy.

I am wondering to what extent the false aura of legitimacy pervades our field. The classic example is the elections in a marxist or democratic regime, or the government institution that's there ostensibly to protect you from harming yourself but is really a gate for preventing competition from small and new entrants into the field. The committees in the markets to maintain order and proper pricing that are really arenas for the members to mark the positions in their favor, and force out the non-members through margin changes and rule changes comes to mind. The rules against competition in all fields, the licensing requirements, and for example the ethics tests that one must pass in certain fields. How pervasive is this and what is the relevance to our field?

Sam Marx writes: 

I agree that the urge not to compete in a fair open market if one is able to set up a monopoly or obtain an advantage is there, and it's a part of human nature. I believe that it cannot be eliminated entirely but there are some changes that would help. I also believe that lying and cheating obtained a large impetus and some begrudging approval when the graduated income tax became constitutional. Therefore, a recommendation I would make is to do away with the graduated income tax and have a flat income tax or replace the income tax with a sales tax. I don't expect to see any of this in my lifetime however. 

Bill Rafter writes: 

Sham credentials. There exist a variety of market-oriented groups whose stated purpose is to identify the truly worthy. However all they really do is confer the aura of legitimacy on those in need of same, while providing income for the executives at group headquarters and hoodwinking the public. The group is frequently a "non-profit", adding more prestige. The legitimacy is conferred by letting the novice fork over not-insubstantial funds, taking a few tests and eventually getting the rights to put letters after his or her name, provided he stays a dues-paying member of the group. The orientation of the group can be fundamental, technical, quantitative, retirement planning or risk aversion.

My personal observation is that some market-oriented groups are worthy, and those which do not offer the paid initials are the best.



 A bittersweet moment in Ty Cobb's life reportedly came in the late 1940s when he and sportswriter Grantland Rice were returning from the Masters golf tournament. Stopping at a Greenville, South Carolina liquor store, Cobb noticed that the man behind the counter was "Shoeless" Joe Jackson, who had been banned from baseball almost 30 years earlier following the Black Sox Scandal. But Jackson did not appear to recognize him, and finally Cobb asked, "Don't you know me, Joe?" "Sure I know you, Ty," replied Jackson, "but I wasn't sure you wanted to know me. A lot of them don't."

Stefan Jovanovich adds: 

Given the fact the Jackson remained a respected figure of the community and the liquor store was owned by Jackson and his wife and his name was above the door, the story could be one of Grantland Rice's maudlin inventions. For the people of his home town, Greeenville, SC, Jackson always was a figure of respect.

The site shoelessjoejackson.org has a link to the PDF of Furman Bisher's interview with Jackson — the only one he ever gave. Eliot Asinof's book (the one John Sayles relied on for Eight Men Out) is a very large pile of crap which completely ignores Bisher's interview and the Jackson's own grand jury testimony. If Jackson had, in fact, been guilty, it is hardly likely he would have prevailed on the civil suit against Comiskey for his pay for the 1920 and 1921 seasons.

Apologies to all — this subject always gets my dander up. During the series Jackson had 12 hits (a Series record) and a .375 batting average—the best record for a player on either team. He had no errors and threw out a runner at the plate. The principal "proof" against him was that the Reds had hit a number of triples to left field (where Jackson played) because Jackson deliberately dogged it in running the balls down. None of the contemporary newspaper accounts mention ANY triples being hit to left field by the Reds. Once again, the lies run round the world while the truth is still putting its boots on.

Thanks, Bill, for bringing up one the 10 greatest ball players of all time.



 This past week there was an item published about the drop in miles driven by the population. The point of the missive was to suggest that miles driven is a surrogate for economic activity, and that we should prepare for another recession because the miles driven showed a huge decline.

Part of the problem in analyzing the data is that mileage did not directly address the price of gasoline, and it is logical to assume that at a high price of gasoline, the public curtails some discretionary driving. So some of the drop in miles driven could be related to price. Another item could be the Cash for Clunkers program, which could offset the price rise to some extent as the newer cars generally get better mileage. The latter apparently did not happen to any large degree.

The chart presented with this link shows the allocation of household expenditures spent on gasoline (top panel) and the annual growth rate of those expenditures.

My conclusion: There is a drop in mileage driven, but I would not bet that it foretells further recession.

Mr. Krisrock comments: 

105 dollars in crude tonight as Obama and his communist central planners try to figure out the new narrative and who to blame this on…



 Do markets learn from each other? For example, is the S&P market this year, following a similar path to bonds last year, with every trepidatious move down being requited with a rise? Are such "learnings" graduated to the point of regularities. And is it a domino effect or a path of least resistance or consilience or convergent evolution or what have you? What do you think? Can it be quantified? Should it be quantified?

Gary Phillips writes: 

Price discovery has become as anachronistic a term as capital formation. The Fed is supposed to be listening to the market to give it guidance in it's policy decisions, not dictating to the market, what the market should do. If investors feel there are no surprises left, as the Fed is concerned, they will once again lever up, and inflate asset prices…QE1 - QE whatever. Rinse and repeat.

Bill Rafter adds: 

Here’s a related (perhaps derivative) question: Do stocks learn from each other?

Let’s say you take a list of ~6,000 stocks and look at them over a 10+ year period encompassing both up and down markets. And you come up with a trading plan that buys a stock if it exhibits pattern ‘A’ and sell it if it exhibits pattern ‘B’. It is not unreasonable to then have a universe of perhaps 150,000 transactions.

On the first pattern you pick you will find positive average results for certain stocks, while other stocks on average will be negative. Some of these winning stocks will knock the cover off the ball by having say 35 of 50 trades positive, and vice-versa.

Now let’s say you pick different patterns, and again you find a collection of stocks that outperform. You think this is going to be somewhat of a random process where some of the winning stocks from pattern A/B become losers when you try patterns C/D or Y/Z. And that does occur. But you also find that some stocks are consistent winners throughout the various patterns. And of course, some are repeat losers (perhaps hoo-do stocks).

That leads to further inquiry to find what constitutes winning qualities or hoo-do qualities. Note that this is not a study of profitable patterns since the behavior is exhibited across different patterns, some mutually exclusive like trend-following and mean-reversion. Nor is it a study of good management styles, as the same behavior is exhibited with ETFs, which typically have no management.

You then try to identify specific characteristics (or group membership) of the winners. You might think sectors, because the behavior also occurs in ETFs, but not all of the constituents of a winning ETF are consistent winners, and ETFs behave differently than their constituents. You trot through the various possibilities: volume, volatility, beta, name recognition, size, sales, earnings, debt, short interest, institutional holdings, etc.

Continuously you come back to the possibility that some stocks are simply winners and others hoo-dos, until you can prove otherwise. It turns out (tongue firmly in cheek) that this is a good thing to know.

Alex Castaldo writes: 

Probably I misunderstand or oversimplify the issue, but I think what is happening is like this.

Among stocks in your database there are some that have considerable price runups and declines, and others that have fewer such features. It is not exactly a question of volatility as conventionally defined, but it is somewhat related to it.

When you examine trading rules, selecting ones that are more successful with some stocks, you are necessarily picking up more stocks in the first category and fewer in the second.

There is a kind of oversampling at work that concentrates the successful population with stocks from the first category.

To take an absurd extreme to make things clear, if there is a stock that spent the entire data period at 10.00 (the ultimate quiet stock), then no method will make money from this stock, not trend-following, not mean-reversion, not seasonality, etc. This stock has zero chance of being among the 'winner' stocks, not because of its industry, or who is the CEO but simply because of the time-pattern of prices.




 I ordered a few fruit trees today from an orchardist in California and had an interesting conversation with him. I wanted some special plums and noted that the guy also had quite a variety of pluot trees for sale. I have never eaten a plumcot, pluot or aprium and asked his opinion. For those who don't know what I am speaking of, a plumcot is a 50-50 cross between a plum and an apricot. If you then take that and cross it with a plum you get a pluot, which is 75% plum to 25% apricot. Should you have crossed the plumcot with an apricot, you get an aprium, the 75-25 split long the apricot. Think of it as a fruit tree version of pairs trading.

According to that professional orchardist, the pluots of many (most?) growers are really plums masquerading as pluots. California has imposed a tax on the sale of plum trees, but not on pluots. So, surprise, surprise, they're all pluots. The tax police are fighting back with DNA testing for pluots suspected of being plums, and the growers are legally challenging the authority of the state to do the DNA tests.

Oh, when will they ever learn?



The problem with most of the polling is that it is done nationally. I argue that my own opinion does not matter as I vote in New Jersey. Stefan's opinion in a poll also does not matter if he votes in California. Both California and New Jersey are blue states and the opinion of anyone polled in those states is a curiosity only. Oklahoma and Texas are solid red states and their voters' opinions are equally valueless. Those opinions would be of value if and only if they reflected a national mood, which is doubtful. Thus the only polls that would seemingly matter are those which concentrated on the swing states: Florida, Ohio, Iowa to name a few. I have only seen one such poll (about a month ago) which did exactly that.



 1. There is a critical point in the market, a critical decision that the market gods weigh on a scale like Zeus with his balance scale deciding whether Achilles or Hector will win, that determines the market fate, and it is key and should be the focus of all news stories and market considerations but never is.

Never trust anyone but your family and best friend because everyone is disloyal in a pinch. Peleus was left for dead by his father in law after killing his brother in law to become ruler and this led to the Trojan war. Caesar trusted his best friends but they turned on him when an opportunity for power, money, and romance reared its ugly head.

3. Deception is key. The most successful Greek was the Deceiver Odysseus, and he tricked everyone he dealt with as the market tries to trick you with Odyssean power.

4. The goal is always to come home. Odysseus went home, as does the market. The only loyal ones were the wife and son and the best servant. The market retraces and comes home to break even an inordinate number of times.

5. Never mix romance with business or the market. The Trojan was was started by Paris intervening in romance and being swept off his feet by Aphrodite, and Achilles killed tens of thousands and prolonged the war by 10 years when Menelaus stole his mistress.

6. Don't try to walk with the Gods. Peleus married a half God and married her the last time the Gods and mortals mingled at a celebration and it caused him to be the most distressful of men. Trying to emulate Soros or the other greats is the seed of destruction.

7. Okay, give me the rest. And correct and tighten the above. I'm out of my depth but wanted to get the gist across.

Ken Drees comments:

 Like using a mirror against Medusa, one must plan against the adversary and sometimes use their expected attacks to beat them. Like shielding oneself from the siren song, one must be totally prepared, seek council before the journey (the trade) about what dangers are expected.

Also, it seems every entity in mythology had a weak spot. It's probably best to note these weaknesses in your thinking and in your emotions, not how can I beat the market, but how can the market beat me today?

Bill Rafter writes:

The greatest two rules:

(1) nothing to excess and (2) know yourself.

Pete Earle writes:

One lesson from mythology which resonates with me is the oracles/prophets/predictors almost always forecast correctly, but rarely in an obvious or immediately relevant way. The predictions made are usually realized, but not before taking extremely circuitous, and usually counterintuitive ways to reach fulfillment.

In my experience, predictions regarding the direction of equities or commodities inferred from option markets so often prove accurate…but only after traveling in the most wrong, most unanticipated ways.

Alston Mabry responds: 

 Pete, I think of that as "shaking the tree", i.e., we're gonna get there, but we're gonna shake out as many weak hands as we can along the way.

Peter Earle replies: 

Absolutely. Stop-running and the like as the "gods" way of seeing who's "worthy"; who can withstand the flood, the fire, the sturm und drang.

Jim Lackey writes: 

In 2008 I learned from Ryan Carlson– Sisyphus. There is a little useless book Wit and Wisdom from Wallstreet. So many of the quotes are the exact opposite from 3 pages ago… yet for a day they are seemingly sage advice. Worse for the long term. It's all good advice, yet in the mean time we must eat, and in the long term we all end up dust in the wind.

Traders lament when we miss profits. We are miserable when we lose. If we are not careful we are never happy. I have the habit of having to work myself up into a fury to win a race, pass a test or trade. My wife calls it "business mode" everyone else calls it being a jerk. Finally this year I have the ability to take a loss and this week miss a glorious rally and profit… yet at 4:20 PM its over. I am done pushing the boulder back up the hill for the day. I will return at 1:30am or by 7am, all but two business days a year. It can be torture if you do not like to trade, but if you love it…

Here is a quote from my kids music, "This is Our Science" by Astronautalis: "Our work is never done/ We are Sisyphus".

p.s I notice that if I don't like the rap beats I miss quite a bit of new poetry. I hear my teenagers say random lines and say what! That is amazing. Then I hear the song and say no wonder I never heard that line before. Damn drum machines.

Jack Tierney adds: 

Recently I've been reading up on complexity, system dynamics, and the unpredictable consequences that occur when tinkering with non-linear systems. The markets seems subject to all and, if I'm even remotely correct in interpreting the literature, there's only one certainty: expecting linear consequences (e.g, provide banks with more liquidity, bringing about an increase in business borrowing, resulting in a resurgent economy) is rarely, if ever, realized.

Instead, the unseen effects on unimagined factors, almost always derails the logic train. A source I've referred to on occasion is "Cassandra's legacy." Appropriately enough, the custodian of that site provides an interesting historical allegory, in the form of Goth Princess/Roman Empress, Galla Placidia, and her part in the demise of the Roman Empire. It's a very lengthy read and, unless history like this interests you, tough going. So, a few highlights:

"Managing any large structure is difficult and we tend to do it badly; a whole empire may be an especially difficult case. To do it well, we would need to use a method what I mentioned before: system dynamics; which is a way to describe systems and the relation of the various elements that compose them.

"…every time that the Romans fought the Barbarians, they could win or lose, but each battle made the Empire a little poorer and a little weaker. The empire was using resources that could not be replaced; non-renewable resources, as we would say today….the solution was not more troops but less troops. It was not more imperial bureaucracy but less imperial bureaucracy, not more taxes but less taxes.

"In the end, the solution was right there and it was simple: it was Middle Ages. Middle ages meant getting rid of the suffocating imperial bureaucracy; it meant transforming the expensive legions into local militias; have people paying taxes locally, in short transforming the centralized empire into a decentralized constellation of small states. Without the terrible expenses of the Imperial court and of the Imperial bureaucracy, these small states had a chance to rebuild their economy and start a new phase of prosperity, as indeed it happened during the Middle Ages.

"What Placidia could do as an Empress was, mainly, to enact laws….It seems that Placidia was acting according to her style; ease the unavoidable, don't fight it….Placidia forbade the coloni, the peasants bound to the land, to enlist in the army. That deprived the army of one of its sources of manpower and we may imagine that it greatly weakened it. Another law enacted by Placidia, allowed the great landowners to tax their subjects themselves. This deprived the Imperial Court of its main source of revenues."

Stefan Jovanovich comments:

As much as King George's scribbler Edmund Gibbon despised Christianity, he had the Middle Ages even more because its bureaucracies were the worst of all — local and mean and stupid.

Professor Bard should revise his history. What he wrote here — "Middle ages meant getting rid of the suffocating imperial bureaucracy; it meant transforming the expensive legions into local militias; have people paying taxes locally, in short transforming the centralized empire into a decentralized constellation of small states. Without the terrible expenses of the Imperial court and of the Imperial bureaucracy, these small states had a chance to rebuild their economy and start a new phase of prosperity, as indeed it happened during the Middle Ages." - is nonsense.

The Roman Empire's tax collections were always "local"; that is why Roman politicians were willing to pay such enormous bribes to be appointed provincial governors. The legions were also "local"; the Empire's expansion came from granting "foreigners" - i.e. the people we would today call Spaniards, French and Syrians - the privileges of citizenship, which meant they were also qualified to serve in the local legions. This was equally true under the Republic; "crossing the Rubicon" would not persist as a bad metaphor if Rome's soldiery had been centralized.

As for economics, whatever the "terrible expenses of the imperial court", they were nothing compared to the ravages of coin clipping. The solidus of the Eastern Empire maintained an unchanged weight and measure for 4+ centuries - a record that is likely never to be broken. (It exceeds the span of sound money for the British Empire and the United States of America put together.) After Princess Placida's day coinage, under the wonderful decentralization of the Middle Ages, effectively disappeared.

"Dearth of provisions, too, increased by degrees, and the scarcity of good money was so great, from its being counterfeited, that, sometimes out of ten or more shillings, hardly a dozen pence would be received. The king himself was reported to have ordered the weight of the penny, as established in King Henry's time, to be reduced, because, having exhausted the vast treasures of his predecessor, he was unable to provide for the expense of so many soldiers. All things, then, became venal in England; and churches and abbeys were no longer secretly, but even publicly exposed to sale." - William of Malmsbury wrote this in 1140 AD - the period that Professor Bard praises so highly for its progress over the degeneracies of the Empire.

Hume deserves the last word on this and most other subjects that interested him.

"Mankind are so much the same, in all times and places, that history informs us of nothing new or strange in this particular. Its chief use is only to discover the constant and universal principles of human nature."

Easan Katir adds: 

The Greeks have fooled people since the Bronze Age. Instead of a horse, they now have Trojan bonds.

Steve Ellison comments: 

Jack, the Atlantic had an article about why projects that had successful pilots often failed when rolled out to the general population.

Why Pilot Projects Fail– Here are some excerpts:

Promising pilot projects often don't scale … Rolling something out across an existing system is substantially different from even a well run test, and often, it simply doesn't translate.
Sometimes the 'success' of the earlier project was simply a result of random chance …

Sometimes the success was due to what you might call a 'hidden parameter', something that researchers don't realize is affecting their test. Remember the New Coke debacle? …

Sometimes the success was due to the high quality, fully committed staff. …

Sometimes the program becomes unmanageable as it gets larger. You can think about all sorts of technical issues, where architectures that work for a few nodes completely break down when too many connections or users are added. …

Sometimes the results are survivor bias. This is an especially big problem with studying health care, and the poor. Health care, because compliance rates are quite low (by one estimate I heard, something like 3/4 of the blood pressure medication prescribed is not being taken 9 months in) and the poor, because their lives are chaotic and they tend to move around a lot … In the end, you've got a study of unusually compliant and stable people (who may be different in all sorts of ways) and oops! that's not what the general population looks like.



 Does anyone have this in PDF? The Amazon hardcover is 300+ dollars.

Propaganda Analysis: a Study of Inferences Made from Nazi Propaganda in World War II

Jonathan Bower comments:

Here it is:

Scribd: The-Jewish-Enemy-Nazi-Propaganda-during-World-War-II-and-the-Holocaust

You can read it for only $21.

William Weaver responds:

That's the problem, there are a lot of books with similar titles, but the one I'm looking for is by Alexander George, written in 1959. The Gladwell article 

(Open secrets, Enron, intelligence, and the perils of too much information [10 page PDF])  that Jeff Watson mentioned referenced the book and it seems like an interesting read. Here is the Amazon listing:

Propaganda Analysis: Study of Inferences Made from Nazi Propaganda in World War II [Hardcover]

Alexander L. George (Author)

ISBN-10: 0837166306

Out of Print–Limited Availability. 

Bill Rafter writes: 

Regarding out-of-print books, some dirtbags have been identifying those books in a local library that are both (a) in demand, and (b) out-of-print. The dirtbag then advertises the book on Amazon as "used" for a substantial sum, say $150, or in your case $300. If someone orders the book the dirtbag then goes to the library and borrows the book.

A few days later he goes back to the library and confesses as to having lost the book in a fire or flood and pays the library their lost book fee, which might be $25. At that point he has paid the library for the book and cannot be charged with selling stolen property. And you as the buyer cannot be charged with receiving stolen books. But the whole thing really stinks.



A good friend of my daughter asked me for advice on the best way of winning a man's heart on a first or second date.

I told her to use the Jennifer Flowers Gambit (the surprise erotic interlude when stopped on a drawbridge) or the Lee Raziwell gambit (listen intently to everything he says and ask about his expansive greatness), or the Leona Helmseley Gambit (pretend that there is another suiter waiting for you that evening so you have to leave at 11 pm as nothing inflames a man more than competition) but I feel that others here are more sapient in this area and others and I  would appreciate your insights.

An Anonymous  writer comments: 

My conclusion is that the number one sign of a good long term relationship with a woman is based on the quality of her relationship with her father.

I am basically engaged to be engaged with a woman, and the emotional commitment on my end happened after a dinner where much of the conversation was her describing her relationship with her dad, and how he helped her with her math and physics homework, and then they would walk to the store for a treat, etc, and just the general way that her face lights up when talking about her dad.

So anyway, that's what worked on me. Perhaps she should try it.

/my 2 cents

 Gary Rogan responds: 

This sounds like good advice and the father thing is pretty well-known, but I'm just amazed that you have made some conclusions about long-term relationships after having dated women in around ten countries over two years. 

 Pitt T. Maner III comments:

Well then there are some who base decisions and strategies on a few minutes of observation. The HFT of the dating scene—your most important impression—the first 3 seconds!

 José Bonamigo shares:

From Forbes Magazine:

The mating practices of human beings offer a reason for thinking beauty and intelligence might come in the same package. The logic of this covariance was explained to me years ago by a Harvard psychologist who had been reading a history of the Rothschild family. His mischievous but astute observation: The family founders, in 18th-century Frankfurt, were supremely ugly, but several generations later, after successive marriages to supremely beautiful women, the men in the family were indistinguishable from movie stars. The Rothschild effect, as you could call it, is well established in sociology research: Men everywhere want to marry beautiful women, and women everywhere want socially dominant (i.e., intelligent) husbands. When competent men marry pretty women, the couple tends to have children above average in both competence and looks. Covariance is everywhere. At the other end of the scale, too, there is a connection between looks and smarts. According to Erdal Tekin, a research fellow at the National Bureau of Economic Research, low attractiveness ratings predict lower test scores and a greater likelihood of criminal activity.


Best regards from Brazil


 Gary Rogan inquires:

 After a while this degenerates into just socially dominant and not necessarily intelligent men. This modified effect can be readily seen in the Charles/Diana coupling, at least in the older Prince William. Of course how did Charles come about if the theory is correct? 

 Stefan Jovanovich comments:

Trusting Forbes magazine on stories of family history is more than a bit like buying a Degas ballerina sculpture from Toby Esterhase's Soho gallery. The notion that the 5 founding brothers were "supremely ugly" is part of the standard viciousness of the portrait of the Jewish banker as Shylock that survives to this day. There is no evidence of any special ugliness in their portraits.






The Rothschilds married money - the Ephrussis, the Guggenheims and the Oppenheims. One suspects that, as in most things, the question of beauty was left to the beholders.

In the 19th century the great minds were certain that criminal behavior could be predicted by examining the bumps on people's heads. It should hardly be surprising that we are back to estimating future viciousness by measuring the asymmetry of human features.



 Jim Wildman comments:

I would say that she can't on the first or second date. Winning someone's heart in a deep, lasting way, takes time. Anyone can fake interest for a while. What about when she is sick? When he is grumpy? When life intrudes on the lovers? Are their hearts still connected?

Granted, I haven't dated anyone for over 3 decades, but I have watched 3 daughters struggle with guys..

 Marion Dreyfus questions:

My question:

And some may find this offensive–

Does the ubiquity of pornography, specifically for the ones who purvey it day and night (I understand that equals a LOT of the male population), make falling in love with and making love with real women –including the physical aspects of affection–much more difficult than it used to be before every late-night channel offered a raft of such virtual substitutes for real relationships?

Rocky Humbert comments: 


(a) Korean BBQ. Nothing excites a man more than watching a lady handle chopsticks amidst an open flame. Alas, times change. Woo Lae Oak has gone out of business. http://nymag.com/listings/restaurant/woo-lae-oak/

(b) Take whatever advice a parent provides, and do exactly the opposite.

(c) Que Sera, Sera

(d) http://www.datingish.com/695368212/how-to-win-your-guys-heart/

Score 1 point for picking the right answer. Deduct 1/4 point for picking the wrong answer.

 Bill Rafter writes:

When you are fishing, you need to match the bait to the fish. Striped Bass like clams, but Bluefish and Flounder will eat anything, so you might as well use bunker. Think of it this way: a young lady would wear one kind of dress on a date and a different dress when meeting the young man’s mother.

If a man is 25 or younger he is probably only interested in one thing and he is not looking for lasting qualities. Not that there’s anything wrong with that. The interlude on the drawbridge is something he will never forget. A woman with an interesting job is attractive as long as it does not threaten him.

At some time the man starts to look for additional qualities in a mate. Maybe because of pressure from his parents he starts to think of having a family. Then he starts looking for someone who might be a good wife and mother. A schoolteacher is attractive in this case.

In foods, women are attracted to chocolate whereas men are attracted to cinnamon.

 Tim Melvin writes:

I told my daughter in response to a similar question that anything won so easily or quickly likely had little value in the long run. She should be herself at all times and the man who liked and fell for that woman was likely a better match. I taught all the tricks her old man had used over the years to win fair lady specifically so she could avoid them.

 Jose Bonamigo responds:

My intention with the Forbes extract was not to present solid evidence, just a likely explanation for couples like Charles and Diana (a common combination), as Gary pointed out.

Looking at the portraits it seemed to me they were "regular" uglies (just kidding)…

For a more scientific approach, at least in the physical part of dating:





FFT (Fast Fourier Transform) constructs a “best cyclic approximation to the data” that can be constructed with N cycles. And you get to pick the N value. Better yet is that the output is a smoothed representation of the raw data without any lags. Wow, no lags! However, FFT assumes that the cyclic behavior is repetitive from the beginning of time to the end of time. That’s great for fitting data, but not generally reliable for forecasting markets. Also, every time you add or drop a datapoint, a subsequent cyclic approximation will have different values over the entire period.

Suggestion: FFT is fine for seasonally adjusting past macroeconomic data, but your expected value of using it for trading will be negative.



"Never let a crisis go to waste." That political statement can have implications for those of us who speculate. Specifically, we can learn from what has recently happened, and by knowing history can hopefully avoid repeating those lessons.

I noticed just today that the option volume attributed to market makers ("MMs") was particularly high relative to that of firms and customers. That is quite logical, as emotional demand in options is usually offset by the MMs providing liquidity. That of course is the function of MMs, for which they are usually rewarded. But has that been the case historically, and if so, can it teach something?

I looked at the volume of market makers relative to their counterparties in two ways; first with the MMs buying and second with the MMs selling. Then I took the differences of those two calculations and smoothed it. More on the smoothing after you see the results:

Here you see SPX shaded to reflect the smoothed balance of MM liquidity. If SPX is blue, the MMs are providing liquidity to put buyers as of the previous day, and if orange, the MMs are providing liquidity to call buyers as of the previous day. These are of course generalizations, as I really do not know who is doing what, but simply following the logic of the positions. Of course it isn't perfect. If you are expecting certainty, you would be better off in another field.

The smoothing period is extremely interesting. These transactions by the MMs usually last 24 hours or less. But their counterparties do take positions and try to hold them. On average options positions by customers and firms last slightly less than a month, although they are frequently rolled-over. Thus the chart represents on-balance accumulated demand by options traders for liquidity. So what would you suspect to be the best smoothing period?

Wrong! It happens that the period used in the chart was a whopping six months. That fooled me also. That is, the data used in the smoothing is as "old" as six months. Of course, that is a static period; the best smoothing period will always turn out to be adaptive. But this works for illustration.

At this point the work is merely anecdotal; there is no statistical significance worth speaking about. But I present it here as something for further study.



 One has always wondered why the banks according to their regulators are being prohibited from investing in this and that thing, derivatives, mortgages, stocks et al, but never have I seen a mandate that they don't invest in sovereign debt of the solid as a rock countries such as those they invested in as did Rome after the Trojan war. Could it be that instead of being prohibited from such investments, the opposite is true, and that is why whenever a country is about to go bust, the banks are in danger of falling. Could it be that they are that foolish as to always hold the short straw?

Gary Rogan writes:

Based on multiple occurrences of coming close to the short end of the stick but somehow being saved by the US or the IMF it has not been a bad strategy. How many times has it happened in Latin America? The IMF resolved the early 80's crisis and Brady bonds were used in '89. So it wasn't just crazy people who would loan to Latin America that is guaranteed to blow up sooner or later. There was clearly an implicit understanding that French and German banks would be bailed out from their losses to the various PI**GS, and the way everyone behaved towards Iceland and Ireland, this was clearly expected that they would be the slaves to the big brothers, and the banks would be helped to be made whole by the taxpayers of the less-important countries, and when the bigger countries are involved the big brother taxpayers would have to chip in.

To the banks this was the frog in the boiled pot situation, except in stages: you warm the pot up a little bit, and then some savior helps you jump out, so you learn that the pot is safe. Then the frog jumps back in, and the pot is warmed up a little more, and the savior helps again, and so on. But now he can't help, but who cares? The old bank CEO's are enjoying margaritas some place where they used to lend to or even nicer and safer, or are dead, so on the average this was worth is to the banking flexion leaders. 

Bill Rafter writes:

Several of the 15th and 16th Century Florentine banks including that of the Medicis had problems with their sovereign loans. Despite problems the banks continued to lend for political/military reasons.

George Parkanyi writes:

Banks are large institutions and, like large institutions at the senior levels, don't pay attention to detail beyond a certain point. (I see that in government a lot for example.) Behind every major transaction is some mid-to-senior manager trying to close a deal, land a big client, or in the aggregate hit some number to make a bonus or whatever. I would think that to win a sovereign account would be a big deal, so of course you would trade or perhaps make a market in a client's debt in that situation. Smart sovereign clients, because of their size, can easily play one bank off against another depending on how hungry and competitive the players are at each. Sure institutions have systems, but ultimately deals are made by people, and the culture in investment banking is typically to do whatever it takes to make the deal, even if it means being "creative" and circumventing part or all of your controls, not digging too deeply in case you find something that might compromise the deal, and/or simply treating widely-accepted assumptions as fact (AAA credit, too big to fail etc…). There are many paths to these untenable outcomes, and they are all rooted in human nature. Nicholas Leeson never set out to bankrupt Barings, he started out by just trying to keep a big client happy.

Gary Rogan adds:

Still, moral hazard is what makes all of this possible (having some implicit savior). You don't see Procter and Gamble negotiating a deal with Walmart or some little dictatorship where they will sell them detergent at what winds up being a big loss, and least not very often. The suppliers who are foolish enough to do that disappear without anyone hearing about them, other than in some CNBC special about Walmart. Socialism in any form will ultimately destroy itself: when people have a right (or the idea that they have a right) to other people's resources, eventually they will consume/destroy enough of them to sink everyone involved.

Stefan Jovanovich writes:

The Bardi and the Peruzzi had two enormous technical advantages. Their staffs had fully mastered the science of double-entry book keeping and taken Pacioli 's discovery (probably lifted from the Byzantines) and improved it to the point that they could easily do present value discounting. This was a very big deal at a time when Italian banks were under the same prohibitions that banks in the Muslim world still operate under - charging interest was a sin. Their skill in double-entry was complimented by their shrewdness in dealing with the intricacies of canon law. The Bardi and Peruzzi were the first to figure out that they could get round the problem of usury by issuing loans at a discount and balancing their books by showing the difference between the cash paid out and the loan amount as a gift from the borrower. In a Christian world gifts were perfectly acceptable and (I love this part) the ability to receive them a proof of worthiness. Most of the discounting was not on loans but on relatively short-term bills of exchange. Many of them were remittances to the Papacy. You can see this in the list of the Bardi branches in 1300 - Barcelona, Seville, Majorca, Paris, Avignon, Nice, Marseilles, London, Bruges, Constantinople, Rhodes, Cyprus and Jerusalem. What is supposed to have killed both banks was, as Bill notes, their difficulty with sovereign debt. But it was only one sovereign - Edward III of England. According to the Peruzzis, Edward borrowed 600,000 gold florins from them and another 900,000 from the Bardi and then, in 1345, told them he would not be able to pay on the agreed upon schedule. The Italians had no choice but to agree to a workout, and they ended up taking much of their eventual repayment in wool rather than specie. The problem for them was that the combination of the Black Death and the exhaustion of the German silver mines had produced a monetary deflation that made the repayments worth far less than the nominal loan amounts. But, it is risky to take even this story at face value. The author of the Wikipedia article on the Hundred Years War (where Edward pissed away all the money) has his doubts. He writes that "the Peruzzis' records show that they never had that much capital to lend Edward III….. Further, at the same time Florence was going through a period of internal disputes and the third largest financial company, the Acciaiuoli , also went bankrupt, and they did not lend any money to Edward. What loans Edward III did default on are likely only to have contributed to the financial problems in Florence, not caused them."

What is not in dispute is that it took another half century for banking in Florence to revive on even a regional scale, and in scale and international reach, the Pazzi and Medici were secondary players compared to their 13th and early 14th century predecessors. The Medici are famous because of their adventures in Italian politics, their family stories and their art patronage; but, in terms of finance, it would be like comparing the current House of Baring with the one active during the Napoleonic Wars.



If you have not read this, you should: "Letters from a Self Made Merchant Man to his Son"  by George Horace Lorimer.

Craig Mee writes:

I simply mention Stan in passing as an example of the fact that it isn’t so much knowing a whole lot, as knowing a little and how to use it that counts.

Oh, how I have learned this the hard way. As an old squadron commander told me in my 20s, “You get a whole lot more bees with honey than you do with vinegar, young man.” Great advice, and I am happy to say I am finally following it many years later.



 Have you ever noticed how those who have done you the most wrong, or those who loathe you the most, when they come onto hard times will often come back to you asking for assistance. This often happens to me with former colleagues. I can't always differentiate between whether the colleagues are in such bad straights that they will go to their most unlikely and ill wanted savior, or whether they wish to take their worst enemy down with them once more before they finally go under. I believe it is a variant of rats deserting a sinking ship. The British Navy and I believe all navies have a standard order from their captain "every man for himself " when the ship is sinking. And there is doubtless maritime law about when it is legal to put the captain in chains, (albeit this is somewhat a different situation). I believe the idea has many market implications, especially when markets have gone to the nadir like last week, but more important is how to protect your life in such situations I think.

One finds that there are only 25 suicides a year at Niagara Falls these days, and The Golden Gate has much more, but one can't speculate as to whether the sight causes the suicides or whether people with suicide on their mind tend to go there to do the deed. As for market moves, they must cause many more such catastrophes but again whether the person seeks out the opportunity or the opportunity causes the action, or both, it would be hard to unravel and a quantitative study of the types of moves that induce same would be helpful for saving lives and profits. 

Russ Herrold writes:

I've had this happen a few times. I think the reason is that the former colleague or friend is sufficiently 'intimate' with the weak spot that their former friend had, and so can 'get past your guard' more easily.

Factor in some perverse pathological character trait, and they may even feel justifies in taking advantage of someone they feel has 'done them wrong' in the past. Indeed, it may be that there was an intent to deceive (conscious, or latent) from the onset of them approaching you, 'the mark'.

The best approach is to probably to buy the lunch, but to keep one's checkbook firmly locked up.

Polonius: (to his son)

Neither a borrower nor a lender be, For loan oft loses both itself and friend, And borrowing dulls the edge of husbandry.

Hamlet Act 1, scene 3, 75.77

and later


This above all: to thine own self be true, And it must follow, as the night the day, Thou canst not then be false to any man. Farewell, my blessing season this in thee!

Laertes: Most humbly do I take my leave, my lord.

Hamlet Act 1, scene 3, 78.82

The thought expressed by Vic is that there should be some heightened sense of gratitude if one is dealing with a moral person and 'offering the hand up' and a hand-out. But Twain echoed the Bard on this topic as well:

If you pick up a starving dog and make him prosperous, he will not bite you. This is the principal difference between a dog and a man.

- Pudd'nhead Wilson

Steve Ellison writes:

 When my children were 5 and 3, we hiked across the Golden Gate Bridge. There had recently been a freak accident in which a small child had somehow fallen through the small gap between the bottom of the railing and the sidewalk to her death. There were plans to replace the railing with one that went all the way down to the sidewalk, but the work had not been done yet, so I was keeping a close eye to make sure the children did not go too close to the railing. While my attention was diverted in this direction, I was almost caught off guard when the 3-year-old climbed on top of the one-foot high barrier between the sidewalk and the speeding traffic.

T.K Marks writes:

I, too, have walked across that bridge on numerous occasions. I'd walk over to Sausalito and take the ferry back. A spectacular stroll. One is still struck mid-span by the ease at which a despondent person could reach their goal. The curiously low railings prompt one to macabre thoughts. Who was the civil engineer involved with this project, Derek Humphry?

Stefan Jovanovich answers:

 The answer is Charles A. Ellis. Joseph B. Strauss did everything he could to claim credit for it (Strauss was to architects and engineers what Douglas MacArthur was to the Army and Navy - even when he was wrong, he was right - just ask him). Ellis reworked Strauss' initial proposal for a cantilevered suspension bridge - which would have been the mating of the Forth bridge with a ropewalk - and produced the design one sees today. Ellis did almost all the actual work - the calculations required for the computation of stresses, the specifications, contracts and proposal forms - singlehandedly, working non-stop for 2 years. After Ellis completed the work but before the final designs were submitted to the Bridge District's Board for its review and final approval, Strauss fired Ellis. There was no mention of Ellis in any report by Strauss, including the final report upon the bridge's completion in 1938. Ellis was the equal of Louis Sullivan, and like Sullivan he spent half his working life in total obscurity, unable to get any further commissions. Moisseiff gets credit for the development of deflection theory; but, as events proved (see "original bridge" section of Tacoma Narrows bridge), Ellis was the person who fully understood the necessary relationship between span length and flexibility. He is literally the father of the modern suspension bridge and the engineering theory behind it.

Bill Rafter comments: 

There was a psychology professor that published a study showing that the vast majority of Golden Gate jumpers took the leap on the side facing the city (facing East) rather than the ocean (West) side. The article then attempted to theorize why this might be the case, and he concluded that it was an attempt by the jumper to say goodbye one last time. Nice thought, but it totally ignores the reality that it would be damned hard to jump on the ocean side as that pedestrian walkway is almost always closed.

It must be particularly interesting to be on the bridge when one of the big carriers goes under, as they have to time it with low tide to clear.



 One can certainly use levels or changes in the price of gold to trade equities indices (i.e. SPY as opposed to gold mining stocks for example).

However that same person would have more success if he used the "volatility" of the price of gold to trade those same markets, at least since 1990.

That condition is not unique to gold.



 My suggestion [for understanding market inter-relations] is to set markets against each other and then look at how they react to a common metric. What you will find is that such an exercise yields valuable timing information. What many perceive as the most difficult part (choosing the metric) is often nothing to worry about. That is, almost all of them work, such that the news of markets turning is writ large across the landscape.

The first consideration however is to compile the various markets as assets in which to invest. That is, instead of using yields, one must use prices as it is prices by which they compete. The best trick I can pass on is for the game theorist to start by looking longer term and then shorten up. Think years and quarters rather than intraday.There's more if there is any interest.

Rocky Humbert writes: 

One notes that "real" interest rates have backed up about 40 bp in the past week, and gold is responding in kind. Eddy Elfenbein is one of several people who have postulated a relationship between Gold Prices, Real Interest Rates, and Gibson's Paradox. Correlation is not causation of course — but his model has been working brilliantly. Read about it here.

It's also a good moment to brush up on Gibson's Paradox which notes that interest rates follow the price level and not inflation (when operating on a gold standard). According to what I've read about Gibson (which is very little) , everyone from Larry Summers to Milton Friedman accept the existence of Gibson's Paradox … but noone seems to agree on the underlying theory.

Just a quick and dirty note: Eddy Elfenbein's model says that for every -1% (annual) move in real interest rates, gold compounds upwards by 8% (annual). So back of the envelope, a 50 basis point rise in real yields "should" clip gold by 4 or 5 percent or so … and amazingly that's what happening. Now … all I need to do is PREDICT where real interest rates will be next week…



This is a medical/physics & math article that I believe has implications for we speculators. The counterintuitive point is that viewing data through a somewhat murky filter can actually give you a better picture that viewing that data directly. Looking at data through filters is essentially what specs do when they look at moving averages, point-and-figure representations, Heiken-Ashi, data clouds or any of the numerous tools available.

I know of no other compact optical system that combines such high resolution with a field of view that large," says Mosk. He hopes to see a hybrid system that combines his resolution with Choi's speed and field of view. Ultimately, he says, the technique could improve surgeons' views of what to cut during keyhole surgery. "Light scattering may seem detrimental to imaging, but in fact a scattering system can make an almost perfect lens.



 Yesterday after the market close I put on my bathing trunks and went to take a swim in the ocean. On my short walk I was surrounded by perhaps 50 dragonflies. That didn't bother me in the least as I know dragonflies do not bother humans and that they eat black flies and mosquitoes that do bother humans. The west wind that we have had for days brought in the flies and mosquitoes, but also their predators. Well here I was in the middle of the swarm, and it occurred to me that I was being used by the dragonflies as bait. That says a lot about their intelligence, which would also be indicated by their exceptionally large eyes. It made me wonder if I have ever been used as bait by other market participants, perhaps with less mutualism in mind.



Does the incessant parade of illegal/insider trading, government manipulations, etc, of smart Ivy grads evidence the difficulty of getting rich in markets, or simply that dishonesty and greed is pervasive at all intelligence levels?

Gary Rogan writes:

It's probably evidence of both, but also of the illusion that highly successful people often seem to have of being invulnerable to normal negative forces, such as being punished for attacking hotel maids or being revealed for having a secret "love child" while running for the Governorship of California, or having an easily identifiable affair while running for the presidency. 

Bill Rafter writes: 

Don't the B Schools all have required ethics classes? Come to think of it, doesn't the industry regulators also require ethics classics?

Rocky Humbert writes: 

The United States has an incarceration rate of 743/100,000 population.

The New York City's financial industry employees 344,700 employees.

If the pro-rata incarceration rate for Wall Streeters were at the national average, there would be 2,561 Wall Streeters in the Big House right now. Or, with 35,400 employees, 263 of these people would be Goldman Sachs employees.

Since neither of these facts are true, the inescapable conclusion is that Wall Streeters are either more lawful than the national average (or they have better defense lawyers).





The Winners of the least effort contest were jointly in a tie. Mr. Gary Rogan and Mr. Steve Ellison. I will split the prize between them. The creative and physical ideas of Mr. Rogan were very excellent and best of all, but there was no testing. Mr. Ellison gave a great test, and a complete answer, but Rogan can't be denied his place either. vic

I'll give a prize of 1000 to the person or locus of his choice that comes up with the best way to test the principle of least action or a related principle of least effort.

It's in honor of my grandfather. Whenever I'd ask him which way he thought the market would go he'd say, "I think the path of least resistance is down" starting with Dow 200 in 1950. We need some more quantification around here.

You might consider max to min or a path through a second market back to home. Or round to round? Or amount of volume above or blow. Or angle of ascent versus angle of descent. Or time to a past goal versus the future? Or some mirror image or least absolute deviation stuff?

Sushil Kedia writes:

With utmost humility and clearly no cultivated sense of any derision for the Fourth Estate, I would submit that since it is the public that is always flogged and moves last, the opinions of all media writers, tv anchors are the catalysts, the penultimate leg of the opinion curve. A test of the opinions of the fourth estate on the markets would provide the most ineffective wall of support or so called resistances. Fading the statistically calculated opinion meter (if one can devise one such a 'la an IBES earnings estimate a media estimate of market opinion) and go against it consistently over a number of trades, one is bound to come out a winner. Can I test it? Yes its a testable proposition, subject to accumulation of data.

Alston Mabry writes:

The following graph (attached and linked) is not an answer but an exploration of the "least effort" idea. It shows, for SPY daily since August last year, the graph of two quantities:

1. The point change for the SPY over the previous ten trading days.

2. The rolling 10-day sum of the High-Low-previous-Close spread, i.e., "max(previous Close, High) minus min(previous Close, Low)". This spread is a convenient measure of volatility.

Notice how these quantities move in tight ranges for extended periods. These tight ranges are some measure of "least effort", i.e., the market getting from point A to point B in an efficient fashion. As one would expect, the series gyrate when the market takes a temporary downturn. Also note how when one of the quantities swings above or below it's mean or "axis", it seems to need to swing back the other way to rebalance the system.

Bill Rafter writes:

 This nicely illustrates how relative high volatility is bearish on future price action.

Jim Sogi writes:

The path of least resistance would be the night session. Low liquidity allows market mover to move market. Every one is asleep. Dr. S did a study some years ago. Updating shows total day sessions yielding 94 pt, but night session yielding 232 points. Don't sleep…stay up all night or move to Singapore. Recent action is in line with hypothesis.

Bill Rafter writes:

Haugen's "The Beast on Wall Street" (i.e. volatility) came to the conclusion that if you want less volatility in the markets, keep them closed more, to essentially force the liquidity into specified periods. That is, 24 hour markets promote volatility. Or a corollary was that a market is never volatile when it is closed. [this is from memory and I may also be regurgitating from a personal conversation with him]. An oft cited example is the period in the summer of 1968 when equities were closed on Wednesdays to enable the back offices to get up to date with their paperwork and deliveries. During that time the Tuesday close to Thursday opening was less volatile than expected (twice the daily overnight vol).

One could take this thought and stretch it to say that the periods of least resistance would be those without heavy participation. One could easily compare the normalized range (High/Low) of those periods versus the same of the well-participated periods.

Craig Mee writes: 

Hi Bill,

You would have to think that in 68 there was sufficient control of price and news dissemination. In these times of high speed everything, that this could create bottlenecks and add to the volatility. No doubt a bit of time to cool the heels i.e limit down and up for the day restrictions, is a reasonable action, even if it goes against "fair open and transparent markets" but unfortunate it seems little is these days.

Bill Rafter replies:

I should have been more specific about the research: take the current normalized range for those periods of high liquidity (when the NY markets are open) and compare that to the normalized range of the premarket and postmarket periods. Do it for disjoint periods (but all in recent history) so you don't have any autocorrelation. My belief is that you will find there is less volatility intra-period during the high liquidity times. While you are at that you can also check to see during which period you get greater mean-reversion versus new direction.

If that research were to show that (for example) you had greater intra-period volatility during the premarket and postmarket times, and that those times also evidenced greater mean-reversion, you could then conclude that those were the times of least resistance. That would answer Vic's question. Okay, now what? Well you could then support an argument that with high volatility and mean reversion you should run (or mimic running) a specialist book during those times. That's not something I myself am interested in doing as it would require additional staff, but those of you with that capacity should consider it, if you are not yet doing so.

Historical sidebar: '68 was a bubble period caused in part by strange margin rules that enabled those in the industry to carry large positions for no money. The activity created paper problems as the back offices were still making/requiring physical delivery of stock certificates. The exchanges closed trading on Wednesday to enable the back offices to have another workday to clear the backlog. The "shenanigan index" was high during that time.

Phil McDonnell writes:

Bill, you said "During that time the Tuesday close to Thursday opening was less volatile than expected (twice the daily overnight vol)."

For a two day period and standard deviation s then the two day standard deviation should be sqrt(2)s or 1.4 s. So the figure of twice the volatility would seem higher than expected.

Or am I missing something? 

Steve Ellison submits this study:

The traditional definition of resistance is a price level at which it is expected there will be a relatively large amount of stock for sale. 
Starting from this point, my idea was that liquidity providers create resistance to price movements. If a stock price moved up a dollar on volume of 10,000 shares, it would suggest more resistance than if the price moved up a dollar on volume of 5,000 shares.

To test this idea, I used 5-minute bars of one of my favorite stocks, CHSI. To better separate up movement from down movement, for each bar I calculated the 75th and 25th percentiles of 5-minute net changes during the past week. If the current bar was in the 75th percentile or above, I added the price change and volume to the up category. If the current bar was in the 25th percentile or below, I added the price change and volume to the down category.

Looking back 200 bars, I divided the total up volume by the total up price change to calculate resistance to upward movement. I divided total down volume by the total down price change to calculate resistance to downward movement. I divided the upward resistance by the downward resistance to identify the path of least resistance. If the quotient was greater than 1, the past of least resistance was presumed to be downward; if the quotient was less than 1, the path of least resistance was upward.

For example:

                           Previous 200 bars
   Date     Time     Up Points Volume  Down Points Volume Resistance

3/25/2011   15:50   53   6.49  99431    61  -7.38  149867     15311

   Down       Resistance     Actual
Resistance      Ratio      net change
     20310       0.754       -0.03

Unfortunately, the correlation of the resistance ratio to the actual
price change of the next bar was consistent with randomness.



 Umberto Eco wrote a great essay about how when new products start they are used first by high end users, and then gradually diffuse to the masses so that by time the masses use them, the marginal utility keeps reducing and the first users that got real value out of it stop using them. He points to such things as railroad use and cell phones as examples.

We have see how IPO's prospectuses follow this model with info in it being completely worthless as they have to go through so many hoops that it becomes merely a boiler plate to reduce the settlements in class action litigations when the case is settled.

One notes now the apparently standard thing in financial statements "cautionary note regarding forward looking statements".

I note in a company like Rimm 30 cautionary notes including "difficulties in forecasting quarterly results" and "regulation certification and health risks". My goodness, there was a time when management statements could actually convey useful information that had a high marginal revenue.

Could we attribute this syndrome to crony capitalism or flexionism or just a natural outgrowth of the law of diminishing marginal utility? 

Rocky Humbert writes:

While the chair's assertion that disclaimers have proliferated since the passage of the PSLRA is correct, there is scant evidence that management statements ever have consistent predictive value w/r/t either the organic performance of the business or its market valuation — over a reasonable investment time period. See wikipedia on the Private Securities Litigation Reform Act.

One reason for this is that companies which are performing well have no need for management cheerleaders or CEO soothsayers; the market will eventually figure that out on its own. In fact, the worst companies are the ones where the CEO is front and center (giving "upbeat" guidance) when things are rosy, but then when things turn challenging, release 8-K's on Friday afternoons using terms such as "exogenous factors" and "one-time adjustments" (and the CEO is nowhere to be seen.) Citing Philip Arthur Fisher's Rule #14: "Does the management talk freely to investors about its affairs when things are going well but "clam up" when troubles and disappointments occur?" It's a rare company that does an IPO or secondary when business is sickly (the exception being banks which sell stock at the behest of regulators.) Hence the entire IPO process can be viewed as a possible violation of Rule #14.

On a related point, one notes that INTC stock (which was mentioned recently by Dr. P) has a compound annual return since 1982 of about 15.6% per year (versus 11% for the S&P). During the same period, AAPL stock has produced a 17.5% compound return. Yet, right now, INTC has a 10x p/e and AAPL has a 17x p/e. Both of these companies have demonstrated good long-term organic growth, RoE, product innovation, and impressive market dominance. Yet, if Mr. Market would reward Intel with only a market multiple, it's return-to-shareholders would blow away Apple — demonstrating once again that Mr. Market's valuation at any given moment dwarfs every other factor for a profitable enterprise. I submit that it's folly to attribute this irrefutable statement to crony capitalism or flexionism or the law of diminishing marginal utility. The blame should be place squarely on the market participants who continue to make the same mistakes (such as buying INTC at a 70x p/e on 3/1/2000) but shunning it at a 10x p/e on 3/1/2011. 

Ken Drees writes:

Consider the cell phone and its recent tracking news out of apple– or police being able to plug a device into your cell phone and download all your data from it– the high end user will now need tech applications to shield their privacy and will demand a next generation product that the masses do not have– a private communication device. The cycle keeps moving forward. Maybe a self destruct feature will come on the scene.on the subject of mumbo useless jumbo in fin states. Is not persistency of litigation like ants digging into the timepiece to blame for the creeping destruction of worthy information?

Bill Rafter writes:

In looking to eliminate stocks in mergers or merger talks I cannot always get that information as quick as I would like. Sometimes I have to resort to looking at the individual stock's news headlines. Before I even get to the news about the merger I see the inevitable: "The law office of Dewey, Cheetham and Howe launches an investigation into possible breaches of fiduciary duty by the Board [of the company]…"

That, I contend, is why you don't get useful information.

An Anonymous Commenter writes:

I recently read an article that the author was try to further disgrace a Euro based company whose board member had made a remark at a meeting referring to "the weaker sex". The article told of the various ways, non business groups and political active parties tried to protest these remarks. However while raising a good smoke screen; the parties complaining were inefficient and did not understand business. Has any body done a study on the stock price of a company whose leadership made non PC remarks? Could it actually increase the price, due to the signal of boldness and management willing to think outside the box? Would not such a study have been quoted in these articles that hold a company up to ridicule? Could such a study have been done but be not published due the opposite than hoped for results?



 There has been discussion both recently and historically concerning the relationship of stocks and bonds. Indeed the Daily Speculations website this year started color coding the days on the calendar based on the different movements of those two basic assets.

Much of the discussion concerning this subject has been oriented around "counting". While I value that approach I must point out that the yardsticks by which stocks and bonds are measured in such analysis are of different lengths. Specifically, those markets do not open and close at identical times, which means the daily recording of their respective changes may not actually reflect their movements when they were contemporaneously open. If the daily recordings are not exactly representative, then counting those recordings in the hopes that the errors will "all come out in the wash" may not be the best approach.

Changes (as opposed to levels) are the appropriate data to study. But because of the different time periods, a period other than daily would be more representative of their respective actions.

Instead of counting the daily changes, I have been following the moving correlation of those markets. Whereas most investment professionals are of the belief that stocks and bonds are essentially opposites, we have found otherwise. For example, if you were designing a most bullish environment/setting, I believe that would be when both stocks and bonds were rising. That is, you had a positively trending stock market and declining interest rates. That can be easily modeled by a moving correlation of the two and a positive slope of their prices. Here's a chart (weekly data) of exactly that:

If it doesn't show up well, here is a link.

As you can see, periods of positive moving correlation and positive slope tend to be good times to be long equities. More importantly however is that price collapses are easily seen as periods when that best condition does not exist. The worst condition is when stocks and bonds are both declining. But the other two "not best" conditions reflect the standard (macroeconomic) business cycle and have interesting trading implications all their own. All of this can be non-subjective and can be tested. We have found it to be statistically significant. Furthermore, I show it here on a weekly chart for convenience, but the data is daily and should be watched daily.

I would posit that Daily Specs could find this an interesting area of study. For years my shop had used this as an important part of avoiding trouble, basically to tell us when to play in equities and when not. We only abandoned it when we found something better.

Some further comments/tips:

For bonds I used the DJCBTI because it is price-based and not expiring every six months. It is also third-party. I could make my own bond index, but then open myself to the charge of jury-rigging the results. N-values are hugely important. But adapt, don't optimize N-values. Also you will have to consider what level of moving correlation is significant (i.e. a positive value of .01 is somewhat of a yawner).

Bill Rafter, MathInvest

Kim Zussman writes:

Using TLT for bonds and SPY for stocks (2002-present, including divs), checked weekly return for stocks after prior weeks which were either (bonds, stocks) up up, up dn, dn up, dn dn, vs zero:

Variable    N      Mean     StDev   SE Mean      95% CI             T
BUSU      113   0.00165  0.0164  0.0015  (-0.0014, 0.0047)      1.07
BUSD      139   0.00156  0.0363  0.0030  (-0.0045, 0.0076)      0.51
BDSU      139  -0.00022  0.0213  0.0018  (-0.0038, 0.0033)   -0.13
BDSD       60    0.00519  0.0242  0.0031  (-0.0010, 0.0114)    1.66



 Very interesting article on Galton:

One, two, many: The prehistory of counting

The Victorian idea that "primitive" tribes can't count has cast a long shadow over efforts to understand the origins of mathematics

LOOKING back, Francis Galton would call it "our most difficult day". It was 4 March 1851, and the young English explorer was beginning to appreciate the obstacles confronting his attempts to map out the Lake Ngami region of south-western Africa. Struggling to navigate a narrow ridge of jagged rock, his wagon had "crashed and thundered and thumped" while his oxen "charged like wild buffaloes".

To make matters worse, Galton had little faith in his local guides from the Damara tribe, who appeared to lack even an understanding of basic arithmetic - a situation Galton found "very annoying". He recounts that having established an exchange rate of one sheep for two sticks of tobacco, he handed four sticks to a local herdsman in the expectation of purchasing two sheep. Having put two sticks in front of the first sheep, the man seemed surprised that two sticks remained to pay for the second. "His mind got hazy and confused," Galton reported, and the transaction had to be abandoned and the sheep purchased separately.

As further evidence of the apparent ignorance of the Damara, Galton wrote that they "use no numeral greater than three" and that they managed to keep track of their oxen only by recognising their faces, rather than by counting them. At a most inopportune time for his expedition, Galton seemed to have stumbled into a world without numbers.

To a modern reader, these tales in Galton's 1853 Narrative of an Explorer in Tropical South Africa seem little more than pithy anecdotes that reflect his prejudices as a gentleman of the growing Victorian empire. (His preoccupation with the supposed inferiority of other peoples persisted in his later work in eugenics.) Within 10 years, however, those same reports of primitive innumeracy were being used by the finest scientific minds of Victorian Britain to glimpse the savage condition of prehistoric humans.

Read the full article here

Victor Niederhoffer writes:

This seems wrong to criticize Galton. What am I missing? 

Steve Stigler writes: 


The author is a 1st year PhD student at Princeton who isn't even working on Galton, and writes carelessly without knowledge. See his bio. He looks bright but has a lot to learn.



What would be the correct way of running a multiple correlation to check this theses that a major part of the Rsquared comes from the DXY? Run one with it and one without it and see the R squared. Perhaps not enough. How much of the variation is explained by DXY amongst a bouquet of "relevant" variables.

Perhaps I am trying asking too many questions in a single one. Quants on the list will perhaps not mind tossing me out of the kitchen table with just a few strokes of their insightful knives.

Bill Rafter writes:

What would be the correct way of running a multiple correlation to check this theses that a major part of the Rsquared comes from the DXY? Run one with it and one without it and see the R squared. Perhaps not enough. How much of the variation is explained by DXY amongst a bouquet of "relevant" variables.

Perhaps I am trying asking too many questions in a single one. Quants on the list will perhaps not mind tossing me out of the kitchen table with just a few strokes of their insightful knives.



 So is the consensus now among us non flexions  that the radiation danger is merely exaggerated 100 fold so that technology in the US will be set back 30 years, and government intervention will be lubricated for the next 4 years to deal with the crisis which seems so much worse to the US than the Japanese and IAEA? This is not meant to diminish the magnitude of the tragedy in Japan, but merely to wonder if we believe that the subsequent dangers have been much exaggerated for flexionic profit?

Anatoly Veltman writes:

Yes, of course. One thing to be sure about is that T.Boone Pickens' funds will start getting ahead, as Natural Gas projects (like gradual highway infrastructure to facilitate filling-up vehicles, especially trucks and such) should finally be given light-of-day.

Bill Rafter comments:

"Never let a crisis go to waste."

Jay Pasch writes:

Buy the clashing of bearish cymbals, and sell the euphoric opposite…

Kim Zussman ironizes:

Buy the clashing of bearish cymbals, and sell the euphoric opposite in flat/choppy markets. If markets ain't flat or choppy, don't buy and sell 'em. 

Steve Ellison writes: 

No doubt it was my poor judgment, but from the perspective of operating a specialty line in panics, the moments of panic in the past week in the S&P 500 seemed too brief and ephemeral to go all in. The changes since the earthquake were:

3/11 +11.7
3/14 -10.7
3/15 -15.2
3/16 -21.4
3/17 +14.9

There were three moderately large down days in a row, but for perspective, the S&P 500 futures are still up 1.5% year to date. Only for the briefest of moments did they trade below the 1247.9 year-end close of 2010.



 What is the geophysics of thinking that more natural disasters are more likely now that the earth quake has occurred?

Kim Zussman shares:

Read this article.

Rudolf Hauser writes:

Another factor to consider is the shifting of the magnetic poles. This is reportedly associated with violent swings in weather and more earthquakes and volcanic explosions. Apparently there has been a marked acceleration in the rate of shifting in the past few years. Some question whether this might be the cause of recent weather extremes and geological activity. Since such shifts occur only every half million years or so we obviously have little idea of how they progress. If this is a real reason for concern it is an issue far more immediate and important that the global warming fears.

Pitt T. Maner III writes: 

There have been suggestions of a connection with renewed (regional?) vulcanism.

The last eruption of Mt. Fuji , for instance, occurred 49 days after the previous largest earthquake in Japanese history.

Another Japanese volcano has resumed activity but cause/effect from the March 11 quake may be tenuous.

The volcano, Shinmoedake, is famous for standing in as the villain's secret rocket base in the 1967 James Bond film, "You Only Live Twice". 

Bill Rafter comments: 

Earthquakes and volcanism are simply different manifestations of the goings on of plate tectonics.

Read this article from New Sceintist: "The megaquake connection: Are huge earthquakes linked?".



 From the description on the NY Public Library's site:

Darwin's Disciple, George John Romanes

Thursday, March 3, 2011, 1:15 p.m.

Stephen A. Schwarzman Building, South Court Auditorium (Map and directions)

Fully accessible to wheelchairs

George John Romanes (1848-1894), best known today to the intellectual community for founding the Oxford University lecture series still bearing his name (1891), was a major figure in the history of biology for his advocacy of Darwinian evolution as well as his contributions in animal physiology-discovery of a nervous system in invertebrates-and in animal behavior-recognition of the ability of animals besides humans to reason. But perhaps Romanes's greatest legacy is the support he gave Darwin when it was most needed.

After publication of Darwin's Origin of Species in 1859, Darwin and his work was under attack almost from the outset, not only by the religious establishment but also by scientists offended either by the theory itself or by its primary mechanism, natural selection. Darwin and his theory needed support from other naturalists, and Romanes became a strong advocate for Darwinian evolution in the decade preceding Darwin's death in 1882, and the years before his own death in 1894, thereby filling the vacuum left by evolutionists who disagreed with Darwin on the mechanism by which species evolve.

A former Writer in Residence in the Library's Wertheim Study, Joel S. Schwartz is Professor Emeritus of Biology at the City University of New York, where he served on the faculty for forty years. His scholarly interests have focused on nineteenth century natural history, on the development of natural history, and on how maritime exploration stimulated discovery in the natural sciences. He has published numerous papers and delivered many talks on Charles Darwin, Alfred Russel Wallace, Thomas Henry Huxley, and other eminent Victorian naturalists. His book, Darwin's Disciple: George John Romanes, A Life in Letters, was published July 2010 by Lightning Rod Press at the American Philosophical Society. Currently, he is Contributing Editor of the Darwin Manuscripts Project, based at the American Museum of Natural History.



What is Different This Time?

 This chart illustrates one of the problems with the equities market of late. The [unnamed]  variable shown was a model of consistency up until September 2010, and then started behaving less reliably. The data represents actual transactions (all of them), but would be unknown to most practitioners. Thus it is not the case of a variable being followed by so many that it becomes second-guessed and thus unreliable.

For what it's worth, this is NOT one of our decision variables.

Bill Rafter is president of Mathinvestdecisions.Com, a quantitative investment firm.

« go backkeep looking »


Resources & Links