The Kelly Criterion, from Ralph Vince

April 22, 2010 |

Many of you have heard me rail about the misapplication of the Kelly Criterion in trading, having witnessed firsthand how many funds are misapplying it. A typical conversation starts out with my being asked what someone should do when his allocation, as given by the Kelly Criterion, "…is greater than 100%." Repeatedly I have made the argument that the Kelly Criterion is not Optimal f, and that people conflate the two. A paper on my web site finally articulates that. Scroll about halfway down the page, in the "Related" section is a link to my paper "Optimal f and the Kelly Criterion."

I really like your exploration of the leverage curve and I'm glad I have someone to talk to about these things. Sometimes I feel that public debate is too sharp, but it's not meant to be that way.

A few things worth noting:

1) (p5) You said, "Since the Kelly Criterion, (1,1a,1b[r=0]), requires percentage returns as input, whereas the more general Optimal f solution requires raw data points." Kelly can be similarly calculated based on raw data, as I showed in a note on my blog post Practical Kelly Betting II.

2) (p13) f is not bounded to the left by the Kelly Criterion. For example in the case of an even bet on a biased coin with probability of heads p, Kelly's f=2p-1<=0 for p<.5, as we intuitively expect — basically short the game if possible.

3) Bounding Optimal-f between [0,1] is simply a change of units. I wouldn't say that one or the other is better. Both have very good arguments in their favor. It's good to be aware of the merits of both. Perhaps both should be given on any fund prospectus.

4) What I don't like about your W, i.e. max loss, is that it is hard to specify. An example is in trading GOOG: The maximum you could lose is 100%, right? since it's equity and if the company goes bankrupt you potentially get nothing. You might argue to base it on historical data, but that's bad too — let's say it's one week since GOOG's IPO. Then your historical data are useless. The point is that W doesn't help, and only adds another hard to specify number into the objective function. The beauty of Kelly is that it crystallizes the essence of risk and leverage so you can see their exact interplay.

In general I would like to see more unification, rather than divisiveness. This is merely a semantic difference of course, but to move the field of risk management forward, it would be nice if the three big factions - Leverage Space, Markowitz, Kelly — could pool resources and ideas.

Max Dama blogs as Max Dama on Automated Trading

Ralph Vince replies:

These issues are addressed in the paper.

I wrote it because I was seeing the possible future damage that was likely to befall people and funds due to the misapplication of the conflated notion of the Kelly Criterion and the Optimal f notion. The values returned by the two are not the same, except in what I refer to in the paper as the "special case." It is only in the special case that the Kelly Criterion solution equals what is the optimal fraction of a stake to risk. In all other cases, it yields a greater number (with no upper bound), representing a leverage factor, and that leverage factor is the same value as the optimal fraction to wager only in the special case. This is the problem that spawned the paper as I repeatedly saw people and funds who were way over-leveraged (where there is never any benefit, only the assuming of potential future danger) because they were unaware of what the Kelly Criterion was an answer to (a leverage factor as opposed to the optimal fraction to wager).

And I find this misguided concept pervasive in the industry as well as academia. Kelly himself refers to it as the optimal fraction, but it is not. Rather, in the "special case," it is a value equal to the optimal fraction (and in all other cases, if someone thinks that are using the optimal fraction, based on the Kelly Criterion solution, they are dangerously over-leveraged beyond what is the optimal fraction).

(Incidentally, if anyone on the list is not aware of Max's blog, it is perhaps the most interesting and meaty reading out there and you're missing something terrific if you don't visit it. And Max, get back to posting on it!)

And this now addresses the notion of "more unification rather than divisiveness which you bring up." Yes, the two can be used interchangeably, and the formulas for converting between the two are provided in the paper. But why would I want to do that?

The answer provided in the objective function of the Kelly Criterion solution is the expected value of the sum of the logs of my returns (this is a meaningless value). The objective function of the Optimal f is the multiple I make on my stake on each play. This value clearly does have a certain, useful meaning. Secondly, there are many virtues to having a number bound between 0 and 1 as opposed to a value which I would have to convert to get to between 0 an 1 (these issues too, addressed in the paper, i.e. the singularity, which, if you start to examine a very bright mind as yours will find tremendous real-world benefits to this, which we won't go into right here, but I just know you will see it and fly with it), as well as allowing a combining of the assets comprising these bound values together.

(You might think I am railing against these things, but the enthusiasm to this end is, hopefully, the fuel to elicit critical examination of these existent concepts. This critical examination was clearly missing or I would not have encountered so many people and funds which were mistakenly over-levered as they were.) Divisiveness, in this aspect, is my responsibility(!) How you perceive these things — how they fit together in your mind and what you do — is your prerogative entirely. I am trying to foster your critical thinking on these aspects. Having met you, it is precisely those (far-too-rare) younger guys like yourself I am trying to stir, and the ultimate unification of these ideas will be the product of your good minds. The older fellows — we won't be here long enough to give a rat's butt what they think!

Lastly (and you mentioned Markowitz), and you have heard me rail that unlike an LSP/Optimal f -based portfolio, it does not address leverage directly. And I say that because the latter has Variance embedded (negatively) in its return aspect, rather than juxtaposed to it. Recall the Pythagorean relationship between the geometric mean, arithmetic mean, and standard deviation in the holding period returns and this becomes evident.

And when I do this, and introduce a risk metric other than variance to juxtapose it to, I find I have optimal allocations which are not on the efficient frontier of a mean-variance configured portfolio (as in the chart which follows this posting, which you have seen, for two 2-1 coin toss games played simultaneously. The mean variance portfolio, which would lie upon the diagonal line — and would, in fact, find the optimal portfolio upon it –does not when a risk metric like drawdown is imposed).

The unification of these ideas is up to guys like you. I'm just an old guy standing on the bus, stepping on people's toes. (That's my job here!) I AM trying to get you stirred and thinking on these things (and I can be, and frequently am, wrong on some of these things and other things too). From where I stand, to have guys like you thinking, critically, on all of this, and putting it together (and sharing your thoughts on it all as you do!) is infinitely better than watching people walking into the paths of loaded guns because of the very ignorance surrounding these matters that precipitated this paper.

BTW — The sun is just peeking through here, it's birdland out there. The dark, spooky Buckeye Trail is calling me, and I am going on a long, slow, indulgent slog — a sabbatical from the world for the better part of today. What could be better than that?

Steve Ellison writes:

As a math enthusiast, I love the Optimal f formula. I have been using, it since I started trading in 2006. My biggest challenge as a practitioner has been changing cycles, i.e., the edge I thought I had based on the historical data did not continue going forward. No doubt my lack of proficiency was to blame, but several such experiences have made me more conservative about position sizing. I want a fudge factor to allow for the possibility that I might be completely wrong. At one point, I explicitly included a fudge factor in the Optimal f calculation itself, by for example adding the 22% drop in the S&P 500 on October 19, 1987 as an extra result if I was evaluating a one-day trade. Doing that sometimes turned the entire profit expectation negative, and the Optimal f calculation would quite sensibly not recommend any position size greater than zero.

I have also noticed that if I include stops as part of a trading strategy, the worst result is approximately bounded by the positioning of the stops. In such cases the Optimal f calculation sometimes seems to indicate I should use leverage. The Optimal f is between 0 and 1, but the position amount per contract is less than the dollar amount of one contract, so I would be trading on margin. If, for example, I have a worst loss of 2%, Optimal f may suggest an f\$ unit of 6%, which I could interpret as 3x leverage if I understand correctly.

Interesting post — but as for the bounding of f values, it turns out they are not between 0 and 1, but rather, between 0 and the sum of the probabilities of winning holding periods.

Thus (and this is where I am hoping Max and his good mind and find something new to augment what they are learning and work with in new ways I have never though of) the notion that everything is terrible sensitive to W isn't quite so (W, in fact, determines where the peak is in that window between 0 and the sum of the probabilities of winning holding periods.). Further, W is innately embedded in the Kelly Criterion calculation as well as you are working with returns (returns based on what? Well, traditionally, on the value of the underlying (times -1), or, in gambling W=-1. But, sensitivity to W isn't the biggest problem (in my humble opinion — I could be wrong, again, I throw this stuff out because I have been working with it). The biggest sensitivity is to:

the sum of the probabilities of winning holding periods (and, if holding periods are trades, assuming you are trading only 1 component, then it would be the percentage, say, of winning trades). That is what determines the upper bound.

But, in discussions with Larry Williams, because he wants weapons that won't jam when you pull them out, he wanted to know what a quick-n-dirty way to determine the Optimal f in the future would be (this really is a vital question, because what you make is predicated to where you are on the curve of f(0..1). And it turns out we can minimize the damage caused by missing the peak in the future (because we don't know where it will be) by using a value in the middle of the window, in other words, by using an f value of the sum of the probabilities of winning holding periods divided by 2.

And this is amazingly simple. And if you are right about the the sum of the probabilities of winning holding periods in the future, you have minimized the price you pay in the future by not being at the optimal f (and, you may well be at it. At worse, you will miss it by the sum of the probabilities of winning holding periods / 2 ).

And so the criticism of W in the equation really becomes not so problematic as what the the sum of the probabilities of winning holding periods will be. And if you can nail that, you've essentially predicted (as close as I can) what the optimal f in the future will be.

Rocky,

Ouch! I was just coming back here as I really don't feel right not answering your questions — so I'll try here now.

You wrote: "[regarding the application of optimal F]…If you want to be in this game and do it mathematically correctly, expect to be nailed for equity retracements of from 30% to 95%." You go on to write, "Most brokerage firms will not even look at a CTA who has historically had a 30% drawdown, nor will they promote a futures pool or fund that has had such a drawdown."

Pretty much, though I think risk tolerances, at least for commodity funds, has gotten better. Earlier in the decade many soverign wealth funds sat through drawdowns of around 50%…..and what happened after that, in energy, the dollar, precious metals, etc., is history.

1) I have never seen a market professional recover from a 95% drawdown (without raising new capital) or a individual recover (without putting fresh capital into his account).

I personally know of two individuals who have. The mathematics of achieving such a recovery are improbable. I disagree entirely. It may seem emotionally improbable, but certainly not mathematically.

If one should "expect" to be nailed for such an outcome, please explain why optimal F approach is more than just a theoretical/mathematical construct…i.e. it's too dangerous to be used in real life? (I ask this with the greatest of respect.)

I think it's too dangerous for anyone who is NOT seeking growth optimality. And growth optimality may NOT be the criteria most funds operate upon. In fact, it's not the criteria most individuals operate upon, although I must say I see some younger guys nowadays who, understand the mathematics and the swings involved, are implementing this. Again, it's only germane if growth optimality is the criteria. HOWEVER, the framework it provides allows us to seek OTHER criteria aside from growth optimality. I have met a couple of individuals who are using this to achieve other criteria with astonishing results.

That is, if one takes the Optimal F, and then cuts the position size back by XX% (as Steve below suggested), does that harm the intellectual integrity of its application?

I don't think it harms things, but to position oneself at a different point on the curve or N+ 1 dimensional surface (where N = number of components) simply to mitigate risk, and picks this point as a mere fraction of the optimal fraction, is doing themselves a disservice. There are other viable points, significant points to be at rather than the optimal point (again, in pursuit of different criteria) but notions like the so-called "Half Kelly," etc. are silly and based in not understanding the dynamics of the curve.2) If your answer to #1 is that there is nothing inherently wrong with a 95% draw down, what sort of long term (compounded) returns do you think a
manager should achieve if he is taking that sort of risk/volatility?

Let me say this. It depends on the individuals criteria. Now, some might argue that this is appropriate for young investors (perhaps, but first, in my humble opinion, only after loading up on whole life insurance with all they can) There is a lot more to the dynamics of this stuff, Rocky, than I can describe here (Frankly, I give 2-day courses in this stuff, and always fear that it is NOT enough time to cover all I want to cover!).

3) Can you explain how optimal-F addresses market volatility at the time a trade is entered; or changes in market volatility subsequent to the trade entry?

Is the largest expected loss, the worst-case scenario you envision over the time horizon you plan to trade, affected by the volatility? Volatility doesn't alter what the optimal fraction to wager is — it CAN alter, however, what that translates to in terms of position size.

`SELECT * FROM wp_comments WHERE comment_post_ID = '4658' AND comment_approved = '1' ORDER BY comment_date`