Coming back from behind

Alex Castaldo writes:

Heres the skinny. from math puzzles volume 1, by  presh talwalkar. doc here. from nature walk. originally to stretch aubrey's mind . odds of a comebak victory

Consider 2 teams a and b that are completely evenly matched. given that a team is behind in score at half time, what is the prob that a team will overcome the deficit and win the game. assume the first halve and the second half are taken to be independent events. Presh solves it as follows logically:

Since the two teams are evenly matched, it is equally likely that the team will score enuf points to overcome the deficit or that it will not score enuf points. fo example the event of falling behind 6 pts in a half game happens with the same prob as gaining 6 pts in a half game. He concludes prob is 0.25

Now we posted the empirical resutls from basektaball games and many others have given the empiriclal results for football games … and i gave some results for the markets.. this seems to be of interest to everyone , had the most views of any posts, and it was good for 7 or 8 points today.. lets have your discussion and solution of this problem. presh says the answer is 0.25 both empirically (NFL in 1995) and logically.

Jared Albert writes:

In a game with two teams where in the first round, the team 1 advantage varies from flat to all the points available in the second round, the probability of  team 0 coming from behind to win are in array with 20 available points in the second round:

[0.49, 0.306, 0.22, 0.129, 0.09, 0.03, 0.018, 0.011, 0.004, 0.002, 0.002, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

For example, if the teams are even going into the second round with 20 available points, .490 chance that team0 wins; with a one point advantage to team1 at the start of round2, team0 wins .306 of the time;

2 points to team1, team0 wins .220 of the time etc

Here's the montecarlo:

import numpy as np

np.random.seed(10)

out_list = []

out_list = []

count = 1000

win = 1

lose = 0

team0_start = 0

team1_start = 0

size=20

def runs():

z = np.sum(np.random.choice([win, lose], size=size, replace=True, p=None))

return z

def outcome(team1_start, count = count, team0_start=team0_start):

l= []

for _ in range(count):

team0_end = runs() + team0_start

team1_end = runs() + team1_start

came_from_behind = team0_end > team1_end

l.append(came_from_behind)

#print(f'l: {l}')

outcome = sum(i > 0 for i in l)

return(outcome)

for i in range(size):

out_list.append(outcome(team1_start=i)/count)

print(f'outlist: {out_list}')

Victor Niederhoffer writes:

up your alley i  think. we have done something similar for market with real empirical results. the  unconditional prob is much less than20%

Stephen Stigler writes:

I am sure you know but I repeat anyway:

1) the simple calculations ignore correlation between teams.

2) they also ignore information on the distribution of changes

3) Calculations using the distribution of changes are not hard.

4) But the information about the probability of extreme events is not well determined so they can be inaccurate

5) In any case  markets unlike sports are not zero sum games.

Study: Does the Broad Market Rise Unusually into IPOs and Then Sell Off After, from Jared Albert

Motivated by the Saudi Aramco IPO, this study tries to answer the question: does the broad market rise into big IPO's (and then sell off after)? This is based on the theory that there is an effort to boost the market before the IPO to benefit the new issue ecosystem.

I'm surprised, but there doesn't seem to be an effect as the chart shows the usual upward drift. I took the 25 biggest US IPOs by proceeds and graphed the mean of LN changes of the SPY to the IPO date of the 20 trading days before and the 20 trading after.

LN changes 20 days before to IPO date

count    25.000000

mean     -0.009702

std       0.051080

min      -0.125424

25%      -0.029615

50%      -0.013882

75%       0.020232

max       0.075516

LN changes from IPO date to 20 days after

count    25.000000

mean      0.004976

std       0.032097

min      -0.069427

25%      -0.008863

50%       0.001278

75%       0.024553

max       0.065406

IPO data is from here

SPY date is from here

script is here.

Multiple Comparison Fail, from Victor Niederhoffer

A case study in multiple comparisons and a warning against using cart for market prediction:

"Exercising for 90 Minutes Or More Could Make Mental Health Worse, Study Suggests"
by Sarah Knapton, Science Editor

Steve Ellison writes:

A statement by Mark Hulbert in Sunday's Wall Street Journal raised my suspicions. He said that the percentage of household financial assets invested in stocks had an R-squared of 61% since 1954 in forecasting the net change of the S&P 500 over the next 10 years.

There have only been 6 non-overlapping 10-year periods since 1954. I have not gotten around to getting the data for household financial assets, but how could any factor possibly have an R-squared of 61% with any significance after 6 observations?

I will grant that the indicator makes some intuitive sense from the perspectives of "copper[ing] the public play" and waiting to buy until the old men are hobbling on canes, but I question the statistics.

The most accurate of the indicators I studied was created by the anonymous author of the blog Philosophical Economics. It is now as bearish as it was right before the 2008 financial crisis, projecting an inflation-adjusted S&P 500 total return of just 0.8 percentage point above inflation. Ten-year Treasurys can promise you that return with far less risk.
Bubble flashbacks
The only other time it was more bearish (during the period since 1951 for which data are available) was at the top of the internet-stock bubble.
The blog’s indicator is based on the percentage of household financial assets—stocks, bonds and cash—that is allocated to stocks. This proportion tends to be highest at market tops and lowest at market bottoms.
According to data collected by Ned Davis Research from the Federal Reserve, this percentage currently looks to be at 56.3%, more than 10 percentage points higher than its historical average of 45.3%. At the top of the bull market in 2007, it stood at 56.8%.
Ned Davis, the eponymous founder of Ned Davis Research, calls the indicator’s record “remarkable.” I can confirm that its record is superior to seven other well-known valuation indicators analyzed by my firm, Hulbert Ratings.
To figure out how accurate an indicator has been, we calculated a statistic known as the R-squared, which ranges from 0% to 100% and measures the degree to which one data series explains or predicts another.
In this case, zero means that the indicator has no meaningful ability to predict the stock market’s returns after inflation over the next 10 years. On the other hand, a reading of 100% would mean that the indicator is a perfect predictor.
Since 1954, according to our analysis, the Philosophical Economics indicator had an R-squared of 61%. In the messy world of stock-market prognostication, that is statistically significant. Our analysis begins in that year because that is the earliest date for which data are available for all of the other indicators that we studied.

Jared Albert writes:

As I understand the statement, the R**2 is generated from the correlation between the end of one ten year period and the end of the other.

Is this a fair model:
1) Use the annual returns for the SP500 for the period 1954-2014 broken in the 6 decade buckets.
2) Use the standard deviation of returns for each of those 10 years periods (STD calculated on only 10 yearly values for simplicity).
3) Generate a random return value from a normal distribution for the end year of each period
4) repeat the above for cash and bonds
5) create the portfolio ratio of stocks:bonds:cash
6) calculate the r**2 value between every 10 year period for stocks
7) do this 1000 times and calculate the summary stats for the R**2

Is this the way to build the model? I may do this later, if I can quickly find the cash and bond return. Thank you,

What it Takes to Make a Profit, from Richard Owen

"My share of the taxes, yes," said the Visitor, "Piketty sent me."

"Are you sure you want to tax capital? I mean, really sure?" said the Keeper.

"It's only fair," said the Visitor.

"Well, to register and receive you must put on this headset," said the Keeper, handing over a kind of halo object, "it will read your Identity Number, calculate your distribution and begin making a fair deposit."

"Perfect!" said the Visitor, and popped the contraption onto his head. The Keeper stared at him directly, a thin smile on his lips.

The Visitor pressed the power button on the halo. "Aaaah! No, please. What." The Visitor spasmed wildly. "Aaargh! Oh my God! Please, please." The Visitor's flight reflex kicked in, his muscles began to shake violently, bringing him to his knees. The tension in his bladder collapsed and piss soaked his pants. The Visitor writhed on the floor, "MAKE IT STOP! What is this?!"

The Keeper quickly pulled a handset from his pocket and clicked the interrupt. Nobody so far had completed the deposit in full. The Visitor fell to the floor, exhausted. With his eyes blood shot, watering, the Visitor cried out, "how dare you, what was that torture? You fiend! This is criminal."

"You asked for your share," said the Keeper, "and your bank account is in credit now. Your share of the capital taxes have been delivered, proportionately."

"Are you some kind of SICKO?" screamed the Visitor.

"No. You see, you asked for your fair share. We decided in transferring capital taxes, we should also make an additional deposit to keep it balanced. We gave you a concentrated dose of every sleepless night, strained relationship, cheating business partner, every lie heard, every deal that didn't close, every set-back, every busted asset, every temptation skirted, idea stolen, regulatory intervention, bankrupt supplier, every loss adjusted insurance policy, every giant competitor… all of it. And there's much, much more. Should I complete the deposit?" asked the Keeper.

The Visitor staggered up to his feet, raised his eyes to the Keeper and paused to speak. But nothing came. Instead, he ran straight for the door.

Jared Albert writes:

I think the basic problem with Piketty style wealth redistribution is that everyone wants to read poetry, while no one wants to take out the trash.

That effort is often necessary for wealth, doesn't answer his basic point that in a fairer world we'd help those who strove and failed as well.

Victor Niederhoffer writes:

Yes, Mr. Albert has encapped the idea that has the world in its grip. When I played ball, I always wished that my opponents would share their points when they beat me. There should have been a law.

Jared Albert replies:

A lot of effort has gone down dead ends in battery technology. Those efforts uncovered what doesn't work, and provided methods that may end up pushing some methods forward. Those failures benefit all of us.

According to an ideal Piketty model, the losers should be compensated in some form by winners as they helped move the sum of the effort forward.

I don't know for sure obviously, but I doubt you can find a nobel laureate who doesn't feel that they stood on the shoulders of others.

My point is that in general people are dis-incetized to try any of the routes if their reward has nothing to do with effort.

Stefan Jovanovich writes:

In a fairer world we do help those who strive and fail; that is how successful teams (right now and for the past 5 seasons under Bruce Bochy, the SF Giants) and families (the anonymous R-Man's to take one of many examples from the List) and enterprises all work. As with most Leftist ideas Piketty has a valid complaint; as with all ideas based on the sacrifice of individual freedoms for collective good his Marxist solution is catastrophically bad. Some people do want to take out the trash rather than let it pile up, but no one does it for very long for the sake of strangers without getting paid in money that he or she gets to keep and spend. That is why inventive and naturally poetic people in Cuba live in a world of uncollected trash and free medical care where the patients bring the medicines to the doctors. But it is fair — everyone lives under the same collective incentive to read official poetry.