Can it be true that what is not understood is easier dismissed as being random. Are we confusing unexplained as being un-orderly?

Why do I ask? Assume, there is some day in the future a moment feasible in human consciousness wherein it would be possible to weed out every possible underlying function that could explain patterns in numbers.

That Eureka moment will be, say when the discovery of all possible numerical pattern explanatory functions will come by. Then, there will be some numerical sequences for which no formula could be discovered at such a Eureka moment also, will be there.

Now let us create two distinct planes. On each plane there will be two sub-sets each of the same type.

One type of sets on a plane called the plane of numbers, will contain sequences of numbers that can be explained by some function and the other set on this plane, the complement of the first set will contain all sequences of numbers that cannot be explained by ANY function.

On the other plane called the plane of functions, the corresponding sets will be the set of all functions that explain the sequences of non-random numbers and the complement set will be the set of functions that EITHER do not generate non-random numbers (that is functions that generate random numbers) or it will be a null set.

Given that there is a one to one mapping between the sets of patterned sequences with the set of pattern explaining functions on the other plane, the mapping should not collapse between the set of unpattern-able sequences and the complement set of pattern explaining functions. But it does, since the set of functions that explain non patterns is believed to be null.

By definition any functions that can generate random numbers is an absurdity. Any sequences that can be generated by a function are non-random! So the only complement on the plane of functions will be the null set.

Now for a moment, lets leave aside the path of induction and try to assemble a pathway to truth using deduction. Since the plane of functions is easier to understand, the complement set there being a null set means it will be possible at some point to have the set of functions that can generate all numerical patterns as the universal set!

Does this idea resemble our ability to explain things or does this idea explain our ability to explain that at the Eureka moment things can no longer be not explained?

Even if many will dismiss me as a clever lawyer who only asks questions without making any assertions, such a moment will extinguish all questions. The spirit of enquiry and therefore growth of human abilities will reach a void and a nullity. If such a moment can be reached there will be no difference left in chance and skill, since everything will merge into everything.

However, if I return back to the present moment from such an imaginary state, then it stands easier to grasp that what is unexplained so far, is what we are referring to as random. Why do we have to assert that without a proof of something being random it deserves to be given such an elevated omnipotent nomenclature as RANDOM?

My two cents on the table therefore. Abandon the search for Randomness as that pursuit is seeking a perfectionist definition of randomness. Stick to the simpler and workable idea that what is unexplained is not skill. Beyond the level of significance is skill and within its unexplained. I would like to place on the table a surmise, that the idea of randomness is definitional and thus a changing notion borne out of evolving cognition.

Gary Rogan writes: 

Imagine you have a small company that has a single facility housing a lot of important stuff. Some day an electrical discharge between two high-voltage wires at that facility either starts or doesn't start a fire that destroys a lot of irretrievable stuff, and the company either goes in decline or doesn't based on a single spark that depends on the rate of disintegration of the insulation between the wires, possibly some pests living in the walls, and humidity on that particular day. It's hard to come up with a reasonable explanation of how the sequence of stock prices of that company doesn't depend on randomness.

anonymous writes: 

You're right, people confuse trying to obtain a sufficiently random number with trying to obtain some mythical "100% random".

Randomness is proportional to the amount of information that someone does not have (informational entropy). Something "100% random", or "RANDOM" just means "I know 0%". It doesn't mean that it somehow occurs without any perturbation in energy, or position, or in some violation of the basic laws of physics. It certainly doesn't mean that someone else isn't saying "I know 65% with 99.9% probability" about the same situation (at which point that person would be learning relative to the "no nothing", and the "no nothing" would be relatively evolving, which implies that external forces in their environment have a greater influence over their outcome, all things being equal).

There are always some forces, initial conditions, or systems of equations, to partially explain (if you're lucky) why something moved from (a,b,c) to (d,e,f). The inexplicable changes in the portion of a system might be called chaotic, or it has high informational entropy, or random. To have exact predictions from perfectly separable equations (have all the information) only happens in models (to my knowledge). That is why everything is random to some degree, and that degree depends on our information.

Jeff Sasmor adds: 

To make things even more. Interesting.confusing, in my work I often need to obtain a random number from a reduced set of integers, say, from 0 to 4.

By way of an example, in some games you need a random value from a 'bag' of numbers in order to make a move or to set the value of a playing piece. 6-sided dice would be one example. Another example would be selecting which type of candy piece appears when playing the famously popular "Candy Crush Saga."

Although not widely known, that latter sort of game (generically called Match-3 games) often give more weight to certain piece types in order to make gameplay more difficult and induce you to pay for in-game items.

For example, you can make a histogram of which pieces already exist on the screen and bias the otherwise random selection process to make the game harder.

So when you need more blue candies to win a level the software sees how many blues are already on screen and produces fewer new ones.

That would undoubtedly be illegal for a casino video poker or video slots game but it's legal on your phone or tablet. At least so far.

Ralph Vince writes:

There's a huge market lesson here. In the vast majority of the cases, people are looking for deviations from randomness to find an edge.

The list's own Larry Williams has frequently spoken in recent years about the next trade having a 50/50 chance of being profitable, regardless of what your historical testing might indicate. My personal experience has been to assume randomness, assume that perverse arrangement of incidents and craft a strategy out of that expected sea of data (aside from the sadly gormless delta-neutral strategies now suffocating from the zirp-world-underwater-liability demands and blind-eyed-mandate-carve-outs). I've often found such seeming adges to be vaporous, ofteh the product of fleeting, temporal correlations (ghosts!).

However, the problem with assuming randomness and crafting strategies around that, the potential danger, is to stop looking for non-randomness, to stop looking for a potential edge to be obtained that way.

It's when we observe, distinguish and conclude that we learn (and most lessons hurt). In the world of perverted randomness, the drive-by, un-preened incidents in convertibles with their beer and their dope and their pornos in the trunk, slowly prowling the schoolyard's perimeter (within the legal distance as required by law and which they agreed to when they checked in with their local police station), failures are quite common but the most painful aspect is the moments of self reflection that appear in the passengers seat, stopping to inquire among us (whose crisp t-shirts now stained with the blood from our own noses), "So what did you learn from that failure?"

And the critical student will torment himself for years, looking for cause and effect that ultimately concluded in the particular failure — and come to see that if it wasn't that cause and that effect, it would have been another. Accepting the world as perversely random (yet, still looking for pattern, however fleeting, therein, as an exercise in art if nothing else, an aesthetic exertion for pleasure) does he come to realize the lessons of failure are not the random dominoes that fell and caused it, but the lesson (of survival)  of what to do ex post facto.

Sushil Kedia asks: 

Some more questions:

1. Would it not require a thorough and complete effort to "rule out" any possible pattern than to figure out a pattern?

2. To prove that nothing exists requires one to eliminate all possibilities. All possibilities is the universal set of total knowledge. If anyone is seeking a perfectly random number then for one to achieve that requires reaching state of total knowledge. Is "perfectly imperfect" possible?

3. Does the spirit of inquiry rest on the shoulders of humility?

John Bollinger writes: 

From my perspective, you'd be better off thinking about volatility.

anonymous writes: 

Ralph's comment about this "world of perverted randomness" convinces me that we may not all be Keynesians now, but almost all of us are tempted to indulge in the national vice of moralizing even on this subject of randomness. How else can we understand the continued success of preaching in the field of economics? Where would Paul Krugman and most of his fellow Nobel Laureates be without their pulpits?

No wonder Armen Alchian never came close to winning the prize. He lacked the necessary techniques for railing against uncertainty and time. Still worse, he remained blissfully untroubled that these phenomena which are the facts of the cosmos continue to escape both summary definition and perfect abstraction.

anonymous adds:

Sushil, I don't think it's true that the one-to-one mapping you want exists.

A pattern can be defined as a function from natural numbers to natural numbers, for example 4,123,19716,9,82,92,1,5. The number of such functions is provably greater than the number of natural numbers. Just use the proof that the reals are bigger than the naturals and recognise that in a large enough base a given real number is a map from natural numbers to (bounded natural numbers, but as big as you like).

Randomness is opposite to predictability. But as in Ellsberg's paradox, there's a crucial difference between an unknown distribution and a known uniform distribution. By "True randomness" people usually mean a totally uniform distribution with no patterns of any kind. But even a "stochastic" function like a drift process, can be considered random enough that you couldn't predict it effectively (at least not enough to make money).

Mathematicians have already been working with your concept of "all possible functions" for a long time. It's not a future Eureka moment. But with securities prices, we don't actually want to talk about any possible price pattern, for example no stocks multiply in value 100x overnight (or at least a negligible number do). And it's possible but relatively more rare for thick-volume stocks to plunge to zero overnight (although they can move faster than a drift process).

Mathematicians solve the problems you're talking about with the concept of measure. It is possible that when I flip the coin heads or tails, an eagle flies over me and snatches it out of my hand. But this is considered to be effectively 0% likely ("almost never" in technical language).





Speak your mind


Resources & Links