Information Flow: Transfer Entropy

In the last chapter you measured how two assets move together. Correlation, and its lead-lag cousins, told you that when one asset moves the other tends to follow. But togetherness is symmetric, and it is mute about direction. Correlation cannot tell you whether oil is driving the airline or the airline is driving oil, and it cannot tell you whether knowing one asset's past genuinely helps you predict the other's future, or whether they are simply two leaves blown by the same wind. This chapter gives you a tool that can speak to direction and to genuine predictive help: transfer entropy.

Start with the intuition, because the intuition is the whole point. Suppose you are trying to predict tomorrow's move in an asset — call it Y. You already have Y's own history, and that history carries information; markets have memory, momentum, mean-reversion, all of it. Now someone hands you a second asset's history, X. The honest question is not "are X and Y related?" It is sharper than that. It is: once I already know everything Y's own past can tell me, does adding X's past reduce my remaining uncertainty about Y's next move? If it does, then information is flowing from X to Y. If it does not — if X's past tells you nothing you could not already have squeezed out of Y itself — then there is no information transfer, no matter how correlated the two series look.

That word, uncertainty, is the bridge to the mathematics, because uncertainty has a precise measure. It is entropy. The entropy of a random variable is the average number of bits you need to pin down its outcome — high when the outcome is wildly unpredictable, zero when it is certain. And the natural way to ask "how much does knowing one thing reduce uncertainty about another" is to compare entropy before and after conditioning. That comparison is what transfer entropy formalizes.

Here is the definition, due to Thomas Schreiber in two thousand. Transfer entropy from X to Y is the difference between two conditional entropies. First, the uncertainty about Y's next value given Y's own past. Second, the uncertainty about Y's next value given both Y's past and X's past. Transfer entropy is the first minus the second: the amount of uncertainty that X's past removes that Y's own past had left on the table. Written compactly, TE(X→Y) = H(Y_future | Y_past) − H(Y_future | Y_past, X_past). Equivalently — and this framing is worth holding onto — transfer entropy is the conditional mutual information between Y's future and X's past, conditioned on Y's own past. The conditioning on Y's past is the load-bearing piece. It is exactly what subtracts out Y's self-prediction, so that whatever credit X receives is credit it could not have gotten by proxy through Y's own memory.

Two properties fall straight out of this, and both matter for how you will use the SVNexus reading. First, transfer entropy is directional. TE(X→Y) and TE(Y→X) are computed separately and are generally not equal. Oil informing the airline is a different number from the airline informing oil, and the asymmetry between them is often the most interesting thing on the screen — it is the closest thing you have to an arrow of influence. Second, transfer entropy is non-parametric and nonlinear. It assumes no linear model, no Gaussian shape; it is built on probability distributions estimated from the data itself. This is its great advantage over linear lead-lag and over Granger causality, which in its standard form is exactly transfer entropy's linear, Gaussian special case. Where a relationship lives in the tails, in the volatility, in a threshold effect — somewhere a correlation would miss it — transfer entropy can still see it.

That power comes at a price, and you must respect the price or you will fool yourself. Transfer entropy requires estimating distributions in several dimensions at once, which is hungry for data and sensitive to how you bin or quantize the values. With finite samples, the estimator is biased upward: pure noise will hand you a small positive transfer entropy simply because of estimation error. So a raw positive number is never, by itself, evidence of anything. The discipline — the thing that separates a real signal from an artifact — is to compare your measured transfer entropy against a null distribution built by destroying the very thing you are testing for. You shuffle or time-shift X to break its temporal alignment with Y, recompute transfer entropy many times, and ask whether your real reading sits out in the tail of that shuffled distribution. Only then does the number become evidence.