Time Series: Stationarity, AR & MA

Up to now you have treated your observations as a bag of independent draws — each return sampled fresh, yesterday telling you nothing about today. That assumption made the last three chapters' regression machinery run cleanly. It is also, for almost every financial series you will touch, false. A time series is a sequence where order matters: today's value is entangled with yesterday's, last week's, sometimes last month's. The whole job of time-series analysis is to measure that entanglement honestly — to separate the part of today that was genuinely predictable from the part that was new. It is the same question the institute keeps circling: when is a number real evidence, and when is it just the past echoing forward and fooling you?

Stationarity, the foundation

Before you can measure persistence, you need a foundation stable enough to stand on: stationarity. A series is weakly stationary when three things hold steady over time:

$\mathbb{E}[Y_t] = \mu, \qquad \mathrm{Var}(Y_t) = \gamma_0, \qquad \mathrm{Cov}(Y_t, Y_{t-k}) = \gamma_k \ \text{(depends only on the lag } k\text{)}.$

The mean doesn't drift, the variance doesn't balloon, and the relationship between a value and one a fixed distance behind depends only on that distance — not on where in the calendar you stand. A stationary series has no preferred moment; a January window looks statistically like an August one. This matters because nearly every estimator you own — a mean, a variance, a correlation — silently assumes the thing you're averaging keeps its identity across the sample. Average a series whose mean is wandering and you get a number that describes nowhere.

Raw prices are the classic offender, and now we can say precisely why. A price behaves like a random walk, $P_t = P_{t-1} + \varepsilon_t$ — and that is exactly an AR(1) with coefficient one, a unit root. With the coefficient at one, shocks never fade; they accumulate forever, so the mean wanders and the variance grows without bound. (This is the same random walk from Tier 100 ch4, now named formally.) The augmented Dickey–Fuller test checks for that unit root, and the fix is differencing: subtract consecutive prices and you get returns, which are close to stationary. That is not an aesthetic choice — it is what makes the rest of this lesson legitimate.

The autocorrelation function

To measure how a series remembers its own past, compute its autocorrelation. Lay a copy of the series beside itself shifted by $k$ steps, correlate the two columns, and you have the autocorrelation at lag $k$:

$\rho_k = \frac{\gamma_k}{\gamma_0} = \frac{\mathrm{Cov}(Y_t, Y_{t-k})}{\mathrm{Var}(Y_t)}.$

The division normalizes every reading to $[-1, 1]$ so you can compare lags fairly. Do it for every lag and you trace the autocorrelation function (ACF). But a sample ACF is never exactly zero even on pure noise, so the guardrail is a band of roughly $\pm 2/\sqrt{T}$: a bar inside it is noise in a costume; a bar poking clearly outside is genuine memory. Same discipline as a regression t-statistic, transplanted into time.

Two engines: AR and MA

With the ACF in hand, name the two elementary engines. The autoregressive process (AR) says today is a fraction of its own past plus a fresh shock:

$Y_t = \phi_0 + \phi_1 Y_{t-1} + \cdots + \phi_p Y_{t-p} + \varepsilon_t \quad (\text{order } p).$

For the simplest, AR(1), $Y_t = \phi Y_{t-1} + \varepsilon_t$ is stationary if and only if $|\phi| < 1$ — and then it is mean-reverting: after a shock, the series is pulled gradually back toward its mean, with a shock's half-life of $\ln(0.5)/\ln(\phi)$. That is the formal engine behind ch1's "regression toward the mean," and it is why an extreme volatility reading tends to revert (the squeeze you met in 201). Push $\phi$ to 1 and reversion vanishes — the random walk again. The AR fingerprint is an ACF that tails off smoothly, because yesterday already contained a piece of the day before, so any shock decays gradually.

The moving-average process (MA) builds today out of recent shocks:

$Y_t = \varepsilon_t + \theta_1\varepsilon_{t-1} + \cdots + \theta_q\varepsilon_{t-q} \quad (\text{order } q).$

Here memory is finite and blunt: an MA(1) knows exactly one step back and nothing beyond, so its ACF cuts off sharply — a spike, then silence. (A word of caution: this MA process is not the moving-average smoother of 102 ch5. There, a moving average was a filter you slid over the observed series to smooth it; here, it is a model that builds the series out of unobserved past shocks. Same two words, entirely different object — context tells you which.) Combine the two and you get an ARMA model.

The identification mirror: ACF and PACF

To tell AR from MA cleanly, bring in the ACF's partner, the partial autocorrelation function (PACF), which measures the lag-$k$ correlation after stripping out everything the shorter lags already explained. The classic signature: an AR process cuts off in the PACF and tails off in the ACF; an MA process does the reverse. That mirror is the workhorse of model identification (Box–Jenkins).

Long memory and the HAR model

One more fact, and it is where volatility forces our hand. The ACF of realized volatility does not tail off in a handful of lags like a well-behaved AR(1) — it decays slowly, over dozens of lags, a property called long memory. Volatility is sticky: today's reading remembers weeks of its own past (the clustering you first saw in Tier 100 ch2, now measured). A single AR(1) can't capture that — its memory fades too fast — and fitting a 22-lag AR means 22 noisy coefficients.

The elegant shortcut is the HAR model (Heterogeneous AutoRegressive). Instead of 22 separate lags, pool them into three economically meaningful bands — the last day, the last week (5-day average), and the last month (22-day average):

$\mathrm{RV}_{t+1} = \beta_0 + \beta_D\,\mathrm{RV}^{(d)}_{t} + \beta_W\,\mathrm{RV}^{(w)}_{t} + \beta_M\,\mathrm{RV}^{(m)}_{t} + \varepsilon_{t+1}.$

Three coefficients capture the long memory that would otherwise take twenty. The name's "heterogeneous" is the reason it works: markets are made of participants acting on different horizons — day traders, swing traders on weekly rhythms, institutions on monthly ones — and each imprints its own layer of persistence. HAR gives each layer its own coefficient.