150 Years of Market Regimes: What a Hidden Markov Model Sees in the S&P (1871–2026)
Most long-horizon retirement models treat equity returns as one stationary process: a single expected return, a single volatility, independent monthly draws. The historical record looks different. Markets spend long stretches in a low-volatility climb, interrupted by shorter episodes where volatility multiplies and returns turn sharply negative — and those episodes cluster. This post documents what a standard regime-switching model finds when you give it the longest monthly U.S. equity series available, and how we turned that fit into a live, monthly-updated regime monitor and a simulation preset.
Everything below comes from one reproducible pipeline: Gaussian hidden Markov models (HMMs) fitted to 1,865 months of real (inflation-adjusted) S&P composite total returns, February 1871 through June 2026, built on the Robert Shiller long-run dataset. The fitted parameters, episode lists, and the current regime probabilities are published on our Market Regime Monitor, which refreshes monthly.
The two-state picture: calm and stressed
Run your own numbers — FREE
10,000 Monte Carlo simulations. Forward-looking forecasts from BlackRock, JPMorgan, Vanguard, GMO, Schwab, Invesco. No account needed.
Try QuantCalc Free →The simplest regime model splits history into two states. The fit is unambiguous about what they look like:
| State | Annualized real return | Annualized volatility | Share of months | Typical duration |
|---|---|---|---|---|
| Calm | +12.5% | 9.8% | 83% | ~32 months |
| Stressed | −20.9% | 25.0% | 17% | ~7 months |
Two features matter for planning:
- The volatility ratio is 2.55× (25.0% vs 9.8%). Stressed months are not slightly worse — they are a different distribution.
- Transitions are asymmetric. In any calm month, the chance of slipping into the stressed state is about 3.1%; in any stressed month, the chance of escaping back to calm is about 15.1%. That asymmetry is why calm stretches run for years while stressed episodes usually resolve within two or three quarters — but with a fat tail of multi-year exceptions.
Model selection supports the added complexity: against a single-regime baseline, the two-state model improves out-of-sample predictive log-likelihood per observation from 1.883 to 1.996, evaluated on 438 months held out after a 1990 training cutoff. The state definitions are also stable: refitting on expanding windows ending in the 1950s through today reproduces the same calm/stressed split, with ~94–95% agreement on which months are which.
Every stressed episode since 1871
The two-state model identifies 22 stressed episodes in 155 years. The full list is in the monitor's published dataset; the notable ones:
| Episode | Duration |
|---|---|
| Oct 1929 – Sep 1934 | 60 months |
| Apr 1937 – Apr 1939 | 25 months |
| Mar 2001 – Feb 2003 | 24 months |
| Nov 2007 – Apr 2009 | 18 months |
| Nov 1973 – Feb 1975 | 16 months |
| Sep 1981 – Oct 1982 | 14 months |
| May 2022 – Oct 2022 | 6 months |
The median episode is short — about six months — but the distribution is heavily right-skewed. The Depression-era episode ran five years. A planning model that assumes drawdowns resolve on a set schedule misses exactly the cases that break retirement plans.
Three states: separating corrections from crises
Adding a third state splits "stressed" into two qualitatively different conditions:
| State | Annualized real return | Annualized volatility | Share of months |
|---|---|---|---|
| Calm bull | +20.6% | 8.7% | 64% |
| Correction | −17.9% | 11.9% | 28% |
| Crisis | −16.0% | 32.7% | 8% |
The interesting result is that corrections and crises have similar average returns but volatility levels that differ by almost 3×. What distinguishes a crisis is not deeper average losses per month — it is the violence of the swings, in both directions. October 1987, October 2008, and the 1929–33 core all land in the crisis state; garden-variety pullbacks land in the correction state. BIC prefers the three-state model; the out-of-sample gain over two states is small (2.012 vs 1.996 per observation), which is why we publish both and treat K=2 as the workhorse.
Where the model places June 2026
As of the June 2026 refresh, the two-state model reads the market as 96.0% calm / 4.0% stressed. The three-state model agrees: 90.6% calm bull, 8.1% correction, 1.4% crisis.
A necessary caution on what this is: a nowcast, not a forecast. The model estimates which regime current returns are most consistent with. The forward arithmetic is conditional and converges fast: starting from today's probabilities, the chance of being in the stressed state is about 6% one month out, about 16% twelve months out, and settles at the long-run 17% — the unconditional base rate. The regime model's value is not telling you when the next episode starts. It is telling you what episodes look like when they arrive: 2.55× volatility, negative drift, and months of persistence rather than independent bad draws.
Parameter uncertainty, stated plainly
Fitting 150 years of data does not make every number precise. Stationary-bootstrap resampling (50 replicates) puts the stressed-state mean at −22.1% with a standard deviation of 6.7 percentage points — the 90% interval spans roughly −32% to −11%. The calm-state mean is much tighter: 12.4% ± 0.8. Volatilities and transition probabilities are tighter still. The qualitative structure — a persistent calm state and a sharply more volatile stressed state with a 2.5–2.6× vol ratio — survives every robustness check we ran; the exact stressed-state drift is the softest number.
From research to simulation input
The reason we built this is not market commentary. It is that the regime structure changes what retirement simulations say about risk — particularly sequence-of-returns risk, which lives in exactly the clustered-drawdown behavior a single-regime model averages away.
The fitted parameters now ship as a one-click preset in QuantCalc's regime-switching panel: transition probabilities 0.031/0.151 per month, a 2.55× stressed-state volatility multiplier, and the fitted bull–bear drift spread, re-centered so that enabling the preset does not change your portfolio's unconditional expected return — it changes where the bad months land. The mechanics, including how the drift spread is de-meaned against the chain's stationary distribution and how the volatility multiplier is variance-compensated, are documented in our methodology. The companion post quantifies what the preset does to success rates on two reference plans: What Regime-Aware Monte Carlo Does to a Retirement Plan.
For ongoing readings, the Market Regime Monitor republishes the nowcast, the conditional outlook curve, and the full episode list after each monthly data update — refreshes are gated on parameter-stability checks, so a degenerate fit never silently replaces a good one.
Data: Robert Shiller's long-run U.S. dataset (monthly average of daily closes, dividends reinvested, CPI-deflated). Models: Gaussian HMMs fitted by EM with multiple restarts; model selection by BIC and out-of-sample predictive log-likelihood with a 1990 train/test cutoff. All figures in this post are produced by the published pipeline and are regenerated monthly.