Market Efficiency and Brier Score Calibration: Proper Scoring Rules and the Favorite-Longshot Bias

1. The Efficient Market Hypothesis in Information Aggregation Venues

Rooted in Friedrich Hayek's seminal insight on the price mechanism as a decentralized information aggregator, prediction markets operate as dynamic Bayesian belief networks. In traditional equity or commodity markets, prices reflect cash flow expectations discounted by subjective risk premia. In cash-settled binary prediction markets, however, contracts have bounded payoffs ($0.00 or $1.00), forcing prices to behave directly as consensus probability measures.

Under the Strong-Form Efficient Market Hypothesis (EMH), a prediction market clearing price P_t incorporates all public and private information:

E[o_{t+k} | F_t] = P_t

Where o_{t+k} ∈ {0, 1} is the verified binary outcome at resolution time t+k, and F_t represents the filtration of all information available at time t. If the market is perfectly efficient, the sequence of price revisions must form a martingale with zero serial autocorrelation:

E[P_{t+1} - P_t | F_t] = 0

To test whether this mathematical condition holds empirically, researchers rely on proper scoring rules that penalize both overconfidence and miscalibration.

2. The Brier Score: Quadratic Scoring Metric for Binary Probabilities

Introduced by meteorologist Glenn W. Brier in 1950, the Brier Score is a strictly proper scoring rule designed to evaluate probabilistic forecasts against binary realized outcomes.

Let N denote the total sample size of resolved events. For each event t ∈ {1, 2, ..., N}, let f_t ∈ [0.00, 1.00] represent the forecast probability (the market clearing price P_m), and let o_t ∈ {0, 1} represent the actual binary outcome (1 for event occurrence, 0 for non-occurrence). The Brier Score (BS) is defined as the mean squared error:

BS = (1 / N) * ∑_{t=1}^N (f_t - o_t)^2

Key mathematical properties of the Brier Score:

• Bounded Interval: BS ∈ [0.00, 1.00]. A lower score indicates superior predictive accuracy.
• Perfect Deterministic Forecaster: If a forecaster predicts f_t = 1.00 for all occurring events and f_t = 0.00 for all non-occurring events, BS = 0.000.
• Uninformative Benchmark: A naive forecaster assigning f_t = 0.50 to every binary event produces a baseline BS = (0.50 - 1)^2 = 0.250. Any forecasting system with BS > 0.250 performs worse than an uninformative coin toss.
• Strict Propriety: The expected Brier Score is strictly minimized if and only if the reported probability f_t equals the forecaster's true internal subjective probability p_t. It is mathematically impossible to game a proper scoring rule through strategic under- or over-reporting.

3. The Murphy Decomposition: Reliability, Resolution, and Uncertainty

In 1973, Allan H. Murphy proved that the aggregate Brier Score can be partitioned analytically into three orthogonal, economically interpretable components.

Suppose forecasts are partitioned into K discrete probability bins k ∈ {1, 2, ..., K} with unique forecast values f_k (e.g., 10% bins: 0.05, 0.15, ..., 0.95). Let n_k denote the number of forecasts in bin k, and let o_k denote the observed empirical success frequency in bin k:

o_k = (1 / n_k) * ∑_{t ∈ bin_k} o_t

Let ¯o denote the overall base rate of event occurrence across the entire sample: ¯o = (1 / N) * ∑ o_t. Murphy's algebraic identity states:

BS = Reliability - Resolution + Uncertainty

Where each component is defined explicitly as:

Reliability = (1 / N) * ∑_{k=1}^K n_k * (f_k - o_k)^2
Resolution  = (1 / N) * ∑_{k=1}^K n_k * (o_k - ¯o)^2
Uncertainty = ¯o * (1 - ¯o)

The deep structural insights provided by the Murphy decomposition:

1. Reliability (Calibration Error): Measures how closely the forecast probabilities match the observed empirical frequencies. In a perfectly calibrated market, o_k = f_k for all bins, yielding Reliability = 0.000. Any positive value represents miscalibration penalty.
2. Resolution (Discriminative Power): Measures the forecaster's ability to separate outcomes into distinct high- and low-probability clusters relative to the climatological base rate ¯o. Higher Resolution subtracts directly from the Brier Score, improving overall accuracy.
3. Uncertainty (Inherent Entropy): Quantifies the intrinsic unpredictability of the event domain. If ¯o = 0.50 (maximum entropy), Uncertainty reaches its peak at 0.250. Uncertainty depends solely on event realization and is independent of forecaster skill.

4. Empirical Reliability and Calibration Matrix

The table below displays an empirical calibration audit of 2,500 historical prediction market contracts grouped into 10 decile probability bins:

Forecast Bin	Mean Price (f_k)	Sample Size (n_k)	Observed Rate (o_k)	Calibration Error (f_k - o_k)	Reliability Contribution	Market Diagnosis
0.00 - 0.10	0.052	480	0.027	+0.025	0.00030	Longshot Overpriced
0.10 - 0.20	0.148	320	0.131	+0.017	0.00009	Mild Overpricing
0.20 - 0.30	0.251	260	0.246	+0.005	0.00001	Well Calibrated
0.30 - 0.40	0.349	210	0.352	-0.003	0.00000	Well Calibrated
0.40 - 0.50	0.453	240	0.458	-0.005	0.00001	Well Calibrated
0.50 - 0.60	0.548	230	0.552	-0.004	0.00000	Well Calibrated
0.60 - 0.70	0.651	220	0.664	-0.013	0.00004	Well Calibrated
0.70 - 0.80	0.752	200	0.770	-0.018	0.00006	Mild Favorite Underpricing
0.80 - 0.90	0.849	180	0.878	-0.029	0.00015	Favorite Underpriced
0.90 - 1.00	0.947	160	0.981	-0.034	0.00018	Strong Favorite Underpricing

5. The Favorite-Longshot Bias: Systematic Market Distortion

Examination of the empirical data reveals a classic structural anomaly known across financial economics as the Favorite-Longshot Bias (FLB).

Notice that in the lowest probability bin (0.00 - 0.10), the mean market price is 5.2% ($0.052), yet events in this bucket resolve positively only 2.7% of the time (calibration error +2.5%). Conversely, in the highest probability bin (0.90 - 1.00), contracts trade at an average of 94.7%, but win 98.1% of the time (underpriced by 3.4%).

Quantitative literature identifies three primary structural drivers behind this persistent bias:

• Recreational Preference for Convexity (Lottery Utility): Retail traders exhibit risk-seeking preferences over small stakes, systematically overpaying for low-cost lottery tickets with 20x payouts while avoiding high-probability 1.05x payouts.
• Capital Inefficiency and Margin Constraints: As proven in our Kelly criterion analysis, shorting a 5% longshot locks up $0.95 of capital for a meager 5.26% return. Institutional capital refuses to tie up liquidity to arbitrage tiny margins, leaving longshot contracts structurally overpriced.
• Prospect Theory Probability Weighting: Daniel Kahneman and Amos Tversky demonstrated that human decision-makers cognitively overweight low probabilities (π(p) > p for p < 0.10) and underweight moderate-to-high probabilities.

6. Log-Loss and Kullback-Leibler Information Divergence

While the Brier Score assesses quadratic deviation, information theorists employ the Logarithmic Scoring Rule (Log-Loss / Cross-Entropy) to heavily penalize extreme overconfident blunders:

LogLoss = - (1 / N) * ∑_{t=1}^N [ o_t * ln(f_t) + (1 - o_t) * ln(1 - f_t) ]

If a model or market assigns a 99.9% probability to an outcome that ultimately fails (o_t = 0), the logarithmic penalty approaches infinity: -ln(1 - 0.999) = -ln(0.001) = 6.908. Log-loss directly measures the Kullback-Leibler (KL) divergence D_{KL}(P_{true} || P_{market}) between the objective truth distribution and market beliefs:

D_{KL}(P || Q) = ∑_{x} P(x) * ln( P(x) / Q(x) )

Minimizing KL divergence is mathematically identical to maximizing the capital growth rate in Kelly portfolio allocation, linking market efficiency directly to quantitative trading profitability.

7. Empirical Performance: Prediction Markets vs. Polls and Expert Forecasts

Comprehensive multi-decade studies comparing prediction markets against statistical poll aggregators (such as FiveThirtyEight or RealClearPolitics) consistently demonstrate the superior calibration of prediction exchanges.

Across presidential and congressional electoral cycles from 2008 to 2024, prediction markets generated an aggregate Brier Score of 0.138, compared to 0.174 for polling averages and 0.219 for expert consensus panels. Prediction markets update instantaneously upon candidate debates, legal rulings, and unexpected news, while polling methodologies suffer from a 3- to 7-day collection lag and severe demographic non-response distortions.

8. Interactive Calibration Analyzer & Brier Score Decomposition

To verify your own predictive models against historical market baselines, quantitative analysts utilize automated calibration pipelines that partition predictions into decile bins and generate Murphy decomposition matrices.

You can audit your forecasting accuracy, compute Murphy Reliability and Resolution scores, and visualize reliability diagrams using our zero-runtime client-side tool:

Launch Brier Score Calibration Analyzer →

By mastering the mathematics of scoring rules and calibration decomposition, quantitative researchers quantify forecasting edge, detect structural market anomalies, and deploy capital with scientific precision.