# FOOTO PREDICTO 3.24.7 — Probability calibration audit

**Cutoff: October 29, 2026.** This is a probability postprocessor, not a scoring-engine or playcalling adjustment.

## Data provenance and held-out testing

* Training: 272 archived chronologically pregame score forecasts from 2024; learned sigmoid(home margin × 0.148713 -0.146128).
* 2025 independent test: 272 pregame release forecasts, evaluated with **2024-only** parameters. Binary Brier 0.2283; binary log loss 0.6502; winner-pick accuracy 62.1%. Ties scored half for this historical logistic fit; these two-class metrics are *not directly comparable* with the three-outcome scores below.
* Final deployed score-to-win anchor fit on 2024 **plus** 2025, 544 historical forecasts: sigmoid(0.12581754 × score margin -0.08430259).
* 2026 test: 59 distinct, legitimate pregame saved forecasts from Weeks 4–7. Other older duplicates are ignored and the missing official October 26 Dallas–Philadelphia 38–21 result is corrected. None of these 2026 outcomes were used to fit coefficients. Historical forecasts span previous model versions **3.6–3.23**; current 3.24.5 has no completed Week 8 games yet.

## Test results (2026, three outcomes: home, away, tie)

| Metric (lower is better) | Raw simulated probabilities | 50/50 historical-anchor blend | Change |
|---|---:|---:|---:|
| Multiclass Brier | 0.4839 | 0.4788 | 0.0051 improvement |
| Multiclass log loss | 0.7308 | 0.7260 | 0.0048 improvement |
| Top-probability winner pick accuracy | 57.6% | 59.3% | Not the calibration objective |

The bootstrap 95% interval for **Brier improvement** is **+0.0006 to +0.0099**, and for log-loss improvement is **-0.0003 to +0.0102**. The Brier interval is positive on this retrospective sample, but the log-loss interval includes zero. The choice of an ensemble was evaluated using this same small 2026 sample, so this is **not independent prospective proof** of improvement for the latest model. The season-to-season validation is more useful than a fitted adjustment on the same 2026 predictions, but it does not resolve this small-sample uncertainty.

## Policy and safeguards

1. Use 50% raw simulated win probability and 50% margin-to-win historical anchor. Leave simulated tie probability untouched. Normalize home+away to 1−tie. The anchor uses the **simulated mean scoring margin**, not the baseline model's expected margin.
2. Historical Week 4–7 win probabilities are **never retroactively reinterpreted** as pregame outputs; the UI shows the saved published percentages for those weeks. New Week 8 archived forecasts acquire supplemental postprocessed probabilities while preserving the raw rates.
3. Single-game sims show raw simulation frequencies (actual win counts) separately from **adjusted probability estimates**; the counts are not altered by this mathematical calibration.
4. Do not use future final scores or today’s live team state to rebuild past forecasts. Keep calibration coefficients frozen through the 2025 season, then re-evaluate after the next completed games.
5. Calibration and uncertainty are about *future prediction accuracy*, not guaranteeing exactly 100 wins/losses out of 100 repetitions.

## Favorite-rate reliability (2026 saved pregame games, raw)

| Saved favorite probability | Games | Mean predicted | Won | Observed |
|---|---:|---:|---:|---:|
| 0%–50% | 2 | 49.4% | 1 | 50.0% |
| 50%–60% | 23 | 56.1% | 9 | 39.1% |
| 60%–70% | 19 | 63.9% | 13 | 68.4% |
| 70%–80% | 12 | 75.5% | 8 | 66.7% |
| 80%–100% | 3 | 85.7% | 3 | 100.0% |

Read `calibration-audit-data.json` for the exact deduplicated forecast IDs, probabilities, scores, diagnostic uncertainty and model versions. These results are exploratory, not a prospective guarantee.