Quality Report — train_38 LSTM + calibrator, KO phase WK 2026

Three things stand out:

1. Almost no complete misses. In 29 of the 30 KO matches, the model scored at least the winner or one correct team score. Only once fully wrong. That's a very stable error profile — the model almost never "derails".

2. More than half top hits. In 56.7% of matches (17/30), the model reached ≥95 points, i.e. the correct winner plus at least one exact team score (or an exact correct draw). That's the category that wins pools.

3. Strong exact-score hit rate. Almost 1 in 4 KO results exactly correct (23.3%). For calibration: a naive Poisson-modal model scored 16.7% exact, and a market-optimized model only 10%. Predicting exact scores is the hardest task, and it's precisely there that the LSTM performs above average.

The model beats the best public predictor in the world on both measures — winners and Brier score — across these 14 out-of-sample knockout matches.

Where the difference lies. Both missed the same three big upsets: Norway's demolition of Brazil (w91), and both semi-finals (Spain over France, Argentina over England). But the model won two duels where Opta missed:

USA–Belgium (w94): Opta favored USA (37.2%). Model: 1-2 Belgium. Result: 1-4 Belgium. Model correct, Opta wrong.
Switzerland–Colombia (w96): Opta favored Colombia (42.7%). Model: 1-1 (draw → penalties). Result: 0-0 (Switzerland on penalties). Model correct, Opta wrong.

In the Round of 16, it was 7/8 (model) against 5/8 (Opta) on winners. That gap is what tips the overall comparison.

Honest caveats

• Winner accuracy is a clean, fair comparison — and there the model wins clearly (78.6% vs. 64.3%).
• Brier is partly structurally favorable to a deterministic model. A model that commits to one outcome scores a Brier of 0 when right (better than any hedged probability) and 2 when wrong. Over a sample where the model is accurate (11/14), that structure works in its favor. So it is fair to say the LSTM matches-to-exceeds Opta on Brier — but the winner metric is the most convincing of the two.
• Sample = 14 matches (everything from the Round of 16 onward). More robust than the earlier 6, but still one tournament.
• Opta probabilities are 90-minute outcomes (win A / draw / win B) from the official previews; penalty decisions count as the 90-minute draw, consistently for both sides.

On the KO phase of WK 2026, train_38 + calibrator is a qualitatively strong model: a high and stable point level (99.3/match), very low miss risk (1 in 30 fully wrong), an above-average exact-score hit rate (23%), and low goal error (MAE 1.17). It beats every statistical baseline on pool scoring, and on the 14 knockout matches with real Opta win-probability data (Round of 16 onward) it also beats the Opta supercomputer itself — higher winner accuracy (78.6% vs. 64.3%) and a better Brier score (0.429 vs. 0.457). The only real limitation is the small sample (30 matches overall, 14 in the Opta comparison) and the two semi-finals — outcomes no model, Opta included, saw coming. Within those bounds: this is a well-performing predictor, not a lucky streak.

• Scoring rules: exact 200 · correct draw 100 · winner+1 team score 95 · correct winner 75 · wrong winner/1 team score 20 · wrong 0. Calibrated against the original scoreboard page (LSTM group 4980, NBinom 4655 reproduced exactly).
• Penalty decisions are counted as a draw (90 min), consistent with the pool.
• The metric is descriptive for this KO phase; 30 matches is a modest sample.

Group phase — assessment vs. Opta/market level:

In the group phase (72 matches, Uncal series) the model scored 5230 points = 72.6/match — on par with Opta/market-grade Poisson EV-optimal (72.4/match) and the >5% ELO rule (72.7/match). Market-level performance, no deficit.

The calibrator's value is in the KO, not the group phase. Cal 71.5/match vs. Uncal 72.6/match (−1.1). That is why the Uncal series is the right benchmark for the group phase.

Across both phases combined: strong in the group phase (Opta-level, 72.6/match, 68% winners), superior in the KO (78.6% winners, beats Opta). This is a world-class predictor.