Two free forecasting models beat ours on 13 of 14 indexes
TimesFM 2.5 and Chronos-2, two open models run with no tuning, beat Forecast v1 on 13 of 14 indexes, with about 10% less error (a score of 0.269 and 0.278 against 0.299 for ours, lower is better). They now run beside it for 4 weeks before the site shows them.
Lower is better.
Forecast v1 picks among nine classical models for each index and gives the next number with an 8 in 10 range. Two open models trained on large collections of time series were given the same history, one period at a time, with no tuning, and asked for the same three numbers. The score is the error of those three numbers, scaled by how much the index usually moves from one period to the next. A model that says the last number again scores 0.324 on average.
| Index | Cadence | Periods | Forecast v1 | Last number | Chronos-2 | TimesFM 2.5 | TimesFM against v1 |
|---|---|---|---|---|---|---|---|
| $TILT | Weekly | 241 | 0.337 | 0.331 | 0.294 | 0.286 | 15% better, clear |
| $STRAPPED | Monthly | 60 | 0.295 | 0.295 | 0.253 | 0.237 | 20% better, clear |
| $LASTCALL | Monthly | 59 | 0.242 | 0.310 | 0.193 | 0.178 | 26% better, clear |
| $OD | Monthly | 56 | 0.192 | 0.301 | 0.309 | 0.164 | 15% better |
| $SHELTER | Daily | 118 | 0.343 | 0.326 | 0.310 | 0.325 | 5% better |
| $SAD | Monthly | 59 | 0.366 | 0.366 | 0.333 | 0.351 | 4% better |
| $LAYOFFS | Weekly | 88 | 0.353 | 0.362 | 0.324 | 0.289 | 18% better, clear |
| $YESNO | Daily | 118 | 0.231 | 0.304 | 0.218 | 0.217 | 6% better |
| $TRENCHES | Daily | 119 | 0.327 | 0.325 | 0.287 | 0.298 | 9% better |
| $MINTS | Daily | 69 | 0.315 | 0.309 | 0.311 | 0.320 | 2% worse |
| $GIGS | Daily | 47 | 0.262 | 0.292 | 0.251 | 0.247 | 6% better |
| $WAGE | Daily | 47 | 0.354 | 0.426 | 0.305 | 0.319 | 10% better |
| $JOBS | Monthly | 14 | 0.237 | 0.341 | 0.224 | 0.230 | 3% better |
| $UNEMP | Monthly | 13 | 0.327 | 0.249 | 0.272 | 0.301 | 8% better |
| Mean of 14 | 1,108 | 0.299 | 0.324 | 0.278 | 0.269 | 10% better |
Clear means a resampling test of the period-by-period difference keeps its whole 95% range on one side of zero. For TimesFM 2.5 that holds on $TILT, $STRAPPED, $LASTCALL, $LAYOFFS. Worse on $MINTS, by a hair. Chronos-2 is clearly worse on $OD, where its range is far too wide.
The ranges.
An 8 in 10 range should hold the number 80% of the time. Forecast v1's held it 76% of the time, a little too narrow. Chronos-2's held it 83% and TimesFM 2.5's 84%, a little too wide. The plain average of the two models scored 0.268, the best of everything tried, but it was tried after seeing the others, so it proves nothing until it is tested forward.
Not from having seen the answers.
A model trained on public data may have seen the older monthly numbers. Scored only on periods after Oct 20, 2025, when both models were already public: Forecast v1 0.283, repeating the last number 0.316, Chronos-2 0.252, TimesFM 2.5 0.250. The gap holds, and the daily crypto and prediction market indexes only began after both models existed.
Four weeks in the open first.
Both models now run every day beside Forecast v1 and write their forecasts to a public file before each number lands, never rewritten: forecasts/shadow.json. The site keeps showing Forecast v1. After 4 weeks the live record decides whether the default changes. A backtest is one run on short series; the live log is the test.
Method and caveats
- Forecast v1's own score: mean pinball loss at the 10th, 50th and 90th percentile, divided by the index's mean period-to-period move. Lower is better. Coverage is how often the number landed inside the 10 to 90 range; 0.80 is right. Every column is scored on the same periods, the ones Forecast v1 scores itself on.
- Each model saw only the numbers before the period it forecast, the number as first published where Tickerz kept it, horizon one period, default settings from each model card, one run.
- Clear means a paired bootstrap of the per-period difference (2,000 draws) puts the whole 95% interval on one side of zero. It treats periods as independent, which they are not, so the clear verdicts are a little generous.
- Under 24 points, so left out: $GOON (20), $AIGF (20), $CPI (23), $CORECPI (23), $LONELY (2), $BROKE (0).
- Both models are Apache 2.0. TimesFM 3.0 is newer but carries a non-commercial licence and was not run.
- Display only, as all of Forecast v1. Nothing here touches a printed, signed or settled number.
Sources
- Chronos-2 (amazon/chronos-2), Apache 2.0. Model card read in full.
- TimesFM 2.5 (google/timesfm-2.5-200m-pytorch), Apache 2.0. Model card read in full.
- Forecast v1, models/forecast.py in tickerz-open. Read in full; the comparison imports it, so the v1 column is its own code.
- Tickerz index history files, read Nov 12 2026. Read in full.
No corrections. First published November 12, 2026.
Live on the site: Forecast v1.