Model Performance & Auditing
Empirical multi-horizon accuracy auditing of AgileCast vs competitor models against confirmed Octopus Agile settlement outturns.
📊 Multi-Horizon Accuracy Scorecard
| Lead-Time Horizon | Model | MAE (p/kWh) | RMSE (p/kWh) | Bias (p/kWh) | Plunge Capture % | PICP | Samples |
|---|---|---|---|---|---|---|---|
| Loading accuracy scorecard... | |||||||
Paired Unsettled-Slot Evaluation & Zero Post-Publication Bias
To guarantee a strictly fair benchmark, any settlement slot that has already been published by Octopus Energy at snapshot creation time
(tSlot <= actuals_cutoff_utc) is excluded across all models. This prevents external scrapers from receiving artificial
0.00 p/kWh error credit for echoing already-published retail rates in short-horizon bins (0–6h, 6–12h, 12–24h).
Furthermore, every horizon bin and CommonHorizonOverall evaluates the exact paired intersection of (snapshot, slot)
where all active models issued a genuine forecast and a confirmed settlement outturn exists—guaranteeing identical sample counts (Samples) across compared models.
Nord Pool UK N2EX Day-Ahead Auction Integration
The UK N2EX_DayAhead wholesale auction closes at 09:30 UK time and publishes hourly clearing prices around
09:55 UK time—nearly 6 hours before Octopus Energy releases retail Agile rates (~16:00 UK time). AgileCast polls the Nord Pool Data Portal
from 09:50 UK time and routes auction-cleared slots through a dedicated 31-D Day-Ahead Auction Refiner GBDT (auction_model.bin),
rolling refined predictions forward in 24-hour autoregressive waves across the 14-day horizon.
Contractual Tariff Inversion to Canonical Wholesale
Competitors publish consumer tariffs in different DNO regions (e.g. London Region C, Exemplar Region G, National Average Region X).
To achieve fair, apples-to-apples comparison, all forecasts are inverted back to the underlying Canonical Wholesale Price (£/MWh)
using the algebraic inverse of Octopus Energy's contractual formula:
Z = (Rate - A_peak) / M_import.
Ground Truth & Zero Synthetic Data
Evaluations are scored strictly against confirmed Octopus Agile settlement outturns. Missing, unreturned, or unsettled intervals remain genuine blanks — never filled with synthetic curves or flat fallback prices.
Quantile Calibration & PICP
AgileCast produces probabilistic forecasts with $P_{10}$, $P_{50}$, and $P_{90}$ quantiles. Prediction Interval Coverage Probability (PICP) measures the empirical percentage of confirmed actual settlement outturns that fell inside the predicted $P_{10}–P_{90}$ interval (calibrated target: 80%).