Early-Feedback Calibration: Online Calibration of Volatility Forecasts with Early, One-Sided Feedback

by Akshat Valse

Advised by Hongyang Gao, Department of Computer Science, Iowa State University

We study online calibration of risk forecasts for a cumulative quantity, such as the realized variance of an asset over the next six or twenty-four hours. Such forecasts are usually scored when the horizon ends, yet the target’s increments are non-negative: once its running sum crosses the issued upper bound, the miss is certain, while coverage is only confirmed at the end. We are developing a model-agnostic calibration method for quantile forecasts that uses this early, one-sided feedback.

Overview

In Experimentation and Development

On a dark slate background, a pale gray staircase climbs from the lower left to the right edge. A white line crosses the image just below the middle, and the steps above the line are blue.

Introduction

Suppose we forecast how turbulent an asset will be over the next twenty-four hours, as its realized variance (a running total of squared returns), and commit to an upper bound. The total can only grow, so if it passes the bound at hour ten, we already know we missed. If it stays below, we learn we were right only when the day ends.

Risk forecasts for a cumulative quantity like this are usually scored when the horizon ends, so a calibrator that waits for the complete outcome receives feedback one full horizon after it issues each forecast. That delay can slow its response to a volatility shift. But the target’s increments are non-negative, so its running partial sum is monotone. If the sum crosses the issued upper bound before the horizon ends, the miss is certain immediately, while coverage is only confirmed at the end (Figure 1). The feedback is early, but one-sided.

Figure 1. One forecast horizon, two running sums against the same issued upper bound. One path first exceeds the bound at step $\tau$ , so from that step on its miss is certain; a scorer that waits for the complete outcome still records it only at $H$ . The other path, drawn in gray, stays below, and its coverage is confirmed only at $H$ . Schematic drawn from a seeded toy simulation, not project data: 24 steps of squared-Gaussian increments (doubled for the path that crosses), with the bound at the upper 0.1 quantile of $\chi^2_{24}$ .

We are developing a model-agnostic calibration method for quantile forecasts of a cumulative target that uses this early, one-sided feedback instead of waiting for each horizon to end. Model-agnostic means that it does not depend on which forecaster issues the bounds. Theory and experiments are both in progress.

What Is Known, and When

We first make the observation structure precise. The public overview states it in words; the notation below is introduced for this page and describes what can be observed about a forecast, and when, not how our calibrator uses it. Let $x_s\ge 0$ be the target’s increment at step $s$ , and let a forecast issued at time $t$ have a horizon of $H$ steps and issued upper bound $\hat u_t$ . Its target and its running partial sums are

Y_t=\sum_{h=1}^{H}x_{t+h},\qquad S_{t,k}=\sum_{h=1}^{k}x_{t+h},\qquad 0=S_{t,0}\le S_{t,1}\le\cdots\le S_{t,H}=Y_t .

A miss is a strict crossing, $Y_t>\hat u_t$ ; with continuous increments a tie has probability zero. Because the partial sums never decrease, a crossing settles the outcome before the horizon ends:

S_{t,k}>\hat u_t\ \Longrightarrow\ Y_t\ge S_{t,k}>\hat u_t .

Let $\tau_t$ be the step at which the miss becomes certain:

\tau_t=\min\{k\in\{1,\dots,H\}:S_{t,k}>\hat u_t\}\quad(\min\varnothing=\infty),\qquad \{\tau_t\le H\}=\{Y_t>\hat u_t\}.

A miss is known at $t+\tau_t$ , which is $H-\tau_t$ steps before the horizon ends. Coverage, $\tau_t=\infty$ , is known only at $t+H$ . This asymmetry is what we mean by one-sided: crossing the bound settles the outcome at once, while staying below it settles the outcome only when the horizon ends (Figure 2).

Figure 2. When the outcome of one forecast becomes known. Watching the running sum, a miss is known at the crossing $t+\tau$ , a lead of $H-\tau$ steps, and coverage only when the horizon ends. Waiting for the complete outcome, both are known only at $t+H$ . The diagram shows the observation structure only; it says nothing about how a calibrator uses it.

How Early Is Early: A Classical Toy

Figure 1 shows that a miss can become certain early; how early depends on the increments. For intuition, we work through a simple model with an exact answer: a classical first-passage calculation, not the project’s method, model or data. Take one horizon of i.i.d. increments $x_h=c\,\xi_h$ , where $\xi_h\sim\chi^2_1$ is a squared standard normal, the classical toy for squared returns at constant volatility, or $\xi_h\sim\mathrm{Exp}(1)$ . The bound $u_\alpha$ solves $\Pr\big(\sum_{h\le H}\xi_h>u_\alpha\big)=\alpha$ , so it is exact in the calm regime $c=1$ , and $c>1$ is a volatility jump the bound has not seen. Nothing is recalibrated: every bound stays fixed.

Figure 3. Classical baseline, not project data or method. Left: one seeded run of the running sum against a bound fixed at the calm-regime quantile; the filled marker is the step $\tau$ at which its miss becomes certain (a run that stays below the bound ends in a hollow marker instead). Right: $\tau$ over $N$ seeded runs against its exact law, with Monte Carlo readouts at ± one standard error beside the exact values. No calibration is applied; bounds are fixed. Whether early misses make recovery faster is the project’s open question.

Exact Reference

Drop the index $t$ and let $\bar F_k$ be the survival function of $\xi_1+\cdots+\xi_k$ . Monotonicity turns the stopping time into a marginal probability:

\Pr(\tau\le k)=\Pr(S_k>u_\alpha)=\bar F_k(u_\alpha/c),\qquad k=1,\dots,H.

Both families have elementary survival functions. For $\mathrm{Exp}(1)$ increments it is the Poisson identity, since $\xi_1+\cdots+\xi_k>x$ exactly when a unit-rate Poisson process has had fewer than $k$ arrivals by time $x$ ; for $\chi^2_1$ increments it depends on the parity of $k$ , with $\bar\Phi$ and $\varphi$ the standard normal upper tail and density:

\mathrm{Exp}(1):\quad \bar F_k(x)=e^{-x}\sum_{j=0}^{k-1}\frac{x^j}{j!},

\chi^2_1:\quad \bar F_{2m}(x)=e^{-x/2}\sum_{j=0}^{m-1}\frac{(x/2)^j}{j!},\qquad \bar F_{2m+1}(x)=2\bar\Phi(\sqrt x)+2\varphi(\sqrt x)\sum_{j=1}^{m}\frac{x^{j-1/2}}{(2j-1)!!}.

The quantities the widget reports follow directly:

\Pr(\text{miss})=\bar F_H(u_\alpha/c),\qquad \Pr(\tau<H\mid\text{miss})=\frac{\bar F_{H-1}(u_\alpha/c)}{\bar F_H(u_\alpha/c)},

\mathbb E[H-\tau\mid\text{miss}]=\frac{1}{\bar F_H(u_\alpha/c)}\sum_{k=1}^{H-1}\bar F_k(u_\alpha/c),

the last from the identity

(H-\tau)\,\mathbf 1\{\tau\le H\}=\sum_{k=1}^{H-1}\mathbf 1\{\tau\le k\},

which holds because both sides equal $H-j$ when $\tau=j\le H$ and vanish when $\tau=\infty$ . $\Pr(\tau<H\mid\text{miss})$ is the share of misses that become certain strictly before the horizon ends; a crossing at the last step, $\tau=H$ , gives no lead.

The bound $u_\alpha$ itself comes from bisection on the closed form. These closed forms were cross-checked against R’s pchisq and pgamma, with a worst relative error of 1.7 × 10⁻¹⁴. The widget computes every exact value from them; at each setting in Table 1 its exact readouts match the tabulated values, rounded to the digits the widget displays.

$H$	$u_\alpha$	$c$	$\Pr(\text{miss})$	$\Pr(\tau<H\mid\text{miss})$	$\mathbb E[H-\tau\mid\text{miss}]$
12	18.549347787	1	0.100000	0.696783	1.8129
12	18.549347787	2	0.679308	0.878181	3.5596
72	87.743047749	1	0.100000	0.864984	5.2231
72	87.743047749	2	0.996409	0.998892	26.7387
288	319.153638462	1	0.100000	0.929895	10.9193
288	319.153638462	2	1.000000	1.000000	126.9232

Table 1. Exact values for the $\chi^2_1$ toy at $\alpha=0.1$ , from the closed forms above, with $c=1$ calm and $c=2$ after the increments double. Toy values, not project results; every row was also reproduced in R. Set the widget to any row to compare.

Experiments

Our evaluation asks whether acting on early misses lets a calibrator respond to a volatility shift sooner than delayed-feedback conformal baselines, without systematically widening its bounds the rest of the time. The question is open, and nothing on this page answers it. For this evaluation we have built:

Data: a leakage-audited pipeline with a sealed test set.
Simulation: a framework with pre-registered hypotheses.
Baselines: delayed-feedback conformal and classical volatility baselines, including a temporal convolutional network.

The main simulation study and the real-data evaluation are still in progress.

Status

The project is in its experimentation and development phase. The method’s construction, the theory and all results are held back until publication, together with the code, proofs, experiment logs and write-ups. Nothing on this page is a result of the project: the figures and the toy describe the observation structure only, with every bound held fixed.

Public overview

Learned PDE Solvers All research Agent Reliability