Value-of-Information Sampling

What goes wrong with a naive approach

A lot of runtime decisions depend on measurements that are themselves expensive — a detailed frame snapshot, a full buffer diff, a conformal calibration pass. The naive approach is a timer: sample every $\Delta t$ milliseconds regardless of whether anything is happening.

The problem is that a timer has no idea whether sampling would change any decision. If the posterior over “is this frame a violation?” is already sharp at $p = 0.001$, sampling makes it sharper at $p = 0.0009$ — not actionable, not useful, just overhead. If the posterior is $p = 0.45$ and any new observation would flip the decision, sampling is absolutely worth the cost.

The right quantity is the value of information (VOI): the expected reduction in posterior variance from taking one more sample. Sample iff the VOI gain justifies the cost.

Mental model

Maintain a Beta posterior $p \sim \text{Beta}(\alpha, \beta)$ over the quantity of interest (typically “probability this measurement violates the SLO”). Compare the current posterior variance to the expected posterior variance after one more observation.

• Current variance $\operatorname{Var}(p)$.
• Expected post-sample variance $E[\operatorname{Var}(p \mid X_{n+1})]$ — averaging over what the next observation might be.

VOI is the difference. Sample iff:

(i.e., the information gain, valued appropriately, exceeds the cost of the measurement) OR the max_interval timer has expired. The timer provides a safety net — VOI alone can starve a stale posterior.

The math

Beta posterior summaries

Post-sample variance (one observation)

If we observed a success, $\alpha \to \alpha + 1$:

If we observed a failure, $\beta \to \beta + 1$:

Expected post-sample variance

VOI and the sample-iff rule

Sample iff:

Defaults (TUI-calibrated)

The prior $\text{Beta}(1, 9)$ encodes “SLO violations are rare” — a reasonable prior for most TUI metrics. The 100–1000 ms clamps the sampling rate to between 1 Hz and 10 Hz.

Worked example — rare-event posterior

Prior $\text{Beta}(1, 9)$ (mean 0.1). After 50 successes, 2 failures the posterior is $\text{Beta}(3, 59)$:

Expected post-sample variance is about $0.00072$. VOI $\approx 10^{-5}$. With $v_{\text{scale}} = 100$ and $c_{\text{sample}} = 0.08$:

VOI says no. But if the timer has been ticking for 1000 ms with no sample, the max_interval clause fires — keeping the posterior from going stale.

When the stream shifts and the posterior pulls toward 0.4 with high variance, VOI grows rapidly and sampling becomes economic again.

Rust interface

use ftui_runtime::voi_sampling::{VoiSampler, VoiConfig, SampleDecision};
 
let mut sampler = VoiSampler::new(VoiConfig {
    prior_alpha: 1.0,
    prior_beta:  9.0,
    mu_0: 0.05,
    lambda: 0.5,
    sample_cost: 0.08,
    min_interval_ms: 100,
    max_interval_ms: 1000,
});
 
// Once per frame:
match sampler.decide(now) {
    SampleDecision::Sample { reason } => {
        let x = take_measurement();
        sampler.observe(x);
    }
    SampleDecision::Skip { reason: _ } => {}
}

VoiSampler also exposes the e-value $e_t$ for pairing with an anytime-valid alert layer; see e-processes.

How to debug

Every decision emits a voi_decision line:

{"schema":"voi_decision","should_sample":false,
 "voi_gain":1.1e-5,"score":0.0011,"cost":0.08,
 "e_value":0.94,"boundary_score":0.55,
 "reason":"voi_below_cost","interval_ms":220}

FTUI_EVIDENCE_SINK=/tmp/ftui.jsonl cargo run -p ftui-demo-showcase
 
# Histogram of sampling reasons:
jq -c 'select(.schema=="voi_decision") | .reason' \
  /tmp/ftui.jsonl | sort | uniq -c

max_interval firing constantly means VOI is saying no every time — either your $v_{\text{scale}}$ is too small or your posterior is already sharp and you’re wasting measurements.

Pitfalls

Cross-references

• Hint ranking — a different consumer of the same Beta-posterior machinery.
• /runtime/overview — where the sampler is wired into the frame loop.
• E-processes — the anytime-valid alert layer that pairs with VOI sampling.

Where next