Sampling
sample({ stride }) and sample({ reservoir }) thin a TimeSeries to a
smaller subset of events, without changing the schema. Two strategies for
two questions:
- Stride (
{ stride: N }) — keep every Nth event, uniform-over-time. Cheap, deterministic, the right pick for sliding-window stats over a thinned stream. - Reservoir (
{ reservoir: { size: K } }) — random K-of-N events, uniformly drawn. The right pick for visualization (uncorrelated scatter points without aggregation's grid collapse) and population summaries.
What's covered:
- When to use which — the decision table.
- Stride — deterministic 1-in-N thinning.
- Reservoir — random K-of-N via Algorithm R.
- Visualization —
series.sample({ reservoir }).toRows()as the canonical scatter shape. - Multi-entity series — see
partitionBy— per-partition sample state. - Live counterpart — when to reach for the streaming variant instead.
When to use which
Picking the wrong strategy is the highest-leverage bug this page can prevent. The use case maps cleanly onto one of the two:
| Use case | Stride | Reservoir |
|---|---|---|
| Sliding-window stats (rolling avg / percentiles) | ✅ default | ⚠️ random correlation across windows |
| Population summary over the snapshot | ⚠️ regular-spacing artifact | ✅ default |
| Visualization (scatter plot, sparkline samples) | ⚠️ regular-spacing artifact | ✅ default |
| Top-K / unique reducers | ❌ misses singletons | ⚠️ also misses, with extra randomness |
Rule of thumb: if downstream is aggregate / rolling, reach for
stride. If downstream is toRows() for a chart or a population-level
reducer, reach for reservoir.
Stride: deterministic 1-in-N
const sampled = series.sample({ stride: 10 });
// length === Math.floor(series.length / 10)
// keeps the 10th, 20th, 30th, ... events (1-indexed)
Cheap (O(N) time, single pass, no RNG). Uniform-over-time, so any
windowed statistic computed downstream is also uniform-over-time —
which is why stride is the right default for sliding-window stats.
The trade-off is the regular-spacing artifact: a periodic source signal whose period is a multiple of the stride disappears entirely into the gaps. In practice this rarely matters for monitoring data (noise is usually broad-spectrum), but for any periodic source — heartbeat events, scheduled probes — pick a stride that's coprime with the period or use reservoir.
stride must be a positive integer; non-integer or <= 0 strides
throw at construction.
Reservoir: random K-of-N
const sampled = series.sample({ reservoir: { size: 500 } });
// length === Math.min(500, series.length)
// each event has probability ~K/N of being included, drawn uniformly
Single-pass Vitter's Algorithm R:
fill the reservoir with the first K events, then for each subsequent
event at index i, replace a random reservoir slot with probability
K / (i + 1). Output is sorted by key on the way out so TimeSeries's
chronological invariant is preserved.
Each event in the source has the same probability of being in the
output (K/N for N >= K, 1 otherwise). Reducer outputs over the
sampled series approximate the population reduction, with standard
error ~σ / sqrt(K) for the sample mean.
size must be a positive integer; non-integer or <= 0 sizes throw
at construction. When K >= N, all events are returned in order
(no sampling).
Sample-rate caveats
Reducer outputs reflect the sampled stream, not the source:
const sampled = series.sample({ reservoir: { size: 500 } });
const counts = sampled.aggregate(Sequence.every('1m'), {
events: { from: 'cpu', using: 'count' },
});
// counts.value().events approximates true count × (K / N)
// → multiply by N/K to estimate true counts
Multiply count-shaped outputs by N / K (or by stride for stride
sampling) to get population-level estimates.
Visualization
The canonical scatter shape:
const points = series.sample({ reservoir: { size: 500 } }).toRows();
// 500 points, drawn uniformly from the source, in chronological order.
This beats aggregate(Sequence.every('1m'), { ... }) for visualization
because:
- No grid collapse. Every point is a real source observation, not a bucket reduction. The chart preserves the variability of the underlying data.
- No regular-spacing artifact. Reservoir's draw is uncorrelated with any periodic structure in the source.
- Fixed point count. A 500-event scatter renders the same way regardless of whether the source has 10k or 10M events. Memory and render time stay constant.
For sparkline / line shapes where ordering matters, the chronological
sort that sample({ reservoir }) already does means the output is
chart-ready without further work.
Multi-entity series — see partitionBy
A bare sample({ stride: 10 }) on a multi-entity series mixes events
across entities — and on a structured input (events arriving in
round-robin host order), it silently keeps the same subset of entities
and drops the rest. The fix is the same as with every other multi-entity
operator: partition first, then sample.
import { Sequence, TimeSeries } from 'pond-ts';
const sampled = metrics.partitionBy('host').sample({ stride: 10 }).collect();
// Each host's events are thinned independently.
Each partition gets its own stride counter (or its own K-event reservoir
for { reservoir: { size: K } }) — bounded, predictable per-partition
state. For 80 hosts × K=100 reservoir, that's 8000 events of state.
This is the same multi-entity consideration that applies to rolling,
aggregate, fill, diff, rate, cumulative, scan, and pctChange —
see Cleaning data → Multi-entity series
for the broader pattern.
Live counterpart
LiveSeries.sample, LiveView.sample, LivePartitionedSeries.sample,
and LivePartitionedView.sample accept { stride: N } and return a
LiveView<S> with a per-instance closure-captured counter. v0.17.0
ships stride only on the live side; reservoir is queued for a
later release once the streaming RFC's exact-removal eviction channel
lands.
// Live: per-host stride feeding per-host rolling
live.partitionBy('host').sample({ stride: 10 }).rolling('5m', mapping);
// Snapshot: reservoir for visualization
series.sample({ reservoir: { size: 500 } }).toRows();
For the live story (firehose framing, the multi-entity bias trap, the
LiveView chainable surface), see
Live transforms → Sampling.
See also
- Cleaning —
dedupe/fill/materializeand the broader multi-entity story - Aggregation — bucketed reduction (the alternative to sample-then-render for charts where bucket-level rollups are wanted)
- Rolling — the primary downstream consumer of stride-sampled streams
- Live transforms → Sampling — the streaming variant