Anomaly Detection
Two methods, one pattern: rolling baseline + threshold. For each
event, compute a rolling average and standard deviation; if the
event's value is more than sigma * sd away from the mean, flag it.
baseline(col, opts)— appendsavg/sd/upper/lowercolumns to the source schema. Drives band charts and outlier filters.outliers(col, opts)— sugar over the same rolling pass, returns just the anomalous events.
Both are built on top of rolling(). Reach for them
directly when the rolling-baseline pattern is what you want; reach
for rolling() itself for arbitrary windowed reductions.
baseline(col, opts)
const bands = cpu.baseline('cpu', { window: '1m', sigma: 2 });
// bands.schema: [time, cpu, host, avg, sd, upper, lower]
Four optional number columns appended:
avg— rolling average ofcolsd— rolling standard deviation ofcolupper—avg + sigma * sdlower—avg - sigma * sd
Then a band chart is two toPoints calls; an outlier filter is one:
const data = bands.toPoints();
// [{ ts, cpu, ..., avg, sd, upper, lower }, ...] — chart-ready
const anomalies = bands.filter((e) => {
const v = e.get('cpu');
const lo = e.get('lower');
const hi = e.get('upper');
return v != null && lo != null && hi != null && (v > hi || v < lo);
});
Warm-up and flat windows
- Before the rolling window has enough samples for a meaningful
baseline,
avgandsdareundefined;upperandlowerfollow. - When the rolling window is flat (every sample identical),
sd === 0.avgandsdstill report their real values, butupperandlowercollapse toundefined— a zero-width band would flag every non-equal point as anomalous, which is usually the opposite of what you want.
Filters that null-check upper / lower will do the right thing
through warm-up and flat-window regions.
The default warm-up only suppresses output until there's at least
one sample. On noisy sub-second telemetry that's usually too eager
— a 2- or 3-sample window has a near-zero rolling stdev, so the
band collapses tight around the first few values and false-flags
the next normal-looking event. Pass minSamples to widen the
warm-up:
cpu.baseline('cpu', { window: '1m', sigma: 2, minSamples: 20 });
While the window holds fewer than minSamples source events,
avg, sd, upper, and lower are all undefined. The option
threads through to the internal rolling
pass and is also available on outliers().
Custom column names
If the defaults (avg / sd / upper / lower) collide with
source columns:
cpu.baseline('cpu', {
window: '1m',
sigma: 2,
names: { avg: 'cpuAvg', sd: 'cpuSd', upper: 'cpuHi', lower: 'cpuLo' },
});
Collisions throw at construction, so the rename is mandatory, not optional.
outliers(col, opts)
Just the anomalies — same schema as the source, filtered to events outside the band:
const anomalies = cpu.outliers('cpu', { window: '1m', sigma: 2 });
// anomalies.length ≤ cpu.length, schema unchanged.
Composes with the rest of the API for dashboards:
// Bucketed anomaly count per 15s for a bar chart.
const perBucket = anomalies.aggregate(Sequence.every('15s'), {
cpu: 'count',
});
// Per-host breakdown.
const perHost = anomalies.groupBy('host');
Conceptually equivalent to baseline(col, opts).filter(...);
implemented independently (one rolling pass either way). Reach for
baseline() directly when you want the band columns for charting.
The two share flat-window semantics — both skip events where
sd === 0.
Picking one
- Drawing a band chart:
baseline— you'll use theupper/lowercolumns directly. - Just want the spike list (alerting, counting, per-host):
outliers. - Both at once:
baseline— one rolling pass for everything.
Beyond rolling-baseline
baseline and outliers are deliberately the only first-party
anomaly-detection helpers. Other patterns — STL decomposition,
seasonal-adjusted thresholds, classifier-based detection — fit
map, rolling, and reducer combinations cleanly enough that
folding them into the library would just add a vocabulary tax. If
you need one and the composition feels heavy, file an issue with the
shape and we'll see if it's worth lifting.