Skip to main content

Caching and performance

Every number here comes from a runnable script in the repo — most in packages/process/scripts/, the conversational-refinement figures in apps/process-demo/scripts/. Run them after npm run build --workspaces rather than trusting prose.

What the cache buys​

Identity is content-addressed, so asking the same question twice hits the same node by construction — there is no cache key to manage and no way to manage it wrong. The shape this rewards is conversational refinement: "smoother" → "try 200 instead" → "back to how it was" returns in ~3 ms against ~75 ms cold in the repo's measurements, because a content-addressed sma(50) is never invalidated by a detour — only unused.

The response makes the cache visible rather than asserted: every node reports cached and ms (the badge row). If an architecture claim about warm graphs matters to you, read it off your own responses.

Byte budgets​

The flip side of content addressing: memory scales with questions asked. A slider dragged from period 20 to 200 mints a node per position, and nothing ever drops one. The cap:

bind(series, { registry, budgetBytes: 64 * 1024 * 1024 });
// or host-wide, per graph:
createHost({ registry, budgetBytes: 64 * 1024 * 1024 });

The budget is graph-wide and in bytes, because entries are not the unit anyone has a limit in — one node over 1M rows outweighs fifty over 5,000. Eviction is LRU with one constraint: a node feeding a retained node is skipped, since dropping it frees nothing while its consumer still holds the outlet. Enforcement runs after each request, never during one.

Observability: graph.retainedBytes and graph.evictions. Sizing counts what is actually retained — backing-buffer capacity, not logical length.

One sharp edge: a Compiled handle held across a run is not durable under a budget — eviction disconnects a node, and a later pull through a stale handle throws. run re-resolves internally and never hits this; a caller holding its own handles should re-compile after runs (a memoized lookup when the node survived).

Ranged recompute​

A live feed appending a bar should not pay for the whole column again. The caller declares what changed; ops that can, patch:

graph.setSourceFrom(series, changedFrom); // rows before changedFrom are unchanged

Three declarations cooperate:

  • OpDef.lookback(params) — rows of history the op needs before a range for its output there to be fully defined. Lookbacks sum along a nested chain (sma(20) over sma(50) needs 69, not 50).
  • OpDef.runRange(ctx) — recompute only [from, to), given the previous output. The fast path writes into ctx.out — prepared output buffers with the kept prefix already copied, values and validity both — and returns nothing; returning a whole result is the slower way to say the same thing.
  • The claim is the caller's to keep. Nothing verifies rows before changedFrom are really unchanged, because verifying costs the scan the feature exists to avoid. When in doubt, setSource recomputes everything.

runRange is opt-in for a correctness reason, not a performance one: a patched result must be bit-identical to a from-scratch one, or answers depend on the sequence of edits that produced them — invisible to any test that only computes from scratch. That property holds for the range-exact rolling kernels and does not hold for median, percentiles, min, or max. Declaring nothing is always correct and merely slower.

graph.recomputes reports { ranged, full } — worth watching because a node silently falling back to full recomputes looks exactly like "the optimisation did not help much". Measured in the repo: 500k rows × 5 studies went from ~209 ms to ~6.5 ms per tick, bit-identical every tick.

Minimum history​

The other half of interactive latency: how short a tail can you bind without truncating warm-ups?

import { requiredHistory } from '@pond-ts/process';

const history = requiredHistory(registry, plan);
// { known: true, rows: 69, … }  — or { known: false, undeclared: ['myOp'] }

known: false names the undeclared ops instead of returning a number — a missing lookback declaration and an element-wise op are the same zero with opposite meanings, so an op that genuinely needs no history declares () => 0. The bound is tight, not merely safe: in the repo's measurement an 8-study stack over 500k rows served a 5,000-row display at ~1.3 ms/tick with zero truncated cells, and exactly one at a tail one row shorter.

The worker pool​

HostPool (@pond-ts/process/pool) routes whole requests across worker threads, each holding a long-lived Host. No engine change makes this possible: a plan is JSON, a registry is a module both isolates import, and result columns travel as transferable buffers.

The measured shape is the honest part: 3–4× on distinct requests at every size tried — and ~0.01× on repeated ones, where the in-process memo would have returned the same column for nothing and a pool ships every answer regardless. Cache-hit rate decides it, not request size. A workload dominated by repetition (most conversational ones are) wants one host, not eight.