How many nodes the budget has dropped over this graph's life.
Ids currently compiled. Node lifetime is a budget question — see [PND-PROCCACHE].
Rows still owed per node, for tests and for explain.
Recomputes that ran ranged, and that ran whole — [PND-PROCRANGE].
Worth having as a counter rather than inferring it from timings: a node silently falling back to a full recompute is the failure mode here, and it looks exactly like "the optimisation did not help much".
Bytes currently retained across every materialized node value.
Reads one output column of a compiled spec, by output suffix.
Compiles a spec (and its inputs) into nodes, memoized by specId.
A returned handle is not durable under a byte budget. Eviction
disconnects a node's inlets, so a Compiled held across a run can
throw UnconnectedInputError on a later pull, naming the input
rather than the budget that took it. run re-resolves through
columnOf and never hits this; a caller holding its own handle
should re-compile after any run, which is a memoized lookup when
the node survived. Only relevant with budgetBytes set — without
one, nothing is ever evicted.
Validation happens here rather than at pull time so a bad plan is rejected before any work: params first, then arity, then the typed input check.
Drops least-recently-used nodes until the graph is inside its byte budget. Called after a run resolves; safe to call at any time.
A node that feeds a retained node is skipped — its consumer holds the outlet, so dropping the lookup frees nothing and would only force a recompile on the next pull.
Reads a fold's fact.
The same memoized pull columnOf does, which is the whole change:
the value is cached against the node's version like any column, so
asking twice costs a version check rather than a rescan.
Replaces the bound data. Every node downstream goes dirty.
Replaces the bound data, declaring that rows before changedFrom
are unchanged — [PND-PROCRANGE].
This is the whole input to ranged recompute: a node that declares a
lookback and a runRange then rebuilds only
[changedFrom - lookback, length) instead of the whole column.
The claim is the caller's to keep. Nothing here verifies that the earlier rows really are untouched, because verifying costs the scan the whole feature exists to avoid. Pass a row that is genuinely at or before the first difference — a live feed appending a bar passes the old length, which is the case this is built for. Getting it wrong yields a stale prefix rather than an error, so when in doubt use setSource, which recomputes everything.
A source plus every node compiled against it.
The budget — [PND-PROCCACHE]
Every distinct spec ever compiled used to be retained forever, so memory scaled with questions asked rather than with anything bounded. A session that walks a slider from period 20 to 200 leaves 180 nodes holding 180 result columns, and nothing ever drops one.
The ticket framed this as an op-level cache: an op declares which of its inputs key a result, and the engine memoizes around
compute. Half of that is already true here and should not be rebuilt. A spec'sspecIdis content-addressed over its op, params and inputs, so asking the same question twice hits the same node by construction — there is nothing for an op to declare, and a per-op cache would be a second key beside a correct one.What was genuinely missing is the other half, and the ticket is right that it does not belong to the op: a per-op capacity is a per-op promise, and nothing supervises the total. Measured at 20 nodes × 5 entries of a 200k-row result, a per-op cap held 100 entries and 157 MB where one engine-wide cap held 10 and 35 MB.
So the budget is graph-wide, in bytes rather than entries — the unit that means anything, and only knowable since [PND-PROCCOL] made node values columns with a reportable
columnBytes. Eviction is LRU with one constraint: a node feeding a retained node is skipped, because dropping it frees nothing while its consumer still holds the outlet.