Node test lanes

How the slowest Node test families are split, balanced, packed, and cached. Part of the CI scope and routing index.

The slowest Node test families are split or balanced so each job stays small without over-reserving runners:

• Plugin contracts and channel contracts each retain two weighted process selections in one job, with the standard GitHub runner fallback. Frozen targets keep separate jobs.
• Additional checks: Current targets combine export-collision, session accessor/transcript reader, SQLite transaction, and SQLite schema checks into one serial source-contract row, preserving each command and collecting every failure. Additional checks use five rows on push/PR and six on current-target dispatch; the SDK API report is allocated only for dispatch. Frozen targets retain their eight original rows and historical command fallbacks.
• Core unit fast/support lanes run separately; unit-src, Control UI, and gateway-core each use three deterministic file-weighted stripes, while the security and media/UI companion configs retain their scoped whole-config support groups; core runtime infra splits into process, shared, hooks, secrets, and three cron domain shards.
• Auto-reply runs as balanced workers, with the reply subtree split into agent-runner, commands, dispatch, session, and state-routing shards; dispatch further isolates core, delivery, and lifecycle entrypoints.
• Agentic gateway/server (control-plane) configs split across chat, auth, model, HTTP/plugin, runtime, and startup lanes instead of waiting on built artifacts.
• Normal CI packs only isolated infra include-pattern shards into deterministic bundles of at most 64 test files, reducing the Node matrix without merging non-isolated command/cron, stateful agents-core, or gateway/server suites. Heavy fixed suites stay on 8 vCPU while most bundled and lower-weight lanes use 4 vCPU. Previously promoted compact workloads retain 8-vCPU capacity through their semantic owners and selected files before packing. Timing or build-ownership changes cannot transfer that capacity to an unrelated row. Hosted stripes inherit their parent owner's capacity; whole named manual and release plans retain their existing routing. Full named bundles include their runtime-build mode when present, so checks-node-bundle-infra-small-runtime-1 and checks-node-bundle-infra-small-1 identify different prerequisite groups. Their test inventories and runner limits are unchanged; unique names let release collectors retain both jobs without ambiguous attempt evidence.
• Pull requests on the canonical repository reuse the changed-test resolver against the synthetic merged-tree diff. Precise core changes use bounded targeted jobs; plugin changes retain their owning configs and pack compatible envelopes into serial jobs. Each selected test file keeps its existing process isolation. The planner combines sibling tests with import-graph dependents and falls back to a metadata-complete compact plan for workspace package, package/lockfile, shared harness, split-config, renamed, or deleted changes, public extension-contract changes, tests with special shard setup, partially resolved or empty targets, oversized path or target plans, and planner errors. Nondist descriptors run as Node jobs; dist descriptors fold into the built-artifact boundary. The number of compact jobs follows the current measured weights. That PR fallback retains all sixteen tooling stripes, the isolated tooling shard, and the TUI PTY shard because scripts and PTY owners intentionally require their Go, dist, and environment metadata. Those timing-sensitive groups remain isolated in concurrency-one exclusive bins. Targeted plans always retain the full boundary gate because its repository scanners cannot be derived from imports.
• When canonical pull requests fall back to compact planning, directly changed, existing tests owned by the release-only plugin shard remain as exact-file selections in the existing compact jobs. Canonical Vitest routing keeps unit-fast, contract, bundled, and E2E tests with their own suites; source files, directories, deleted tests, and live tests do not widen plugin coverage. The broad agentic-plugins sweep remains release-only, and push and manual CI plans are unchanged.
• Canonical main pushes use a Blacksmith integration compact with nondist Node jobs plus the dist boundary descriptor. Former multi-config walls (CLI plus CLI-process, isolated plus fake-timers unit fast, and the logging/process/runtime-config trio) are split into per-config shards so no single group floors a lane. Real Node+TSX command tests belong to the isolated CLI-process catalog, so ordinary CLI tests do not prepare a runtime. The process catalog splits by complete file costs, with split sizing bounded below by its complete file costs and runtime prerequisite so an older aggregate timing cannot hide newly owned work. File packing also includes the prerequisite; runtime-consuming CLI children may share one preparation in the same serial job when their complete combined estimate fits the existing 150-second budget. Each child retains its selected files, isolated process and two-worker limit; deliberately separated fixed stripe families remain apart. The gateway process file stays alone because its cold proof already takes 200 seconds. CLI children retain the 150-second sizing target and their two-worker limit. Ordinary hybrid bins containing only non-build CLI children may combine up to 250 predicted seconds, with each original child still admitted separately below 150 seconds; the hosted runtime prerequisite itself has a 160-second floor. They omit the low-signal-per-push tooling and TUI PTY groups while retaining all product-runtime groups, including three file-weighted stripes apiece for unit-src, Control UI, and gateway-core. Blacksmith serial admission stays at 200 seconds for the large class and 276 seconds for the small class. Ordinary groups that can share two process slots admit 360 predicted aggregate seconds; a group already above its serial cap stays alone. Manual dispatches and Full Release Validation retain the full named per-shard matrix. No scheduled workflow currently runs that full Node suite; this is a known coverage-timing gap, not coverage supplied by this compact plan.
• Doctor session and cron tests keep the two longest SQLite files in separate shards, with the remaining files together. Each shard runs complete files under the commands config's existing execution policy. The three owners cannot share a compact job; Blacksmith gives them 8-vCPU placement and its 200-second admission target. The two new owners charge the observed file body plus the full group overhead until the canonical timing refit has two successful main-run samples. Fixture sizes, assertions, and process isolation are unchanged.
• Direct measurements for generated hosted stripes own their packing weights just like native groups. GitHub runs use hosted measurements and hybrid attempt-one runs use Blacksmith measurements; only unmeasured children divide the parent estimate. This keeps an expensive measured child from being packed beside extra work under an artificially small estimate while preserving its test partition, worker pins and per-child file execution policy.
• Changed-plugin jobs run their independent config envelopes serially, including configs whose inner process chunks also run sequentially. The existing worker policy can then use the detected CPU budget without reserving capacity for overlapping plans; multi-target jobs retain their separate concurrency policy.
• The sixteen tooling stripes use measured file weights to keep the Git owner, managed-process, worker-artifact, and transform-cache proofs in separate jobs. Go setup follows explicit selected files first. For whole-config tooling plans, only the ordinary tooling config owns the docs i18n Go tests; isolated and Docker catalogs do not. Unknown historical configs retain their original Go setup. The shard containing the unified declaration compiler fixture retains the existing larger runner; hosted splits carry that placement through each child's selected files before packing. Its synthetic compiler graph uses the existing 1,024 MB heap override, while production full builds keep their resource guard. Git owner cases each own an isolated checkout and process tree, so their real timeout and cleanup checks overlap at most two cases per runner. Worker transform proofs use a separate suite with its own fixture lifetime; late child cleanup cannot remove another suite's inputs. These changes preserve the test cases, deadlines, and process isolation; measured admission determines the job count within the existing hybrid row cap.
• Blacksmith numbered tooling bins request the 32-vCPU class after packing while retaining their logical runner classes, names, file inventory, serial project/file execution and two-worker pins. In run 33689551111, the three slowest tooling bins received two CPUs and 8 GB of memory; their 314–342-second bodies set an eight-minute non-Windows wall. The larger request addresses that host-capacity mismatch without adding shards. Hosted and hybrid tooling placement is unchanged; the request needs native timing proof before claiming the eight-minute target.
• Control UI browser projects greedily pack discovered test files using committed per-file timings, then cold-start basename hints, then source byte size. Bundled files default to at most two workers, bounded by the shared worker limit and local throttling; explicit Vitest CLI worker overrides remain available. Private source servers, real Gateways, and runtime-budget tests stay in a single-worker project; local whole-suite runs finish bundled work before starting that project. Per-file overhead is refitted from serial invocations; parallel invocations still update their file weights. Timing keys supply weights only: discovery still determines the complete test inventory, including new files and files without measurements.
• Broad browser, QA, media, and miscellaneous plugin tests use their dedicated Vitest configs instead of the shared plugin catch-all. Include-pattern shards record timing entries using the CI shard name, so .artifacts/vitest-shard-timings.json can distinguish a whole config from a filtered shard.
• The browser-extension Chromium bootstrap command prepares qaRuntime. It builds the native-host and relay JavaScript and runtime assets; the separate artifact job owns Control UI and plugin SDK declaration validation. The real Chromium flow and its assertions are unchanged.
• The browser native-host launch test is a separate POSIX E2E case. Linux build-artifacts runs it explicitly after building or restoring dist, using OPENCLAW_E2E_USE_PREBUILT_DIST=1 so the test cannot start another build. Its JSON report must contain exactly the named passing assertion in the expected file, with one passed test and zero failures, pending tests, or todos; missing artifacts, skipped tests, and absent results fail. The workflow step skips only when a frozen historical checkout lacks the test file: that is unavailable historical proof, not coverage. Current checkouts with a missing file still fail. Changes to the case, its installation fixture, or its relay-key fixture select the artifact job even on test-only diffs; unrelated browser unit tests stay build-free. Manual CI uses the same artifact step, independently of the release-only plugin sweep.
• Linux Node shard jobs persist Vitest's filesystem module cache through the upstream Actions cache API. On Blacksmith runners, official cache actions use Blacksmith's colocated cache backend instead of GitHub's, so cache entries are backend-local even when their keys match. Blacksmith CI shards are restore-only and unpack the protected Blacksmith seed into isolated runner-local roots. While the GitHub-hosted outage backend is active, every checks-node-* test shard, checks-ui, the ordinary sharded checks-ui-e2e job, both fast contract matrices, and the Vitest-running checks-fast-core tasks restore a separately published immutable transform seed from GitHub's backend. The composite action's single default-off restore-test-caches input keeps the expansion easy to disable without changing cache keys or writer policy; mixed fast-core rows enable it only for tasks that invoke Vitest. The real-Gateway UI job does not restore these test caches; native and Control UI i18n lanes do not invoke Vitest. The hybrid planner profile uses matching key contracts across two backend-local archives: attempt-1 Blacksmith rows read the Blacksmith seed, while hosted retries can read only a separately published GitHub seed. Ordinary CI jobs remain restore-only; the separate trusted warmer owns protected backend-local cache publication. The non-cancelling warmer follows main, runs daily, accepts manual or repository dispatch, and serializes per ref. Its Linux row follows OPENCLAW_CI_RUNNER_BACKEND and publishes only to the cache backend serving that selected runner. A hosted fallback therefore requires its own trusted hosted warmer seed; a successful Blacksmith publication does not populate GitHub's cache backend. Maintainers can select the existing hosted backend for an isolated trusted warmer without canceling a main writer. The Linux row launches each selected shard/config envelope through the normal runner in a fresh child process with concurrency one and --testNamePattern=(?!). Collection preserves the include patterns, environment, compiled imports and per-file cleanup while reusing the same serial cache leaf. Test bodies run in ordinary CI; imports reached only inside those bodies can remain cold until that run. The warmer finishes every selected envelope and saves the content-keyed transform and compile caches even when collection fails, then reports the failure after the cache saves; ordinary CI shard execution remains fail-fast. This prevents config-global state from leaking, avoids expanding filtered shards into whole configs, and retains transforms produced by the previous child. Setup computes the transform-input fingerprint once when transform caching is enabled and passes it to restore and generation validation; disabled caches do not scan the checkout. The fingerprint clears incompatible lockfile, package, tsconfig, and Vitest-config generations. Before publishing, the trusted warmer scans and prunes the transform cache to 75% after it exceeds 2 GiB, and the Node compile cache to 75% after it exceeds 1 GiB. Consumer jobs never prune the restored seed. Vitest hashes module id, source content, environment, and resolved transform config, so ordinary partial source changes keep unchanged entries warm while changed modules miss safely. Coarse restore prefixes bridge workflow runs; normal Actions cache LRU and inactivity eviction bound old immutable archives.
• Trusted Blacksmith Linux Node jobs restore root node_modules, workspace importer trees (including plugin-local versions and links), and the workspace-local pnpm store from one immutable upstream Actions cache, which Blacksmith transparently serves from its colocated backend. Pnpm imports with hard links where the filesystem permits, and keeping the complete installed tree and store in one archive preserves those links. Pnpm's metadata cache lives beneath that same archived store root, so restored installs can verify supply-chain policy without depending on the producer's home directory. Source postinstall and build preparation leave pnpm-owned dependency trees intact. The key includes an explicit archive format, runner OS and architecture, the exact Node patch, and the semantic install-input fingerprint; there are no stale-prefix fallbacks. Manifests are canonicalized before hashing. The repository-owned openclaw metadata block and non-install scripts are excluded because pnpm and the audited direct root hooks do not read them, so runtime schema, publication metadata, formatting, and ordinary test/build script edits keep the dependency tree warm; unaudited lifecycle-hook drift fails closed until its source inputs join the fingerprint contract. Dependency, package-manager, hook-source, and lockfile changes always select a new immutable archive. Every exact restore runs frozen offline pnpm reconciliation, so an unchanged archive validates without registry access or importer relinking. If reconciliation fails, setup first clears every importer tree and rebuilds it offline from the restored store, then clears both modules and store and retries from the network rather than serving a partial tree. Setup then disables pnpm's redundant pre-run dependency check so install and frozen reconciliation remain the only dependency writers; shard commands must not launch concurrent implicit installs. The separate trusted warmer publishes the toolchain and exact dependency archives immediately after setup succeeds, before build and transform warming; preflight and downstream CI jobs are restore-only. Canonical pushes and same-repo pull requests opt into exact restores only on actual self-hosted runners, including hybrid attempt 1. An exact miss automatically falls back to the coarser pnpm store cache. Manual CI dispatches, fork pull requests, hosted lanes, and hosted retries use only that store cache. Cache restore/save failures are optimization misses rather than correctness failures, and normal branch scoping, LRU, and inactivity eviction bound obsolete archives. The former mutable dependency StickyDisk path was retired after repeated successful writers acknowledged commits that later runs still restored as empty filesystems.
• Node shard and build-artifact jobs also restore Node's portable on-disk compile cache through immutable Actions caches. In GitHub-hosted outage mode, the hosted Vitest lane set above restores the separately published GitHub test-scope archive alongside its transform seed. Independent test and build namespaces keep their bytecode separate. The trusted warmer owns the protected test seed; ordinary build-artifacts and test jobs only restore caches. PR and ordinary test jobs only read protected snapshots, so feature-branch bytecode never enters the shared seed and PR traffic creates no cache archives. This reuses V8 bytecode for Node-loaded orchestration, build tooling, and external dependencies across different checkout paths, including when only part of the source graph changes. A maximum-size 2 GiB transform archive costs roughly 15–20 seconds to restore at about 125 MB/s; measured fast-contract transforms are roughly 21 seconds against an approximately 8-second restore, and broader cold imports reach roughly 100–143 seconds. The optimization should be reverted if measured savings fall below restore cost. Vitest child processes disable an inherited compile cache because coverage can be enabled inside dynamic configs and V8 coverage can lose source-position precision when scripts are deserialized from bytecode.
• The existing Linux cache warmer publishes the native SDK declaration archive to the cache backend serving its selected runner before its full build. Hosted lint stripes and the dedicated package-boundary lane restore the matching archive from their runner's backend and validate native compiler inputs and complete outputs before reusing it. A Blacksmith warmer does not populate GitHub's cache backend, so hosted fallback requires a separately published hosted seed. After saving, the warmer removes its native SDK output so the subsequent packaged declaration cache describes the same tree as ordinary build consumers. Package-store contents and pnpm store-location bookkeeping are not compiler inputs; installed dependency bytes, resolution topology, explicit inputs, and compiler identity still invalidate stale declarations. The all-Blacksmith profile retains its read-only sticky-disk path. The Control UI and UI E2E jobs share a Linux Playwright Chromium archive keyed by the exact pinned Playwright version. The protected warmer also runs one standard hosted macos-15 row that installs dependencies and publishes only the pnpm store after an exact miss. It uses the existing OS, architecture, Node-version, package, and lockfile key; macOS CI remains restore-only. This row leaves exact dependency, build, transform, and compile caching disabled and runs no build or test warming. Pnpm owns pruning, while the before/after disk usage and saved archive size expose retained content; pruning does not impose a content-store size bound.
• The build-artifact and Docker seed jobs restore the protected full-build cache through the shared Node setup action. Full, package, and ciArtifacts builds share scripts/write-plugin-sdk-entry-dts.ts: it stages the canonical public/private tsdown SDK declaration groups and caches each group independently. A hit restores into fresh staging; both cold and cached generations must contain every selected SDK entry and a complete relative declaration closure before publication to dist/. The SDK cache never adopts declarations from live dist/. Local plugin lint and package-boundary compilation use independent native declaration trees, not packaged declarations; see declaration ownership. The built Doctor plugin-index proof reuses that exact dist/ output instead of invoking the E2E harness's fallback TypeScript build a second time.
• Full and package builds separately cache the AI, workspace-package, and remaining unified declarations. The unified runtime always rebuilds JavaScript before the shared declaration owner stages one base group and five plugin groups; the later SDK stage uses the same owner for its two groups. Both stages restore unchanged groups into private staging and compile only misses with the existing serial executor. They share one before/after input snapshot and validate every selected entry, successful compiler receipt, relative declaration edge, and shared-chunk owner before publication, including after package preparation clears dist. Conflicting shared bytes or changed consumed inputs fail before live declarations are written; cache records refresh only after successful publication. Canonical DTS configuration enables TypeScript stable type ordering so an unrelated literal allocation does not reorder an unchanged exported type in a rebuilt group. Full publication prunes obsolete declarations while preserving signed app bundles and Control UI assets; SDK publication owns only its flat entries and preserves other groups’ shared chunks. The AI and workspace-package steps also rebuild JavaScript on cache hits. Implicit and explicit declaration-enabled builds share those seeds; runtime-only profiles use an uncached graph and cannot publish declaration generations. Each declaration group hashes ordinary source bytes from its successful compiler Program, so edits to existing unconsumed tests, UI sources, and workflows retain its cache hit. The exact checkout-local .cache/vitest scratch root is excluded from resolution discovery; explicitly consumed inputs, installed aliases, and adjacent or nested paths still invalidate normally. It still validates inherited configuration, generator and package/plugin metadata, compiler identity, and resolution topology; consumed declaration dependencies and new resolution candidates invalidate the generation. Workspace-package changes conservatively invalidate the AI and package declaration caches. The protected warmer publishes build archives immediately after a successful full build, before unrelated test warming and pnpm maintenance. Every warmer attempt gets a new immutable archive key; coarse restore prefixes supply prior groups, and per-step signatures remain the sole content-validity owner. Rebuilt groups replace their complete owned cache trees, including obsolete bytes whose previous record is missing or invalid. GitHub's cache quota and inactivity eviction bound old generations; identical warmer attempts can publish separate archives. The weekly Node 26 minimum lane instead publishes a 14-day artifact after successful main runs and restores only artifacts whose immutable producer identity resolves to that workflow on main, avoiding quota churn without allowing PR code to write a shared cache. Private-QA declarations are never persisted in Actions caches because cache namespaces are not confidentiality boundaries.
• check-additional-boundaries runs the complete supplemental guard list (scripts/run-additional-boundary-checks.mts) with four concurrent child processes and per-check timings. Its 20 checks retain individual failures, deadlines and process cleanup. The shared four-rule focused scan runs once across all source roots; the narrower public lint commands remain available. Prompt snapshots run in their separate lane. Package-boundary compile/canary work stays together, and runtime topology architecture runs separately from the gateway watch coverage embedded in build-artifacts.
• On the 32-vCPU self-hosted build runner, Gateway watch, channel tests, and the core support-boundary shard start together inside build-artifacts after dist/ and dist-runtime/ are already built. GitHub-hosted fallback runs keep Gateway watch serial so low-core contention cannot consume its readiness deadline. Full Node builds then verify Discord component attachment filenames through a serial public Gateway message action, checking the built revision and retaining the named-test JSON result; frozen targets that predate the case explicitly report unavailable proof. Both paths then run the two built TUI PTY artifact canaries alone.