On this page
On this page
Testing and CI
Manual dispatches
Manual CI dispatch behavior, release-gate fallbacks, and the Windows Testbox Check. Part of the CI scope and routing index.
Manual dispatches
Ordinary manual CI dispatches run the same job graph as normal CI but force every non-Android scoped lane on: Linux Node shards, bundled-plugin shards, plugin and channel contract shards, Node 24 minimum compatibility, check-*, check-additional-*, built-artifact smoke checks, docs checks, Python skills, Windows, macOS, full iOS build/test and screenshot qualification, and Control UI/native app i18n. Their logical runner profile is always github, independent of the physical fallback selected by runs-on. Node 24 minimum compatibility runs in Full Release Validation and manual dispatches only; push and pull request CI skip it. The exact-head release_gate fallback instead keeps the pull request's macOS, iOS smoke, and generated-native-locale scope without selecting iOS screenshots or native tests. Automatic source PRs and release gates verify native extraction inventory and Android/Apple localization safety without requiring translated or platform-generated output in the same PR. The serialized Native App Locale Refresh workflow rebuilds those artifacts in one isolated PR and enables exact-head auto-merge after required checks pass. Native parity remains blocking for generated-artifact PRs, generated-scope release gates, ordinary manual CI, full-scope release validation, and release prep. CI reports proven obsolete native translation IDs, Android generated rows, and Apple catalog rows as warnings while the locale refresh catches up; active-key coverage, correctness, and all other parity checks remain blocking. See local checks for the exact boundary. Control UI locale parity remains advisory on automatic PR and main runs and blocking on manual/release CI. Standalone manual CI dispatches run Android only with include_android=true (the release_gate input also forces Android); full-scope release validation enables Android by passing include_android=true without setting release_gate; npm qualification scopes defer Android. Plugin prerelease static checks, the full agentic-plugins sweep, the full extension batch sweep, and plugin prerelease Docker lanes are excluded from CI. The Docker prerelease suite runs only when Full Release Validation dispatches the separate Plugin Prerelease workflow with the release-validation gate enabled.
PR baseline ratchets derive their comparison state from the checked-out synthetic merge tree and verify its head parent against the event head. The max-lines entry chains the environment-variable budget with the same fork-point ref before the assertion-safety check, so production source growth cannot first surface on main. Manual runs use a unique concurrency group so a release-candidate full suite is not cancelled by another push or PR run on the same ref. The optional target_ref input lets a trusted caller run that graph against a branch, tag, or full commit SHA while using the workflow file from the selected dispatch ref; ratchet baselines are compared with the target's merge base against the default-branch head resolved for that run. The release_gate input is an exact-SHA maintainer fallback for capacity-stalled PR CI: it requires target_ref to be a full commit SHA that matches the dispatched branch head and pull_request_number to identify the open PR whose merge tree is validated. Release-gate merge-tree lint uses the same five core stripes as hosted PR CI plus one extension stripe, so no single hosted runner owns the full type-aware lint workload.
Ordinary canonical manual CI also retains QA Smoke's full profile and Control UI
performance without owner-path filtering. When the target declares
docker-seed-e2e-contract-v1, it selects published-upgrade-survivor, preserving
the exact legacy-operator-state plus auto-auth proof used by every admitted
canonical main run. Every ordinary manual dispatch adds cron-mcp-cleanup,
mcp-channels, mcp-code-mode-gateway, and update-channel-switch through
resolveDockerSeedLanes, including npm-beta and npm-stable qualification.
Older targets without the tier selector retain the survivor. Ordinary manual CI
builds the full package. The main-only ciArtifacts preparation also applies to admitted main-shape qualification
dispatches, which reproduce the main job for timing comparisons.
Pull requests and exact-head release_gate fallbacks omit Docker seed and QA Smoke.
Full Release Validation reaches these lanes through its normal CI child without
setting release_gate; frozen targets retain their existing capability checks.
Gateway extended-stable shared publication requires complete candidate-owned
Full Release Validation from an independently admitted release-ci/* harness
at Q=C, targeting the frozen extended-stable/YYYY.M.33 tip. The raw diagnostic
example above and direct main producers do not satisfy that publication
contract. Current
manifests also supply qualified npm preflight artifacts. The shared
OpenClaw Release Publish parent dispatches from a protected lightweight
release-publish/<sha12>-<epoch> tag at the frozen trusted-main Tooling SHA and
uses npm_dist_tag=extended-stable to publish official npm plugins and core, attach evidence, publish Docker, and
finalize a non-Latest GitHub Release. Only extended-stable* container aliases
advance; ClawHub, native-app, website, regular npm latest, and private
dist-tag surfaces are excluded. Core-resume recovery verifies existing registry
bytes before resuming evidence and finalization; Docker-only recovery leaves
GitHub finalization untouched. See Monthly Gateway extended-stable
publication
for commands and recovery.
Windows Testbox Check
The manual windows-testbox-probe.yml workflow keeps Windows/WSL checking and
headless Windows CI on the selected runner_label. The run_windows_ci input
(default false) requests both headless CI and a separate native Scheduled Task
proof job on GitHub-hosted windows-2025 when no installed package binding is
supplied. Neither job depends on the other, so
their results remain independently visible; either requested proof failing fails
the workflow.
Set skip_defender_exclusions=true to leave Windows Defender policy unchanged
throughout the workflow, including native and installed Scheduled Task proof jobs.
This skips only the workspace and Node process exclusions; proof selection,
isolation checks, native lifecycle tests, cleanup, and evidence upload remain
unchanged. The default is false, preserving the existing best-effort exclusions
for Windows CI and exact replay.
For both proofs, set target_ref to an exact 40-character commit SHA. Both jobs
check out that target, and native proof verifies checkout equality before running
the lifecycle test. Native preflight runs before setup and requires an interactive
Windows session. A noninteractive runner fails qualification rather than silently
skipping proof. Selecting windows-2025 does not establish native qualification:
the unchanged lifecycle assertions and cleanup must pass on the actual runner.
Cleanup and diagnostic upload still run after failure, and retained evidence is
removed only after cleanup and upload succeed.
Installed Scheduled Task upgrades
Set run_windows_ci=true with one installed_startup_package binding from a
successful Package Acceptance run, runner_label=windows-2025, and
keepalive_minutes=0. Dispatch the reviewed workflow revision and use its exact
SHA as target_ref; the binding independently pins the product package. Leave
the other proof modes off. With an empty package binding, the independent
source-only CI and native jobs above retain their behavior.
The existing package resolver authenticates the candidate, then passes a verified
artifact within the same workflow run to the native job. That job retains
contents: read permissions. It runs a fresh installed candidate and the unchanged
published 2026.9.3 and 2026.9.4 CLI updaters against the candidate. Each upgrade
also preserves a separate running profile under its canonical Task name. Live
status handshakes must match the verified package version and build ID; the
updated Gateway must have a new PID, while the peer retains its PID and baseline
identity. Candidate update preview and deep Doctor discovery inspect actual
registered Task actions, including a disabled custom-named Gateway CMD task
reported as an extra and an owned disabled task with a missing launcher. The fresh cell also inspects a disabled
direct executable action and verifies that packaged update preview leaves its
unsupported automatic service management unavailable. This proves read-only
inspection and preservation, not direct-action execution or update mutation.
Cells prepare their own normal npm installations in sequence. Actual Task and descendant cleanup and immutable proof upload must succeed before disposable prefixes and caches are retired or the next cell starts. A failed cell retains its evidence and stops the sequence; forced command cleanup cannot establish a passing natural-exit result. The final artifact requires all three cells.
The fixture checks actual free space after tooling setup against provisional 13 GiB fresh-install and 29 GiB published-pair allowances. These are planning allowances, not measured package bounds or runner capacity. It records space and owned allocations at lifecycle boundaries; transient peaks between observations remain unknown. Existing command, test, step, and job deadlines remain unchanged.
Exact Windows test replay
For an ordered diagnostic from a recorded CI failure, set windows_ci_replay
to a JSON object with nodeVersion, packageManager, vitestVersion,
maxWorkers, files, and projects. Use an exact Node 24 patch and the
checkout's complete pnpm integrity pin and Vitest version. maxWorkers is an
integer from 1 through 4. files is the original ordered array of literal,
tracked test paths under src/, test/, extensions/, or packages/;
projects is the original ordered array of
test/vitest/vitest.<name>.config.ts paths. Globs, shell text, arbitrary CLI
arguments, and environment overrides are not accepted.
Set target_ref to the exact source SHA, select the original runner_label,
set keepalive_minutes=0, and leave all other proof modes off. This runs only
the existing scripts/test-projects.mts entrypoint with --fileParallelism,
one project process at a time, the requested worker count, and the original
Windows CI heap and extension-shard settings. Normal CI worker policy is
unchanged. Runtime preparation and process lifetime stay with the frozen
source's dispatcher; the workflow does not copy current test tooling into it.
Run baseline and failing sources as separate, serialized workflow invocations at the same reviewed workflow revision. Each gets its own runner checkout and frozen install; never switch sources in an active checkout. Record actual CPU/RAM from both runs rather than inferring capacity from the runner label.
The windows-ci-replay-<runId>-<attempt> artifact retains the input, source and
workflow identities, dependency hashes, native runtime/resources, exact argv,
combined output, process status, observed project order, and retained-namespace
diagnostics. A successful test command without the complete expected project
sequence fails qualification. An earlier failure retains its partial sequence
and remains failed. No replay retries or pass/fail waivers are added.
There is no persistent Testbox lease, SSH setup, or keepalive. The existing test owner handles ordinary lifetime; Actions owns final job/runner teardown. A frozen Windows runner can retain temporary namespaces when descendant settlement is unverified. Do not delete them during the run or infer settlement from a zero process status. Preserve that diagnostic, inspect final runner cleanup, and report any missing teardown evidence separately from the test result.
Installed repair-worker compatibility and cleanup
Set installed_repair_worker=true with one installed_startup_package binding,
runner_label=windows-2025, and keepalive_minutes=0. Pin target_ref to the
same reviewed commit as the dispatched workflow. Leave other proof modes off.
The existing package owner installs and verifies the exact candidate artifact;
the native admission gate requires a fresh hosted runner without credentials,
operator mounts, Tailnet attachment, or managed identity.
The check authenticates and installs npm versions 2026.9.4 and 2026.9.5, then calls their unchanged published repair controllers against the installed candidate worker. Version 2026.9.4 delegates its verifying phase; 2026.9.5 also delegates validation. Each must receive the deferred unavailable result without provider requests, validation calls, or changes to synthetic state. This proves controller compatibility, not a complete installed-updater upgrade.
Separate cells use the candidate's packaged executor and ledger owners to admit real delegated work against a loopback model fixture and reject a wrong receiver. A real tool starts descendant processes. Both normal and forced worker exits must remove them before the parent executor settles, while the outer observer and its Windows Job launcher remain alive. Parent or outer cleanup cannot satisfy that assertion. Original 90-second worker, 60-second turn, 120-second cell, and two-second extinction deadlines remain fixed.
The windows-installed-startup-<runId>-<attempt> artifact retains
repair-results.json, the failed or completed cells, native Job observations,
PID/start identities, exact package/controller/runtime/tooling hashes, synthetic
provider counts, state effects, and final cleanup. The check stops after a failed
cell and never substitutes successful runner teardown for worker qualification.
Installed Gateway startup measurements
The same workflow can measure one immutable npm package on the selected Windows
runner. Set target_ref to the full tooling commit, run_windows_ci=false,
keepalive_minutes=0, and startup_node_version to an exact Node version
(default 26.9.0). Leave WSL and Defender inputs at their defaults. The optional
installed_startup_package input is a JSON object with runId, runAttempt,
workflowSha, artifactId, artifactDigest, packageSha256, and sourceSha.
Use the immutable package-under-test-<runId>-<runAttempt> artifact from a
successful Package Acceptance run. The workflow verifies its producer and
artifact metadata, resolves it through the package-candidate owner, and installs
and rebuilds with normal npm lifecycle scripts.
After the benchmark's lifecycle fixtures pass on Windows, the installed
openclaw.mjs runs once with new synthetic state, then eight more times with that
same state. Each sample records HTTP readiness, first status and health RPC
responses, and acknowledged graceful shutdown. An outer managed Windows Job
contains the controller and all descendants; final success requires both clean
Gateway shutdown and observed descendant settlement before forced Job cleanup.
No synchronous process sampler or startup profiler runs during measurement.
The windows-installed-startup-<runId>-<runAttempt> artifact retains all nine
sample slots, errors, package/runtime/helper hashes, source and tooling commits,
runner hardware, the raw installed npm lockfile, and cleanup evidence. A streamed
cohort.log retains the active PID, phase, child output, and completed check/RPC
observations even if cancellation prevents the final sample checkpoint. Synthetic databases and compile caches
stay in the runner's temporary directory. A failed or interrupted cohort has no
established summary. “Fresh” means new state, not a cold filesystem; dedicated
runner results establish a new baseline and do not establish a speedup relative
to a different desktop. Health RPC success is separate from recorded plugin
availability and degraded diagnostics.
For a matched comparison, pass installed_startup_package as
{"baseline": <package-binding>, "candidate": <package-binding>}. Both bindings
use the same fields above. The workflow resolves and normally installs both
packages on one runner before measurement. Their complete npm lock records must
match, except the independently verified OpenClaw tarball references and integrity.
Any dependency drift stops the comparison before a Gateway starts; both raw locks
remain evidence. The packages must have the same version and dependency graph.
Each package has its own immutable install and synthetic state/cache directory. The cohort plans 18 slots: fresh baseline then fresh candidate, followed by eight restart pairs alternating baseline/candidate and candidate/baseline order. Both arms retain their own state across restarts. There are no discarded warmups or replacement samples. A failure stops the cohort, retains the remaining unrun slots, and invalidates the comparison summary. The per-sample deadlines stay the same; the outer lifecycle budget scales from 30 to 60 minutes for two arms.
After all samples and descendants settle, the comparison reports each arm's established readiness and absolute first status/health completion, plus paired candidate-minus-baseline differences. Negative differences favor the candidate. Fresh samples remain separate. Alternating order reduces time-order bias; it does not make the filesystem cold or establish performance on other machines.
For CPU attribution, set installed_startup_cpu_diagnostic=true with one package
binding. This separate mode runs one unprofiled fresh prime, then one native CPU
profile of an established launch using the same synthetic state. Both launches
still require readiness, first status/health requests, acknowledged shutdown,
and descendant settlement. It produces no timing summary or paired comparison.
The artifact retains raw .cpuprofile files, the main process/thread identity,
preload clock observations, startup trace metrics, config hashes, and backup-existence
facts. The preload's monotonic and performance-clock observations remain available
for calibration within the Gateway process. Controller, Gateway, and native-profile
timestamps have separately identified clock domains; their alignment is not
established. Do not compare absolute values across those domains or assume that a
profile's last sampled timestamp includes the preload's exit observation.
Console and stream trace output share the diagnostic observer, including under Bun.
Profiling and trace observation add overhead; main-isolate samples do not
account for unprofiled child or Worker CPU. Ordinary timing cohorts remain
uninstrumented.