On this page
On this page
Testing and CI
Full Release Validation
The Full Release Validation umbrella, release publish, and Docker Release dispatch. Part of the Release validation workflows index.
Mobile store releases
iOS Store Release (ios-store-release.yml) and Android Store Release
(android-store-release.yml) are separate manual workflows. Choose main and
click Run workflow with the defaults; no release parameters are needed.
The iOS workflow also offers a screenshots operation for capture without an
upload. Each platform queues its own runs without cancelling an active upload.
The workflows run the same commands available from a clean, current local
main checkout: pnpm ios:release:upload and pnpm android:release:upload.
Both freeze the source commit, plan the release using repository versions and
current store state, generate platform release notes, then build, sign, and upload. Set
OPENAI_API_KEY locally; Actions reads the repository secret with that name.
Platform setup, signing, and store credentials are documented in each app's
fastlane/SETUP.md.
Release preparation stays outside tracked source files and creates no commits
or pull requests. Successful upload records point at the original source SHA
under refs/openclaw/mobile-releases/. Actions retains the platform plan,
release-notes.json, and available signed binaries for 30 days. Inspect these
artifacts and store state after a failure before starting another upload.
App Review submission and Android production promotion remain manual.
To preview notes without building or uploading, use a saved platform plan and a new output path:
Use --platform android with android-plan.json for phone and Wear notes. The
plan identifies the target version/build, exact source SHA, and public baseline
builds. For a historical candidate, add --source-sha <full-commit-sha> matching
that plan; the commit must exist locally. Existing output is validated and reused
without a model call. Use a new output file to evaluate a different draft.
Generation uses the OpenAI Responses API with structured output, source evidence extraction, and a separate factual review. It covers the app and bundled shared code, including Play-specific sources and Watch RTC. Drafts must fit the store's character limit, and a failed generation or review stops the upload. A model can still omit or misunderstand a change; inspect the saved notes in the run summary before manual public promotion. Notes are reproducible by retaining the artifact, not by expecting a fresh model request to return identical words.
Full Release Validation
Full Release Validation is the manual release umbrella. Every run binds an
exact candidate C, qualification Q, and admission/publication P tuple and rejects
an expected_sha mismatch before child dispatch. Fresh publication uses Q=C;
P is independently trusted and cannot supply substitute tests. Validation SHA maps to the Code SHA for product
validation or the Release SHA for changelog-only validation; it is not a third
release identity. Beta-publish maps to release_profile=beta with
run_release_soak=false. Regular stable releases use release_profile=stable.
See Full release validation for the
stage matrix, exact workflow job names, profile differences, the npm-beta-v1
and npm-stable-v1 coverage policies, artifacts, and focused rerun handles.
The normal_ci child dispatches ci.yml with the exact target and release scope,
without release_gate. Every selected failure remains blocking, including
Windows Node failures.
Complete campaigns (rerun_group=all) retain QA Smoke's
full scenario profile and Control UI performance independently of changed paths.
Docker seed runs all six lanes in every ordinary manual/release scope:
cron-mcp-cleanup, fleet-cache, mcp-channels, mcp-code-mode-gateway,
published-upgrade-survivor, and update-channel-switch. This includes
npm-beta and npm-stable qualification. The survivor uses legacy-operator-state
with auto-auth, so the published driver must update
the running managed Gateway. Every admitted canonical main run retains this
exact combination. Frozen targets retain it when their declared scenario catalog
supports legacy-operator-state; historical targets retain base with auto-auth.
Missing historical catalogs keep that fallback; malformed or invalid catalogs fail.
PRs defer the complete survivor to hourly main and Full Release Validation, while
the other Docker seed lanes and QA Smoke retain their owner maps.
Ordinary manual/release CI builds the full declaration-complete package. Main and
selected PRs, including exact-head release_gate fallbacks, use the
existing ciArtifacts profile and canonical packer with --skip-build, retaining
the runtime, public SDK declarations, and unchanged tarball integrity check.
The hourly protected cache warmer also retains full declaration generation.
Hosted manual CI splits QA Smoke into six parts; normal hybrid first attempts use four parts
with the same coverage.
For targets with test runtime selection,
normal_ci retains the complete Node test inventory and also runs each admitted
Bun-compatible selection on Bun. Both results are required; they share existing
jobs and execute sequentially within each worker slot. Older targets without this
capability retain Node-only testing.
This includes the complete Control UI selection on both runtimes, including the
retention assertions, when the target's runtime owner admits it. Older targets
retain the UI's Node pass and only the Bun selections their runtime owner admits.
Package Acceptance separately retains expanded published-upgrade scenarios:
current unpublished candidates include native operator state, and stable/full
profiles force the reported-issues soak. Its ordinary survivor restart mode
and separate update-restart-auth base scenario do not replace the exact Docker
seed combination. Full Release Artifacts and Full Release Candidate prepare
the immutable package/image inputs; candidate-phase release checks consume them
without moving or weakening that coverage.
Existing frozen-target contracts still apply: Docker seed requires its declared capability, and targets without the Docker tier selector retain the survivor fallback. QA Smoke requires a supported harness, and historical performance checks retain their availability handling. Focused reruns select their requested groups, and validated evidence reuse can reuse completed proof. Regressions outside the remaining automatic owner paths, or in the five release-only Docker seed lanes, can first surface in this manual/release tier.
The live/E2E selected-ref validator fetches the complete commit and ref history with a sparse checkout. Ancestry and release-ref checks remain unchanged, while historical file contents stay out of this metadata-only job. Build and test jobs check out their own complete source trees.
OpenClaw Release Publish is the manual mutating release workflow. Dispatch
regular beta and stable publishes from a protected lightweight
release-publish/<tooling-sha12>-<epoch> tag at the frozen Tooling SHA after the
release tag exists and after the OpenClaw npm preflight has succeeded (the preflight runs
pnpm plugins:sync:check among its checks). The tag still selects the exact
release commit, including a commit on release/YYYY.M.PATCH. For current validation runs,
set preflight_run_id and full_release_validation_run_id to the same successful
Full Release Validation run ID and pin full_release_validation_run_attempt.
The publisher resolves the independent Full Release Artifacts producer from
that validation manifest's sealed publicationArtifacts.npmPreflight descriptor.
The producer ID alone does not carry Full Release Validation authorization.
Docker Release requests the docker-release environment approval in a
separate job after source identity and image preparation succeed (or prepared
artifacts are supplied). Approval waits stay outside docker-release-publish;
only the approved publisher enters that global registry/alias lock. It then
revalidates the immutable source, prepared artifacts, attestations and alias
state before writing. Docker Hub credentials remain required caller-provided
secrets. Failed preparation, denied approval and cancellation cannot publish.
Historical recovery may still supply a separate successful OpenClaw NPM Release
preflight run ID alongside the matching successful Full Release Validation run
and attempt. Create the tooling tag with the release publish commands;
real core npm, plugin npm, or ClawHub publication from main is rejected before
child dispatch. Docker-only recovery may still use main.
The publisher dispatches Plugin NPM Release for all
publishable plugin packages, dispatches Plugin ClawHub Release for the same
release SHA, then dispatches OpenClaw NPM Release after plugin npm succeeds.
Stable Windows promotion is optional: supply both an exact windows_node_tag
and candidate-approved windows_node_installer_digests to dispatch its signed
installers after GitHub release finalization. Omit both to skip Windows.
For npm-stable evidence, when the tagged apps/android/version.json matches
the stable tag's base version, a separate native qualification job starts full
CI for the exact release SHA with Android enabled. A successful result is revalidated
after core publication before the separate Android job creates its existing
approval receipt and dispatches the tag-owned APK workflow. This keeps frozen
release tags usable without allowing narrower npm evidence to authorize an
unqualified native build. Native failure remains visible and prevents Android
approval; core npm and GitHub release finalization do not wait for it. The whole
parent can remain active after core publication while native qualification
finishes. Existing full evidence and macOS's independent validation retain their
native qualification contracts. A mismatched Android pin skips both native
qualification and APK publication, with the pin, release train, and Android
version pinning (pnpm android:version:pin -- --from-gateway) remedy recorded
in the parent summary and release proof.
Focused plugin-only repairs use plugin_publish_scope=selected with a nonempty
package list. Plugin-only all-publishable runs require the same immutable npm
preflight and Full Release Validation evidence as a core publish.
For pinned commit proof on a fast-moving branch, use the helper instead of
gh workflow run ... --ref main -f ref=<sha>:
Choose npmDistTag=beta for a beta and route=prepared only for an intended
prepared-button consumer. Source admission verifies committed metadata; fresh
publish runs also retain separate selected npm and ClawHub registry admission
before fanout. Neither grants publication authority. See
Dispatch for both contracts.
For nonpublish work, explicitly select
diagnostic, main-qualification, or postpublish-confidence and omit the
publication selection; profile and filters still select the actual coverage.
GitHub workflow dispatch refs must be branches or tags, not raw commit SHAs. The
helper first proves GitHub serves the exact Validation SHA by bare-SHA fetch in a
fresh temporary repository, including in dry runs. After independent admission,
it pushes one immutable release-ci/* workflow ref at Q=C and passes C through
ref and expected_sha. Reused evidence must retain its original Q and coverage;
every new child workflow headSha must match Q. Record all three roles and never
refresh qualification from moving main. Regular release branches accept only their final package
version or a matching beta prerelease.
release_profile controls live/provider breadth passed into release checks. The
manual release workflows default to stable; use full only when you
intentionally want the broad provider/media matrix. Stable and full
release checks always run the exhaustive live/E2E and Docker release-path soak;
the beta profile can opt in with run_release_soak=true.
fail_fast defaults to false: the umbrella waits for each dispatched child
workflow and reports its independent failures together. Set fail_fast=true
only when cancelling a child after its first failed job is more useful than the
complete failure inventory. In Release Checks, this also enables the Matrix QA
CLI's own first-scenario cancellation.
betakeeps the fastest OpenAI/core release-critical lanes.stableadds the stable provider/backend set.fullruns the broad provider/media matrix.
The umbrella records dispatched child run ids, and Verify full validation
checks them during that parent attempt. Parent cancellation or timeout leaves
adopted exact children running; cancel one explicitly when it is no longer
needed.
For recovery, decide blocker or flake for every failed test, then classify product, harness/tooling/provenance,
infrastructure/credential, and wrapper failures before editing. Only confirmed
product or qualification-harness failure changes C/Q. Diagnose and fix the owning defect before an explicit narrow rerun_group
validation run; never retry a failed test automatically or widen to all.
Flakes get at most two explicit same-SHA reruns and a tracked fix on main;
record an eligible still-failing job instead of changing tooling, re-cutting,
or starting another Full Release Validation. Narrow evidence is not publish authorization by itself.
OpenClaw Release Checks uses the trusted workflow ref to resolve the selected ref once into a release-package-under-test tarball, then passes that artifact to cross-OS checks and Package Acceptance, plus the live/E2E release-path Docker workflow when soak coverage runs. That keeps the package bytes consistent across release boxes and avoids repacking the same candidate in multiple child jobs. Source package preparation has a 35-minute whole-job budget so cold declaration generation and package inventory can finish before artifact upload; downstream jobs still consume only the immutable uploaded package. For the Codex npm-plugin live lane, release checks either pass a matching published plugin spec derived from release_package_spec, pass the operator-supplied codex_plugin_spec, or leave the input blank so the Docker script packs the selected checkout's Codex plugin.
Full Release Validation concurrency is keyed by Validation SHA, Tooling SHA,
rerun group, release profile, and effective soak coverage with
cancel-in-progress: false. Release Checks uses the same coverage identity in
each phase, so beta, stable, and full requests do not queue behind each other.
Stable/full always include soak; setting their soak flag explicitly does not
create another concurrency group. Parent cancellation does not cancel adopted
children.
In the canonical repository's hybrid runner mode, target resolution, evidence
reuse, candidate discovery, candidate binding, and candidate resolution use
the small Blacksmith runner pool. These serial jobs otherwise compound hosted
runner admission delays before tests can start. Other modes and noncanonical
repositories retain GitHub-hosted runners; the reusable harness also honors
its explicit hosted-runner override. Long-running decision and diagnostic
collectors remain hosted.