On this page

On this page

Testing and CI

Full Release Validation

The Full Release Validation umbrella, release publish, and Docker Release dispatch. Part of the Release validation workflows index.

Mobile store releases

iOS Store Release (ios-store-release.yml) and Android Store Release (android-store-release.yml) are separate manual workflows. Choose main and click Run workflow with the defaults; no release parameters are needed. The iOS workflow also offers a screenshots operation for capture without an upload. Each platform queues its own runs without cancelling an active upload.

The workflows run the same commands available from a clean, current local main checkout: pnpm ios:release:upload and pnpm android:release:upload. Both freeze the source commit, plan the release using repository versions and current store state, generate platform release notes, then build, sign, and upload. Set OPENAI_API_KEY locally; Actions reads the repository secret with that name. Platform setup, signing, and store credentials are documented in each app's fastlane/SETUP.md.

Release preparation stays outside tracked source files and creates no commits or pull requests. Successful upload records point at the original source SHA under refs/openclaw/mobile-releases/. Actions retains the platform plan, release-notes.json, and available signed binaries for 30 days. Inspect these artifacts and store state after a failure before starting another upload. App Review submission and Android production promotion remain manual.

To preview notes without building or uploading, use a saved platform plan and a new output path:

bash
pnpm mobile:release:notes -- generate --platform ios --plan /path/to/ios-plan.json --output /path/to/preview-notes.json

Use --platform android with android-plan.json for phone and Wear notes. The plan identifies the target version/build, exact source SHA, and public baseline builds. For a historical candidate, add --source-sha <full-commit-sha> matching that plan; the commit must exist locally. Existing output is validated and reused without a model call. Use a new output file to evaluate a different draft.

Generation uses the OpenAI Responses API with structured output, source evidence extraction, and a separate factual review. It covers the app and bundled shared code, including Play-specific sources and Watch RTC. Drafts must fit the store's character limit, and a failed generation or review stops the upload. A model can still omit or misunderstand a change; inspect the saved notes in the run summary before manual public promotion. Notes are reproducible by retaining the artifact, not by expecting a fresh model request to return identical words.

Full Release Validation

Full Release Validation is the manual release umbrella. Every run binds an exact candidate C, qualification Q, and admission/publication P tuple and rejects an expected_sha mismatch before child dispatch. Fresh publication uses Q=C; P is independently trusted and cannot supply substitute tests. Validation SHA maps to the Code SHA for product validation or the Release SHA for changelog-only validation; it is not a third release identity. Beta-publish maps to release_profile=beta with run_release_soak=false. Regular stable releases use release_profile=stable.

See Full release validation for the stage matrix, exact workflow job names, profile differences, the npm-beta-v1 and npm-stable-v1 coverage policies, artifacts, and focused rerun handles.

The normal_ci child dispatches ci.yml with the exact target and release scope, without release_gate. Every selected failure remains blocking, including Windows Node failures. Complete campaigns (rerun_group=all) retain QA Smoke's full scenario profile and Control UI performance independently of changed paths. Docker seed runs all six lanes in every ordinary manual/release scope: cron-mcp-cleanup, fleet-cache, mcp-channels, mcp-code-mode-gateway, published-upgrade-survivor, and update-channel-switch. This includes npm-beta and npm-stable qualification. The survivor uses legacy-operator-state with auto-auth, so the published driver must update the running managed Gateway. Every admitted canonical main run retains this exact combination. Frozen targets retain it when their declared scenario catalog supports legacy-operator-state; historical targets retain base with auto-auth. Missing historical catalogs keep that fallback; malformed or invalid catalogs fail. PRs defer the complete survivor to hourly main and Full Release Validation, while the other Docker seed lanes and QA Smoke retain their owner maps. Ordinary manual/release CI builds the full declaration-complete package. Main and selected PRs, including exact-head release_gate fallbacks, use the existing ciArtifacts profile and canonical packer with --skip-build, retaining the runtime, public SDK declarations, and unchanged tarball integrity check. The hourly protected cache warmer also retains full declaration generation. Hosted manual CI splits QA Smoke into six parts; normal hybrid first attempts use four parts with the same coverage.

For targets with test runtime selection, normal_ci retains the complete Node test inventory and also runs each admitted Bun-compatible selection on Bun. Both results are required; they share existing jobs and execute sequentially within each worker slot. Older targets without this capability retain Node-only testing. This includes the complete Control UI selection on both runtimes, including the retention assertions, when the target's runtime owner admits it. Older targets retain the UI's Node pass and only the Bun selections their runtime owner admits.

Package Acceptance separately retains expanded published-upgrade scenarios: current unpublished candidates include native operator state, and stable/full profiles force the reported-issues soak. Its ordinary survivor restart mode and separate update-restart-auth base scenario do not replace the exact Docker seed combination. Full Release Artifacts and Full Release Candidate prepare the immutable package/image inputs; candidate-phase release checks consume them without moving or weakening that coverage.

Existing frozen-target contracts still apply: Docker seed requires its declared capability, and targets without the Docker tier selector retain the survivor fallback. QA Smoke requires a supported harness, and historical performance checks retain their availability handling. Focused reruns select their requested groups, and validated evidence reuse can reuse completed proof. Regressions outside the remaining automatic owner paths, or in the five release-only Docker seed lanes, can first surface in this manual/release tier.

The live/E2E selected-ref validator fetches the complete commit and ref history with a sparse checkout. Ancestry and release-ref checks remain unchanged, while historical file contents stay out of this metadata-only job. Build and test jobs check out their own complete source trees.

OpenClaw Release Publish is the manual mutating release workflow. Dispatch regular beta and stable publishes from a protected lightweight release-publish/<tooling-sha12>-<epoch> tag at the frozen Tooling SHA after the release tag exists and after the OpenClaw npm preflight has succeeded (the preflight runs pnpm plugins:sync:check among its checks). The tag still selects the exact release commit, including a commit on release/YYYY.M.PATCH. For current validation runs, set preflight_run_id and full_release_validation_run_id to the same successful Full Release Validation run ID and pin full_release_validation_run_attempt. The publisher resolves the independent Full Release Artifacts producer from that validation manifest's sealed publicationArtifacts.npmPreflight descriptor. The producer ID alone does not carry Full Release Validation authorization.

Docker Release requests the docker-release environment approval in a separate job after source identity and image preparation succeed (or prepared artifacts are supplied). Approval waits stay outside docker-release-publish; only the approved publisher enters that global registry/alias lock. It then revalidates the immutable source, prepared artifacts, attestations and alias state before writing. Docker Hub credentials remain required caller-provided secrets. Failed preparation, denied approval and cancellation cannot publish.

Historical recovery may still supply a separate successful OpenClaw NPM Release preflight run ID alongside the matching successful Full Release Validation run and attempt. Create the tooling tag with the release publish commands; real core npm, plugin npm, or ClawHub publication from main is rejected before child dispatch. Docker-only recovery may still use main.

The publisher dispatches Plugin NPM Release for all publishable plugin packages, dispatches Plugin ClawHub Release for the same release SHA, then dispatches OpenClaw NPM Release after plugin npm succeeds. Stable Windows promotion is optional: supply both an exact windows_node_tag and candidate-approved windows_node_installer_digests to dispatch its signed installers after GitHub release finalization. Omit both to skip Windows. For npm-stable evidence, when the tagged apps/android/version.json matches the stable tag's base version, a separate native qualification job starts full CI for the exact release SHA with Android enabled. A successful result is revalidated after core publication before the separate Android job creates its existing approval receipt and dispatches the tag-owned APK workflow. This keeps frozen release tags usable without allowing narrower npm evidence to authorize an unqualified native build. Native failure remains visible and prevents Android approval; core npm and GitHub release finalization do not wait for it. The whole parent can remain active after core publication while native qualification finishes. Existing full evidence and macOS's independent validation retain their native qualification contracts. A mismatched Android pin skips both native qualification and APK publication, with the pin, release train, and Android version pinning (pnpm android:version:pin -- --from-gateway) remedy recorded in the parent summary and release proof. Focused plugin-only repairs use plugin_publish_scope=selected with a nonempty package list. Plugin-only all-publishable runs require the same immutable npm preflight and Full Release Validation evidence as a core publish.

bash
PUBLISH_REF="release-publish/<tooling-sha12>-<epoch>"FRV_RUN_ID="<successful-full-release-validation-run-id>"FRV_RUN_ATTEMPT="<successful-full-release-validation-run-attempt>"gh workflow run openclaw-release-publish.yml \  --ref "$PUBLISH_REF" \  -f tag=vYYYY.M.PATCH-beta.N \  -f preflight_run_id="$FRV_RUN_ID" \  -f full_release_validation_run_id="$FRV_RUN_ID" \  -f full_release_validation_run_attempt="$FRV_RUN_ATTEMPT" \  -f npm_dist_tag=beta

For pinned commit proof on a fast-moving branch, use the helper instead of gh workflow run ... --ref main -f ref=<sha>:

bash
PUBLISHER_SHA="<recorded-full-trusted-main-ancestor-sha>"VALIDATION_SHA="<full-release-candidate-sha>"PUBLICATION_SELECTION='{"route":"normal","npmDistTag":"latest","publishOpenclawNpm":true,"pluginPublishScope":"all-publishable","plugins":[]}'pnpm ci:full-release \  --sha "$VALIDATION_SHA" \  --target-ref release/YYYY.M.PATCH \  --admission-workflow-sha "$PUBLISHER_SHA" \  --admission-workflow-ref main \  --request-file <private-request-file> \  -f validation_purpose=publish \  -f publication_selection_json="$PUBLICATION_SELECTION"

Choose npmDistTag=beta for a beta and route=prepared only for an intended prepared-button consumer. Source admission verifies committed metadata; fresh publish runs also retain separate selected npm and ClawHub registry admission before fanout. Neither grants publication authority. See Dispatch for both contracts. For nonpublish work, explicitly select diagnostic, main-qualification, or postpublish-confidence and omit the publication selection; profile and filters still select the actual coverage.

GitHub workflow dispatch refs must be branches or tags, not raw commit SHAs. The helper first proves GitHub serves the exact Validation SHA by bare-SHA fetch in a fresh temporary repository, including in dry runs. After independent admission, it pushes one immutable release-ci/* workflow ref at Q=C and passes C through ref and expected_sha. Reused evidence must retain its original Q and coverage; every new child workflow headSha must match Q. Record all three roles and never refresh qualification from moving main. Regular release branches accept only their final package version or a matching beta prerelease.

release_profile controls live/provider breadth passed into release checks. The manual release workflows default to stable; use full only when you intentionally want the broad provider/media matrix. Stable and full release checks always run the exhaustive live/E2E and Docker release-path soak; the beta profile can opt in with run_release_soak=true.

fail_fast defaults to false: the umbrella waits for each dispatched child workflow and reports its independent failures together. Set fail_fast=true only when cancelling a child after its first failed job is more useful than the complete failure inventory. In Release Checks, this also enables the Matrix QA CLI's own first-scenario cancellation.

  • beta keeps the fastest OpenAI/core release-critical lanes.
  • stable adds the stable provider/backend set.
  • full runs the broad provider/media matrix.

The umbrella records dispatched child run ids, and Verify full validation checks them during that parent attempt. Parent cancellation or timeout leaves adopted exact children running; cancel one explicitly when it is no longer needed.

For recovery, decide blocker or flake for every failed test, then classify product, harness/tooling/provenance, infrastructure/credential, and wrapper failures before editing. Only confirmed product or qualification-harness failure changes C/Q. Diagnose and fix the owning defect before an explicit narrow rerun_group validation run; never retry a failed test automatically or widen to all. Flakes get at most two explicit same-SHA reruns and a tracked fix on main; record an eligible still-failing job instead of changing tooling, re-cutting, or starting another Full Release Validation. Narrow evidence is not publish authorization by itself.

OpenClaw Release Checks uses the trusted workflow ref to resolve the selected ref once into a release-package-under-test tarball, then passes that artifact to cross-OS checks and Package Acceptance, plus the live/E2E release-path Docker workflow when soak coverage runs. That keeps the package bytes consistent across release boxes and avoids repacking the same candidate in multiple child jobs. Source package preparation has a 35-minute whole-job budget so cold declaration generation and package inventory can finish before artifact upload; downstream jobs still consume only the immutable uploaded package. For the Codex npm-plugin live lane, release checks either pass a matching published plugin spec derived from release_package_spec, pass the operator-supplied codex_plugin_spec, or leave the input blank so the Docker script packs the selected checkout's Codex plugin.

Full Release Validation concurrency is keyed by Validation SHA, Tooling SHA, rerun group, release profile, and effective soak coverage with cancel-in-progress: false. Release Checks uses the same coverage identity in each phase, so beta, stable, and full requests do not queue behind each other. Stable/full always include soak; setting their soak flag explicitly does not create another concurrency group. Parent cancellation does not cancel adopted children.

In the canonical repository's hybrid runner mode, target resolution, evidence reuse, candidate discovery, candidate binding, and candidate resolution use the small Blacksmith runner pool. These serial jobs otherwise compound hosted runner admission delays before tests can start. Other modes and noncanonical repositories retain GitHub-hosted runners; the reusable harness also honors its explicit hosted-runner override. Long-running decision and diagnostic collectors remain hosted.

Was this useful?