On this page

On this page

Testing and CI

Scheduled and maintenance workflows

PR vs hourly vs release

Ordinary pull requests, including fork contributions, no longer select these expensive jobs in ci.yml:

  • Real-Gateway Control UI E2E (mocked/bundled checks-ui-e2e remains owner-selected).
  • Windows Node tests, macOS Node tests, macOS Swift jobs, and iOS build/simulator smoke.
  • The published-npm-driver × candidate update cell, including on update-owner PRs.
  • Dependency/dead-export scanning (Knip) and the full runtime topology/architecture job. Runtime import-cycle checks remain in the existing PR guard; TypeScript changes also select Madge there. TypeScript changes under src/, extensions/, or packages/ also select Kysely guardrails in that same job; generated Kysely types remain deferred. See scope selection.
  • Android screenshot capture.

The first four groups retain their existing hourly main-tier and full release inventories. Android screenshots were already excluded from hourly main and continue in full manual/release validation. Ordinary manual dispatch and exact-head release_gate fallback behavior are unchanged. SwiftLint and SwiftFormat are part of the moved Apple jobs; they are not split into new PR jobs. Other source tests, static correctness gates, Android unit/lint work, and independent native localization checks keep their existing PR selection.

This moves frequent PR critical paths to their existing later tiers without changing what the jobs execute. Failures unique to those jobs can therefore appear after merge. openclaw/ci-gate requires only the jobs the manifest selected; its required-check identity and repository protection are unchanged. See scope selection for the tier table and owner boundaries.

Hourly main CI

The complete main validation tier runs directly from ci.yml at minute 23 of each hour. GitHub's scheduled event selects the upstream main revision; manual dispatch inputs cannot claim scheduled-run policy. The schedule selects the complete main tier, including Android, without filtering to the last commit. Node, native platforms, docs, QA Smoke, browser process proofs, and the published-updater survivor all run against that revision. Node tests use the complete compact inventory, including tooling and PR-exempt files, within its 77-row main-tier cap.

The existing Plugin Prerelease workflow owns the complete extension runtime inventory separately, at minute 37 each hour. Scheduled runs pin the scheduled upstream main SHA and select only their extension matrix and required summary. Manifest-only bundled plugins and tests directly under extensions/ use its existing file-shard execution path; package-backed plugins retain their existing batch owners. The existing twelve-job concurrency limit stays unchanged. One non-canceling hourly slot lets active proof finish while GitHub coalesces pending tips; manual and release runs retain independent concurrency groups and all existing phases. Normal CI no longer appends a second, partial extension inventory. Inspect both CI and Plugin Prerelease for hourly coverage; Full Release Validation pins both existing children to its exact target.

Full Release Validation and ordinary manual CI retain validation_tier=full by default. They additionally run release-only runtime/UI tests, minimum-Node compatibility, iOS screenshots, native Release builds, Android packaging, and all six Docker seed scenarios. Hourly iOS retains ios-build (tests): Swift lint, Rust tests, voice cleanup, native Access, and the complete focused app/notification lifecycle inventory. Managed attachment UI/export proof, Watch operation simulator suites, and Watch delivery UI proof run only in full-tier manual/release validation, with every case and assertion retained there. Android retains phone/Wear tests and lint. The Docker survivor uses the existing main smoke package, including runtime, assets, public SDK declarations, and tarball integrity. The manual SDK API diff report stays manual-only: a scheduled tip has no change range to compare.

Main-tier iOS builds target the selected simulator's native architecture and disable compiler indexing, as PR smoke already does. Voice and lifecycle tests disable Xcode's verbose diagnostic collection while retaining their logs and xcresult bundles. Full-tier manual/release validation keeps universal simulator builds and failure diagnostics. This removes duplicate architecture work and diagnostic stalls observed in 76–94-minute hourly jobs; it does not establish a new completion bound.

Scheduled CI uses automatic-main runner placement, including hybrid placement on the first attempt when configured. Native runner labels, worker limits, and retry fallbacks remain unchanged. Control UI and native translation source checks stay mandatory; generated locale drift is advisory because the post-merge translation workflows own its repair. Ordinary manual runs, including validation_tier=main, retain strict locale parity.

A main-tier run does not emit the full-validation revision confirmation. Docs Agent's automatic write gate requires that full-tier receipt; its explicit manual dispatch remains available.

Inspect the scheduled CI run and its openclaw/ci-gate job directly. Each scheduled run starts independently so an older iOS simulator phase cannot hold the next hourly core checks. Only scheduled ios-build jobs share a non-canceling slot: the active proof finishes while GitHub replaces a pending iOS job when another arrives. Arrival order need not match revision order. At the aggregate owner, openclaw/ci-gate accepts a selected ios-build result of cancelled only for scheduled openclaw/openclaw runs on main, and emits an “Hourly iOS proof coalesced” notice. A real iOS failure or unexpected skip still fails the gate, as does cancellation of another selected lane. Manual and PR iOS cancellations retain their existing failure policy. A passing gate with this notice delegates iOS proof to a later scheduled job; it does not validate iOS at the canceled revision. GitHub's workflow-level conclusion can still be cancelled even when the aggregate succeeds. Manual/release CI stays independent, and security-only pushes cannot cancel scheduled work. CI remains available during release validation; OPENCLAW_RELEASE_PRIORITY_RUN does not control admission.

Hourly CI owns docs checks, including RunsOn routing. The standalone Docs workflow retains manual and opted-in push runs.

Node Runtime Conformance and Plugin Init Scaffold Validation check for changed inputs hourly at minute 23. They reuse proof only when the required jobs passed on the same branch within 24 hours and the complete input diff is unchanged. Skipped jobs cannot advance the comparison base. Missing proof, incomplete diffs, or API errors request fresh validation. Each decision reports its reason. Both workflows retain their PR scopes and refresh proof daily for dependency drift.

Sandbox Common Smoke runs daily at 05:23 UTC and retains PR and manual runs. Plugin NPM Release runs its nonpublishing metadata and unpublished-package pack preview hourly. Schedules cannot enter its manual-only approval or publication jobs. Vitest Cache Warm runs at minute 17 of every hour, retaining its manual and repository-dispatch recovery paths. The warmer is independent. Its completion before CI is not guaranteed.

Restore per-push CI

Set the repository Actions variable OPENCLAW_CI_ON_PUSH to true under Settings → Secrets and variables → Actions → Variables. This restores the previous path-filtered full main-push admission in CI, the standalone checks, plugin artifact preview, and cache warming. GitHub string comparisons are case-insensitive; use the documented lowercase true. Unset, empty, false, and other values keep hourly-only main-tier CI. Delete the variable or set it to false to return to the default. Hourly runs remain enabled either way. No secret, commit, or protection-setting change is required.

For an immediate main-tier CI run, choose CI → Run workflow → main, select the main validation tier and Android, or run:

bash
gh workflow run ci.yml --ref main -f validation_tier=main -f include_android=true

The individual standalone workflows can also be run manually. Direct CI dispatches and release-validation children retain their existing inputs and behavior; set include_android=true when requesting complete platform coverage. To measure hourly coverage on a development branch, dispatch ci.yml on that branch with validation_tier=main and include_android=true. The main tier requires the workflow and checkout to share one revision; it cannot qualify a frozen release target or replace an exact-head PR release gate.

What stays on pushes

CodeQL retains all seven main-push security categories. CI retains security-fast (committed private keys and changed-workflow security auditing) on its existing non-docs push scope. Pull-request, push, and scheduled CI skip production dependency auditing; only release dispatches run it, as a warning. Default main pushes also run the baseline-growth, assertion-safety, test timeout race, and new protocol-method metadata guards there against the exact push before SHA, so scheduled CI's main-against-itself comparison cannot lose these checks; Workflow Sanity checks tracked conflict markers on every admitted push. Its workflow lint and security tools run only when workflow, action, or lint policy inputs change. The full CI aggregate job is skipped on default main pushes, not on runnable PRs or full manual runs. CodeQL and Workflow Sanity also remain available during release validation. PR required-check names and security-review enforcement are unchanged.

Publishing and its prerequisite checks stay event-driven: docs mirror and website installer synchronization, runner-image publication, locale-generation PRs, and ClawSweeper activity forwarding retain their existing admission. Stable main closeout runs when release inputs change. The existing pnpm release:stable closeout phase waits for successful publication, verifies main, and dispatches closeout. With saved orchestrator state, resume it with pnpm release:stable YYYY.M.PATCH --from closeout.

If you publish directly from the Actions UI after the main forward-port, dispatch closeout after Release Publish succeeds and main carries the shipped version and changelog:

bash
gh workflow run openclaw-stable-main-closeout.yml --ref main -f tag=vYYYY.M.PATCH

Unrelated source pushes no longer poll for release completion. Manual recovery retains its existing evidence checks. Docs Agent verifies the exact successful full-tier CI attempt before admitting its automatic write job. Opted-in full main pushes qualify; hourly main-tier runs and security-only pushes do not. With OPENCLAW_CI_ON_PUSH unset, automatic Docs Agent writes are therefore disabled. Explicit non-bot Docs Agent dispatch remains available, and its current-main and hourly cadence guards remain in place for eligible workflow-run invocations. Security Review still handles PR and manual CI completion. Release/tag and PR-only workflows retain their existing triggers; this change adds no merge-queue support where none existed and changes no repository rulesets.

GitHub cron is best-effort on the default branch, not a one-hour latency SLA: runs may be delayed or dropped under load, and public-repository schedules can be disabled after inactivity. Main-tier CI deliberately rechecks unchanged SHAs rather than introducing a separate last-success ledger. A failure can be retried at the next hourly opportunity, and manual dispatch remains available. If iOS proof takes longer than an hour, later hourly runs can finish their other checks while waiting for that slot. A superseded pending iOS job can leave its aggregate green with delegated iOS proof, but cannot qualify Docs Agent. Those completed checks still consume runner time; per-run worker limits do not bound concurrent hourly runs together. The slot does not cover manual/PR iOS jobs or runs admitted by an older workflow. This removes workflow admission blocking, not runner-capacity waits. No measured cost savings or strict completion interval is claimed.

Weekly mobile store releases

iOS Store Release and Android Store Release run every Monday at 11:00 AM Pacific (0 11 * * 1, America/Los_Angeles). GitHub adjusts for daylight saving time. Each run pins the scheduled main commit and follows the same qualification, screenshots, signing, and upload path as the manual release operation. Existing platform concurrency queues serialize daily, weekly, and manual releases without cancelling active runs. Apple App Review submission and Google Play production promotion remain manual.

The separate daily 7:00 AM Pacific TestFlight/internal schedules retain their IOS_TESTFLIGHT_ENABLED and ANDROID_INTERNAL_ENABLED activation variables. Those variables do not disable weekly store releases. Store runs use the existing ios-store-release and android-store-release environments and their credentials; environment approval rules still apply.

Mobile Store Release Notifications observes completed weekly attempts and posts one short status message per platform with the run link. Successful runs say iOS release uploaded successfully or Android release uploaded successfully; other outcomes say iOS release needs attention or Android release needs attention. Manual store releases, daily distributions, and screenshot-only runs stay silent. A rerun posts its own result with the attempt number. Reporting success requires a successful run and a successful upload job and step; it does not mean the app is publicly available. Cancellation, timeout, skipped releases, and unconfirmed outcomes report that the release needs attention. Details come from job/step metadata, never raw logs. If the combined preparation/upload command fails, store acceptance is unconfirmed; the notifier does not infer whether an inner upload completed.

Before enabling notifications, create a Discord webhook in the destination text channel and save its URL as DISCORD_MOBILE_RELEASE_WEBHOOK_URL in each of the ios-store-release and android-store-release GitHub environments. Each notification job enters its matching environment and receives only the webhook secret; it does not access signing assets or run release code. It checks out trusted default-branch notification code, reads only the completed attempt's job metadata, disables mentions and link embeds, and waits for Discord to confirm message creation. Transient delivery errors receive bounded retries; an ambiguous network failure can produce a duplicate message. Exhausted retries or a missing webhook fail only the notification workflow, never rerun an upload.

The schedule is best-effort: GitHub can delay or drop a scheduled run. There is no missed-run watchdog, and no completion notification if a run never starts. Deploying these workflow changes enables the next Monday occurrence; no manual release is needed to activate it.

Nightly Full Release Validation

Full Release Validation Nightly (full-release-validation-nightly.yml) runs every 3 hours at minute 7 UTC (7 */3 * * *) so release readiness stays current as commits land, with the stable profile, soak and blocking performance, reuse_evidence=true, rerun_group=all, and main-qualification purpose. It checks out the scheduler's exact main SHA and runs the SHA-pinned helper (pnpm ci:full-release --sha <sha> --workflow-sha <sha> --trusted-workflow-ref main), explicitly retaining the scheduled, non-publishing main-qualification route. It uses that SHA as both Validation and Tooling SHA and dispatches from an immutable release-ci/<sha12>-<id> transport ref. A raw dispatch from main fails once main moves, because the parent refuses to dispatch children from a moved workflow ref. A still-active parent for the same SHA shares the SHA-specific Full Release Validation concurrency group and queues this dispatch; a completed one is validated again and adopts its own exact-target evidence through reuse. The parent automatically uses OPENCLAW_RELEASE_RUNNER_GROUP when configured. The scheduled job runs on the same runner selection and 720-minute budget as the parent's release_decision waiter, matching the helper's 720-minute watch.

The job watches the parent through its Release Decision and evidence verification, so its conclusion is the validation result. One concurrency group covers scheduled and manual runs without cancellation: while a job still watches its parent, the next trigger waits, and GitHub keeps only the newest pending run, so runs never overlap and the latest waiting trigger's SHA runs next. Child reuse is exact-target: when main has not moved since the previous run, reuse_evidence=true lets each child adopt that run's receipt instead of running again, while a new SHA dispatches fresh children. The job log prints the parent run URL; the uploaded full-release-validation-nightly-request artifact holds the helper's request record for node scripts/full-release-validation-at-sha.mjs --reconcile-request <path>. After a failure the helper keeps both transport refs for reruns and diagnosis. To list nightly parents, filter on the transport branch:

bash
gh run list --workflow full-release-validation.yml --event workflow_dispatch \  --json databaseId,headBranch,headSha,status,conclusion \  --jq '.[] | select(.headBranch | startswith("release-ci/"))'

OpenClaw Performance

OpenClaw Performance is the product/runtime performance workflow. It runs daily on main and can be dispatched manually:

bash
gh workflow run openclaw-performance.yml --ref main -f profile=diagnostic -f repeat=3gh workflow run openclaw-performance.yml --ref main -f profile=smoke -f repeat=1 -f deep_profile=true -f live_openai_candidate=truegh workflow run openclaw-performance.yml --ref main -f target_ref=v2026.5.2 -f profile=diagnostic -f repeat=3

Manual dispatch normally benchmarks the workflow ref. Set target_ref to benchmark a release tag or another branch with the current workflow implementation. Published report paths and latest pointers are keyed by the tested ref, and each index.md records the tested ref/SHA, workflow ref/SHA, Kova ref, profile, lane auth mode, model, repeat count, and scenario filters.

The workflow installs OCM from a pinned release and Kova from openclaw/Kova at the pinned kova_ref input, then runs three lanes:

  • mock-provider: Kova diagnostic scenarios against a local-build runtime with fixed fake OpenAI-compatible auth.
  • mock-deep-profile: CPU/heap/trace profiling for startup, gateway, and agent-turn hotspots. Runs on schedule, or on dispatch with deep_profile=true.
  • live-openai-candidate: a real OpenAI openai/gpt-5.6-luna agent turn. Selected on schedule, or on dispatch with live_openai_candidate=true. Candidates ineligible for live credentials are skipped. For a selected, eligible lane, missing OPENAI_API_KEY fails the lane rather than skipping it.

Before tolerating a partial Kova verdict, the report gate requires aggregate RSS and CPU samples to match the individual records, including repeated measurements. A missing or substituted sample keeps the gate nonzero even when the sample count matches.

OpenClaw-native source checks run in the separate source_performance job, in parallel with the Kova lanes after resolve_target: gateway boot timing and memory across default, skipped-channel, internal-hook, and fifty-plugin startup cases; bundled plugin import RSS, repeated mock-OpenAI channel-chat-baseline hello loops, CLI startup commands against the booted gateway, and the SQLite state smoke performance check. When the previous published mock-provider source report is available for the tested ref, the source summary compares current RSS and heap values against that baseline and marks large RSS increases as watch. The publisher includes these source artifacts in the mock-provider report bundle, with the Markdown summary at source/index.md and raw JSON beside it.

Every lane uploads its complete GitHub artifact, including CPU, heap, trace, and compressed diagnostic bundles. A separate publisher job downloads and validates those artifacts, then mints a short-lived ClawSweeper GitHub App token scoped only to openclaw/clawgrit-reports contents and passes it only to the Git push step. It commits report.json, report.md, index.md, source-check artifacts, and bundle metadata/checksums under openclaw-performance/<tested-ref>/<run-id>-<attempt>/<lane>/; the full diagnostic archive stays in the linked Actions artifact. The publisher rejects any report file over 50 MB before attempting a push. The current tested-ref pointer is openclaw-performance/<tested-ref>/latest-<lane>.json. Scheduled runs and profile=release dispatches fail if app-token creation or report publication fails. Manual non-release dispatches keep publication advisory and retain the GitHub artifacts when authentication or publishing fails. The previous source baseline is fetched anonymously from the public reports repository, so a successful baseline fetch does not prove publisher authentication.

All explicit Performance workflow Git commands use the pinned Git lifecycle owner, prepared in RUNNER_TEMP before each job's selected checkout. Target resolution, Kova revision/install Git, source revision and baseline Git, and local publisher operations remain unbounded. Only the initial reports fetch, each push, and each reconciliation fetch have a 120-second deadline. The owner drains the entire Git process tree before reads, checkout reuse, artifact consumers, outputs, or retry; exclusive reports fetches reclaim only invocation-created locks after extinction.

Report preparation and all fetches are anonymous. The App token is created only after a new report is prepared, removed from the environment immediately, and passed as a masked Basic header to push commands alone; it never enters the remote URL or repository config. A verified existing report succeeds before token creation. Only a successful empty ls-tree lookup means a baseline or report is absent; repository/read failures are terminal. Malformed baseline pointer JSON remains advisory, as does an ordinary baseline fetch failure after verified cleanup.

Publication allows exactly five pushes. Every failed push, including the fifth, gets a 2/4/6/8/10-second backoff followed by one anonymous reconciliation fetch. A fetched remote report proves success even after the fifth ambiguous push; direct push success needs no fetch. Otherwise, attempts 1–4 replay the report commit on detached FETCH_HEAD with cherry-pick -X theirs, preserving concurrent unique reports while the current writer wins the latest pointer. There is no fifth-attempt replay. Ordinary fetch failures warn and retry on attempts 1–4. Only typed Git failure or timeout after verified cleanup permits recovery; owner setup, census, cleanup failure, and cancellation stop before fallback, retry, replay, or success. Full Release Validation continues to disable the publisher entirely and retains performance evidence only as workflow artifacts.

Gateway concurrency benchmark

The manual gateway-concurrency mode measures a busy isolated Gateway and retains its results as Actions artifacts:

bash
gh workflow run openclaw-performance.yml --ref main \  -f mode=gateway-concurrency -f live_openai_candidate=true

One job builds the selected source once, runs the mock provider, then optionally runs OpenAI with a real key. Each run seeds 1,000 sessions across 32 agents and completes 96 turns alongside session updates, history reads, subscriptions, and 64 check rounds. Dreaming is disabled; normal indexing, recaps, and the database idle retention policy remain unchanged. The live run denies tools and caps model output at 128 tokens. Results include load-phase main-thread and Worker CPU profiles. The two providers are different workloads, not a before/after speed comparison.

Live execution requires the default-branch workflow and the tested SHA to equal the workflow SHA. A requested live run fails if that condition or OPENAI_API_KEY is missing. Omit live_openai_candidate for mock-only evidence. This mode runs no Kova lanes, source-check jobs, or report publication, and is not selected by the daily schedule. See Gateway concurrency for local invocation and result interpretation.

Vitest paired benchmark

The manual-only vitest-pair mode compares two exact commits with the workflow implementation from the candidate commit:

bash
gh workflow run openclaw-performance.yml \  --ref <candidate-branch> \  -f mode=vitest-pair \  -f baseline_ref=<40-character-baseline-sha> \  -f target_ref=<40-character-candidate-sha>

Both inputs must be lowercase full SHAs, target_ref must equal the workflow SHA selected by --ref, and reruns are refused. Dispatch a fresh workflow run instead of retrying an attempt. Kova, source checks, report publication, and their artifact-only guard stay skipped in this mode. The benchmark job has read-only repository permission, does not receive secrets, does not restore or save Actions caches, and checks out the helper, candidate, and baseline with credentials disabled.

The committed lane manifest covers representative core unit, Gateway, Control UI jsdom, and worker-lifecycle tests. Both commits must expose identical selected test/config paths and bytes and pass correctness before timing state is created. Correctness also requires both sides to report the same normalized test files, test identities, statuses, and counts. Every later run must match that established execution digest. The harness then runs one excluded warmup per side and lane, seven paired rounds with alternating side order and rotated lane order, plus one separately labeled cold pair with fresh caches. Frozen installs are setup and are never timed.

Each child has a fixed deadline and process-group owner. A separate 165-minute harness deadline reserves 15 minutes inside the 180-minute job timeout for cleanup, terminal-manifest finalization, and artifact upload. It aborts and joins the active managed child before refusing further child starts. Every install, correctness, warmup, measured, and cold process receives the exact pinned pnpm executable through npm_execpath, with private Corepack and pnpm state; the resolved executable and version are recorded in the environment and run records.

The artifact includes raw logs, raw Vitest JSON reports, execution digests and counts, GNU time user/system CPU, wall timing, environment and Git identities, source/config hashes, per-run records, paired-ratio analysis, and a terminal success or failure manifest. The workflow attempts finalization and artifact upload after harness failures. Runner loss or external workflow cancellation can still prevent those steps from running. Mutable pnpm and runtime caches stay in an unuploaded scratch tree. Thresholds are fixed in scripts/vitest-pair-benchmark-lanes.json. Acceptance uses the median of seven per-round aggregate ratios, with each round weighted by total lane duration, and fails above 5%. A critical lane fails only when its median measured ratio is above 10% and its median paired delta is at least one second. The single cold pair remains diagnostic evidence and never fails acceptance. The report claims an improvement only when every representative lane's median clears the improvement ratio and at least five of its seven pairs individually meet that ratio. Otherwise it reports per-lane evidence without a broad improvement claim. Artifacts use only the trusted workflow run ID and attempt in their name; the exact baseline and candidate commits remain recorded inside the artifact.

Security Review reconciler

Every ten minutes, Security Review reconciles CI completions from five minutes before the previous successful scheduled pass started until five minutes ago (sixty-minute fallback, twelve-hour cap). Passes tile without gaps; late or dropped cron ticks only widen the next window, up to the cap. Run listing covers creation times from three hours before the window through the current time. GitHub caps each filtered query at 1,000 results, so the resolver bisects ranges whose reported total exceeds that limit. Each smaller range is paged until a short page, and its distinct run count must cover the total reported on its first page. Ten full pages or fewer distinct runs than reported fail the pass before any status publication or matrix output, retaining the anchor for a complete retry. Inclusive range endpoints are separated by one second, and run IDs are deduplicated across pages and slices. A range shorter than ten minutes that still exceeds 1,000 runs fails before status publication or matrix output, so the covered window does not advance. The next pass rescans from the last successful pass. Scheduled resolver passes share one concurrency group without canceling an active pass; GitHub keeps one pending pass, which still starts from the last successful window. Each pass selects at most 100 PR heads, oldest CI completion first, with run ID breaking ties, and stops reading statuses when the cap is reached. If candidates remain, the reconcile-backlog job fails the workflow after the selected reviews finish. This keeps the same anchor: reviewed heads have fresh statuses, allowing the next scheduled pass to select the remainder from the same window.

After a reconciler outage longer than twelve hours, older lost completions need a new push or a Security Review rerun.

A CI rerun keeps its original creation time. A rerun of a run created more than three hours before the window relies on its own completion delivery; if that is lost, a new push or a Security Review rerun recovers it.

Wholly skipped CI runs are ignored. The resolver reads status history in reverse chronological order and uses only the newest openclaw/ci-gate status from github-actions[bot] with creator type Bot. Other publishers are ignored. Normal review runs only when that Actions-owned status is missing, older than CI completion, or pending. Only a non-pending status created at or after CI completion is settled and stops reselection. Every pending status remains eligible, including a review wait published after a pre-completion CI read. Tiled windows bound the harmless extra review when a head legitimately waits on newer in-progress CI. It never checks out PR code and uses one hosted ubuntu-24.04 resolver job per pass, run-list reads plus paginated status-history reads per newly completed head, and no Blacksmith registrations. See Security review checks.

QA Lab

QA Lab has dedicated CI lanes outside the main smart-scoped workflow. Agentic parity is nested under the broad QA and release harnesses, not a standalone PR workflow. Use Full Release Validation with rerun_group=qa-parity when parity should ride with a broad validation run.

  • The QA-Lab - All Lanes workflow runs nightly on main and on manual dispatch; it fans out mock parity plus live Matrix, Telegram, Discord, WhatsApp, and Slack jobs. Live jobs use the qa-live-shared environment; Telegram, Discord, WhatsApp, and Slack use Convex leases, while Matrix provisions disposable local credentials.
  • Manual and scheduled aggregate runs retain the default all concurrency scope. Trusted release calls use separate matrix and buzz scopes so those lanes can run together for one target SHA; Matrix calls for the same SHA still serialize, while Buzz calls serialize across SHAs because they share pooled credentials.
  • Release Matrix catalog validation runs on a 16-vCPU Blacksmith runner with a 90-minute job budget. Changes to that timeout, runner size, or concurrency require a matching workflow guard and exact-candidate release proof.
  • QA Profile Evidence balances taxonomy category groups across eight isolated jobs, keeps non-isolating live channels on one shard, then asks QA Lab to merge their validated evidence into one attested qa-evidence.json. A timed-out or missing shard always fails aggregation; allow_failures applies only when every shard completed and produced valid evidence. Direct Maturity scorecard dispatches default allow_failures on so incomplete evidence can still render a diagnostic docs artifact. A terminal result gate runs after optional generated-PR publication and fails the run when any scenario failed or remained blocked; reusable release calls remain strict by default.

Scheduled, manual, and release Matrix checks use the fixed-response mock provider so the live transport contract is isolated from model latency and normal provider-plugin startup. Telegram release checks use the same fixed-response model setup. The live transport gateway disables memory search because QA parity covers memory behavior separately; provider connectivity is covered by the separate live model, native provider, and Docker provider suites.

OpenClaw Release Checks also runs the release-critical QA Lab lanes before release approval; its QA parity gate runs the candidate and baseline packs as parallel lane jobs, then downloads both artifacts into a small report job for the final parity comparison.

For normal PRs, follow scoped CI/check evidence instead of treating parity as a required status.

CodeQL

The CodeQL workflow is intentionally a narrow first-pass security scanner, not the full repository sweep. Daily, manual, main push, and non-draft pull request guard runs scan Actions workflow code plus the highest-risk JavaScript/TypeScript surfaces with high-confidence security queries filtered to high/critical security-severity.

The pull request guard stays light: it only starts for changes under .github/actions, .github/codeql, .github/workflows, packages, scripts, src, or process-owning bundled plugin runtime paths, and it runs the same high-confidence security matrix as the scheduled workflow. Android and macOS CodeQL stay out of PR defaults.

Security categories

Category Surface
/codeql-security-high/core-auth-secrets Auth, secrets, sandbox, cron, and gateway baseline
/codeql-security-high/channel-runtime-boundary Core channel implementation contracts plus the channel plugin runtime, gateway, Plugin SDK, secrets, audit touchpoints
/codeql-security-high/network-ssrf-boundary Core SSRF, IP parsing, network guard, web-fetch, and Plugin SDK SSRF policy surfaces
/codeql-security-high/mcp-process-tool-boundary MCP servers, process execution helpers, outbound delivery, and agent tool-execution gates
/codeql-security-high/process-exec-boundary Local shell, process spawn helpers, subprocess-owning bundled plugin runtimes, and workflow script glue
/codeql-security-high/plugin-trust-boundary Plugin install, loader, manifest, registry, package-manager install, source-loading, and Plugin SDK package contract trust surfaces

Platform-specific security shards

  • CodeQL Android Critical Security — scheduled Android security shard. Builds the Android app manually for CodeQL on the smallest Blacksmith Linux runner accepted by workflow sanity. Uploads under /codeql-critical-security/android.
  • CodeQL macOS Critical Security — weekly/manual macOS security shard. Prepares the generated Mermaid resources on GitHub-hosted Linux, then builds the ARM64 macOS app manually for CodeQL on a GitHub-hosted Intel runner without unused index-store or debug-info artifacts; filters dependency build results out of uploaded SARIF while retaining first-party generated protocol models; and uploads under /codeql-critical-security/macos. Its macOS job has a 90-minute ceiling because the complete traced build and analysis exceed the previous 45-minute budget. Kept outside daily defaults because macOS build dominates runtime even when clean.

Critical Quality categories

CodeQL Critical Quality is the matching non-security shard. It runs only error-severity, non-security JavaScript/TypeScript quality queries over narrow high-value surfaces on GitHub-hosted Linux runners so quality scans do not spend Blacksmith runner-registration budget. Its pull request guard is intentionally smaller than the scheduled profile: non-draft PRs run only the matching shards for the surfaces they touch, from thirteen PR-routable shards — agent-runtime-boundary, channel-runtime-boundary, config-boundary, core-auth-secrets, gateway-runtime-boundary, mcp-process-runtime-boundary, memory-runtime-boundary, network-runtime-boundary, plugin-boundary, plugin-sdk-package-contract, plugin-sdk-reply-runtime, provider-runtime-boundary, and session-diagnostics-boundary. ui-control-plane and web-media-runtime-boundary stay out of PR runs. CodeQL config and quality workflow changes run the full PR shard set (the network runtime shard keys off its own CodeQL config files and network-owning source paths).

Manual dispatch accepts:

text
profile=all|agent-runtime-boundary|config-boundary|core-auth-secrets|channel-runtime-boundary|gateway-runtime-boundary|memory-runtime-boundary|mcp-process-runtime-boundary|network-runtime-boundary|plugin-boundary|plugin-sdk-package-contract|plugin-sdk-reply-runtime|provider-runtime-boundary|session-diagnostics-boundary

The narrow profiles are teaching/iteration hooks for running one quality shard in isolation.

On pull requests, the network runtime shard starts with a fast diff scan. Sensitive socket imports/calls and proxy-policy tokens, edits to its queries/config/fixtures, and changes to the Codex transport select full CodeQL analysis in the same PR job. Absent or null patches for monitored non-test sources also select full analysis; metadata fetch or parse failures stop shard selection rather than silently skipping it. Known ordinary diffs keep the fast path. The full path runs semantic query tests before analysis, including coverage of the configured packages/net-policy/src directory and preservation of exact owner/function allowances and test-path exclusions. Full analysis fails the job on any SARIF finding or missing SARIF output; a sensitive diff is a routing signal, not a finding.

Category Surface
/codeql-critical-quality/core-auth-secrets Auth, secrets, sandbox, cron, and gateway security boundary code
/codeql-critical-quality/config-boundary Config schema, migration, normalization, and IO contracts
/codeql-critical-quality/gateway-runtime-boundary Gateway protocol schemas and server method contracts
/codeql-critical-quality/channel-runtime-boundary Core channel and bundled channel plugin implementation contracts
/codeql-critical-quality/agent-runtime-boundary Command execution, model/provider dispatch, auto-reply dispatch and queues, and ACP control-plane runtime contracts
/codeql-critical-quality/mcp-process-runtime-boundary MCP servers and tool bridges, process supervision helpers, and outbound delivery contracts
/codeql-critical-quality/memory-runtime-boundary Memory host SDK, memory runtime facades, memory Plugin SDK aliases, memory runtime activation glue, and memory doctor commands
/codeql-critical-quality/network-runtime-boundary Network policy package, raw socket and proxy-capture runtime, SSH tunnel, gateway lock, JSONL socket, and push transport surfaces
/codeql-critical-quality/session-diagnostics-boundary Reply queue internals, session delivery queues, outbound session binding/delivery helpers, diagnostic event/log bundle surfaces, and session doctor CLI contracts
/codeql-critical-quality/plugin-sdk-reply-runtime Plugin SDK inbound reply dispatch, reply payload/chunking/runtime helpers, channel reply options, delivery queues, and session/thread binding helpers
/codeql-critical-quality/provider-runtime-boundary Model catalog normalization, provider auth and discovery, provider runtime registration, provider defaults/catalogs, and web/search/fetch/embedding registries
/codeql-critical-quality/ui-control-plane Control UI bootstrap, local persistence, gateway control flows, and task control-plane runtime contracts
/codeql-critical-quality/web-media-runtime-boundary Core web fetch/search, media IO, media understanding, image-generation, and media-generation runtime contracts
/codeql-critical-quality/plugin-boundary Loader, registry, public-surface, and Plugin SDK entrypoint contracts
/codeql-critical-quality/plugin-sdk-package-contract Published package-side Plugin SDK source and plugin package contract helpers

Quality stays separate from security so quality findings can be scheduled, measured, disabled, or expanded without obscuring security signal. Swift, Python, and bundled-plugin CodeQL expansion should be added back as scoped or sharded follow-up work only after the narrow profiles have stable runtime and signal.

Maintenance workflows

PR CI Sweeper

PR CI Sweeper checks recent pull requests hourly at minute 7. It repairs missing pull_request CI and GitHub startup_failure runs through a bounded close/reopen cycle, warning when it starts infrastructure recovery. Drafts, recently updated or conflicted PRs, and PRs with auto-merge enabled remain unchanged. Attached queued, running, failed, or canceled CI prevents recovery: cancellation does not prove a provider failure or that tests never executed, so the sweeper never automatically re-executes those workflows.

Comment automation

Comment jobs reject known no-ops before acquiring a hosted runner. Maintainer Command Reactions skips comments without / only when using its default command list; any nonempty MAINTAINER_COMMAND_REACTIONS override retains the full matcher, including commands without slashes. Auto response skips Bot-authored issue comments that Barnacle already ignores and targets whose author association already exempts them. Its edit admission retains title, body, and PR-base changes. PR context checks omit the policy's known bot and privileged-author exemptions. Labeler retains title/base PR edits and skips issue edits that leave the title unchanged. These decisions happen before checkout and runner allocation.

Auto response, Labeler, PR context checks, and ClawSweeper acquire concurrency slots only after job admission. Skipped events cannot replace useful pending work. Labeler uses separate groups for each PR and issue, with manual backfills serialized separately; PR context checks cancel superseded checks on the same PR. ClawSweeper retains its per-item cancellation rules for issue and PR events and all label/comment intake, skipping only explicitly empty metadata edits. Each event kind has its own lane: comments serialize per comment, and review activity runs never share a group. PR events and reviews leave check runs on the PR head, where GitHub counts a cancelled run as non-passing even after a newer run succeeds, so only a newer PR event on the same PR may supersede a PR event run, and nothing cancels a review run. Security Review omits PR prose-only edits while retaining base/permission changes, head changes, and approval revocations; its per-head review serialization remains non-canceling.

Dependency Audit

Dependency Audit runs the production lockfile audit daily at 07:23 UTC and on manual dispatch. It stays separate from PR CI and fails on findings, unavailable advisories, or invalid data. This red triage signal is not a required PR check and never gates merging. Follow up with a dependency bump on main. Each dependency graph is submitted as one request; release checks keep their product and tooling graphs separate.

Both release CI and this strict audit publish the outcome, package count, duration, timestamp, and bounded failure reason in the job summary. A completed npm check covers npm bulk advisories only, not every upstream advisory source.

The triage owner is @steipete, set on 2026-09-03 in #137960. No .github/CODEOWNERS rule covers .github/workflows/dependency-audit.yml, so this line is the only record of that ownership. Investigate failed scheduled runs and rerun the strict workflow to confirm recovery:

bash
gh workflow run dependency-audit.yml --repo openclaw/openclaw --ref main

GitHub sends scheduled-run notifications to the workflow creator or the latest cron editor, subject to that account's Actions notification settings. Keep those notifications enabled when taking ownership; a summary mention does not send an alert. See GitHub's workflow notification rules.

For local reproduction, run node scripts/pre-commit/pnpm-audit-prod.mjs --audit-level=high. Adding --ci selects a shorter 30-second diagnostic budget but preserves exit codes: 0 means no matching findings, 1 means findings or an error, and 2 means incomplete coverage. This direct diagnostic command and the daily Dependency Audit retain those non-zero exits. Ordinary pull-request, push, and scheduled CI skip the audit. CI runs it only for workflow_dispatch IDs beginning with full-release-validation- or release-native-android-; every non-zero audit exit becomes a warning with exit code 0. The optional pnpm-audit-prod pre-commit hook uses the same warn-only wrapper and retains the audit output. Dependency advisories cannot block CI, local commits, or releases. The separate release pnpm deps:vuln:gate still blocks known malware; vulnerability advisories there remain warnings.

Docs Sync Publish Repo

Docs Sync Publish Repo assembles English, ClawHub, and preserved translated pages in openclaw/docs. After each final rebase and before pushing, it installs the publisher's existing npm lock with npm ci and checks the resulting docs tree. Push retries reuse that install unless the publisher manifest or lock changes. The checker runs natively on Node 24; no separate checker dependency graph or TypeScript loader is installed.

The disposable Actions validation cache records successful page checks by repository-relative path and complete raw content hash. Checker and helper changes, Node/runtime changes, or requested/installed npm lock changes invalidate reuse. Missing or corrupt cache data triggers full checking; deleted pages are pruned from the next successful cache. docs.json is checked every time, including on warm runs. Only successful main-branch workflows save the artifact, outside the publish repository. The checker preserves its Markdown/MDX format detection, poison-text checks, and component-indentation checks; it does not replace the site renderer or cross-page link validation.

Docs Agent

The Docs Agent workflow keeps existing docs aligned with recently landed changes. It has no pure schedule. An opted-in full main-push CI run can admit automatic writes; hourly main-tier CI cannot. With OPENCLAW_CI_ON_PUSH unset, use explicit non-bot Docs Agent dispatch to run it. A read-only job verifies the upstream CI workflow, exact completed run attempt, current main SHA, successful aggregate, and successful revision-confirmation step before the write-capable job is admitted. That producer step is absent/skipped for main-tier runs, security-only pushes, failed full CI, and manual validation of another target or reduced scope. Ordinary bot pushes remain excluded.

Only the admitted write job occupies the non-canceling docs concurrency slot, so a skipped push cannot displace pending eligible automatic or manual work. Workflow-run invocations recheck main freshness and inspect exact-attempt job evidence for up to 100 recent runs. A queued or active write job, or a recent attempt that actually ran the agent, counts toward the one-hour cadence. Canceled and skipped workflows, denied verification, and completed writer gates that skipped the agent do not count. When admitted, the agent reviews from the source SHA of the previous successful write job whose agent step succeeded to current main.

Failed agent attempts can throttle another attempt within the hour, but do not advance the review base. If history evidence cannot be read, the gate fails before running the agent.

Duplicate PRs After Merge

The Duplicate PRs After Merge workflow is a manual maintainer workflow for post-land duplicate cleanup. It defaults to dry-run and only closes explicitly listed PRs when apply=true. Before mutating GitHub, it verifies that the landed PR is merged and that each duplicate has either a shared referenced issue or overlapping changed hunks.

bash
gh workflow run duplicate-after-merge.yml \  -f landed_pr=70532 \  -f duplicate_prs='70530,70592' \  -f apply=true

Update Migration

Update Migration runs the expanded published-upgrade baseline set weekly on Sunday at 03:17 UTC and on manual dispatch. It keeps both plugin-deps-cleanup and legacy-operator-state, with no provider secrets. Its separate non-canceling schedule group coalesces pending runs and cannot cancel manual validation. See Package Acceptance suite profiles for runtime baseline resolution, per-baseline grouping, the 78–90 runner-minute weekly planning allowance, and the successful-upgrade requirements. Runner registration budgets account for the weekly burst separately from PR and main admission.

ClawSweeper activity forwarding

.github/workflows/clawsweeper-dispatch.yml is the target-side bridge from OpenClaw repository activity into ClawSweeper. It does not check out or execute untrusted pull request code. The workflow creates a GitHub App token from CLAWSWEEPER_APP_PRIVATE_KEY, then dispatches compact repository_dispatch payloads to openclaw/clawsweeper.

Dispatch API calls retry rate-limit failures for up to five attempts with quadratic backoff. Other API errors stop immediately, and exhausted retries preserve the final API exit code. Dispatch callers warn and continue on failure rather than reporting a successful dispatch.

The workflow has three lanes:

  • clawsweeper_item for exact issue and pull request review requests;
  • clawsweeper_comment for explicit ClawSweeper commands in issue comments;
  • github_activity for general GitHub activity that the ClawSweeper agent may inspect.

The github_activity lane forwards normalized metadata only: event type, action, actor, repository, item number, URL, title, state, and short excerpts for comments or reviews when present. It intentionally avoids forwarding the full webhook body. The receiving workflow in openclaw/clawsweeper is .github/workflows/github-activity.yml, which posts the normalized event to the OpenClaw Gateway hook for the ClawSweeper agent.

Main pushes remain github_activity observations. They do not produce hosted per-commit reports or commit Check Runs.

General activity is observation, not delivery-by-default. The ClawSweeper agent receives the Discord target in its prompt and should post to #clawsweeper only when the event is surprising, actionable, risky, or operationally useful. Routine opens, edits, bot churn, duplicate webhook noise, and normal review traffic should result in NO_REPLY.

Treat GitHub titles, comments, bodies, review text, branch names, and commit messages as untrusted data throughout this path. They are input for summarization and triage, not instructions for the workflow or agent runtime.

Barnacle treats bug-labeled issues as verification candidates rather than inactivity-close candidates. It may add the stale label, which dispatches one exact ClawSweeper review, but it cannot close that issue. ClawSweeper may then apply an evidence-backed resolution; a proven fix on current main closes as completed, while current or inconclusive bugs stay open. The stale workflow also audits recent close events and fails when a Barnacle identity closes a bug as not_planned.

Was this useful?