Testing and CI
Docker test suites
Full Docker suite (pnpm test:docker:all)
Builds the shared live-test image, packs OpenClaw once as an npm tarball, builds/reuses a bare Node/Git runner image plus a functional image that installs that tarball into /app, then runs Docker smoke lanes through a weighted scheduler. scripts/package-openclaw-for-docker.mjs is the stable local/CI package packer entrypoint and validates the tarball plus dist/postinstall-inventory.json before Docker consumes it.
- Bare image (
OPENCLAW_DOCKER_E2E_BARE_IMAGE): installer/update/plugin-dependency lanes; mounts the prebuilt tarball instead of copied repo sources. - Functional image (
OPENCLAW_DOCKER_E2E_FUNCTIONAL_IMAGE): normal built-app functionality lanes. - Lane definitions:
scripts/lib/docker-e2e-scenarios.mts. Planner:scripts/lib/docker-e2e-plan.mts. Executor:scripts/test-docker-all.mjs. node scripts/test-docker-all.mjs --plan-jsonemits the scheduler-owned CI plan (lanes, image kinds, package/live-image needs, state scenarios, credential checks) without building or running Docker.
Scheduling knobs (env vars, defaults in parentheses):
| Env var | Default | Purpose |
|---|---|---|
OPENCLAW_DOCKER_ALL_PARALLELISM |
10 | Process slots. |
OPENCLAW_DOCKER_ALL_TAIL_PARALLELISM |
10 | Provider-sensitive tail pool. |
OPENCLAW_DOCKER_ALL_LIVE_LIMIT |
9 | Heavy live-provider lane cap. |
OPENCLAW_DOCKER_ALL_NPM_LIMIT |
5 | npm-resource lane cap. |
OPENCLAW_DOCKER_ALL_SERVICE_LIMIT |
7 | Service-resource lane cap. |
OPENCLAW_DOCKER_ALL_LIVE_CLAUDE_LIMIT / _CODEX_LIMIT / _GEMINI_LIMIT / _DROID_LIMIT / _OPENCODE_LIMIT |
4 | Per-provider heavy-lane caps. |
OPENCLAW_DOCKER_ALL_LIVE_OPENAI_LIMIT / _TELEGRAM_LIMIT |
1 | Narrower per-provider caps. |
OPENCLAW_DOCKER_ALL_WEIGHT_LIMIT / OPENCLAW_DOCKER_ALL_DOCKER_LIMIT |
- | Override for larger hosts. |
OPENCLAW_DOCKER_ALL_START_STAGGER_MS |
2000 | Delay between lane starts, avoids local Docker daemon create storms. |
OPENCLAW_DOCKER_ALL_LANE_TIMEOUT_MS |
7,200,000 (120 min) | Per-lane fallback timeout; selected live/tail lanes use tighter caps. |
OPENCLAW_DOCKER_ALL_LIVE_RETRIES |
1 | Retries for transient live-provider failures. |
OPENCLAW_DOCKER_ALL_DRY_RUN |
off | Print the lane manifest without running Docker. |
OPENCLAW_DOCKER_ALL_STATUS_INTERVAL_MS |
30000 | Active-lane status print interval. |
OPENCLAW_DOCKER_ALL_TIMINGS |
on | Reuse .artifacts/docker-tests/lane-timings.json for longest-first ordering; set to 0 to disable. |
OPENCLAW_DOCKER_ALL_LIVE_MODE |
- | skip for deterministic/local lanes only, only for live-provider lanes only. Aliases: pnpm test:docker:local:all, pnpm test:docker:live:all. Live-only mode merges main and tail live lanes into one longest-first pool so provider buckets pack Claude/Codex/Gemini work together. |
OPENCLAW_LIVE_CLI_BACKEND_SETUP_TIMEOUT_SECONDS |
180 | CLI backend Docker setup timeout. |
Env var pattern for resource caps is OPENCLAW_DOCKER_ALL_<RESOURCE>_LIMIT (resource name uppercased, non-alphanumerics collapsed to _).
Other behavior: the runner preflights Docker by default, cleans stale OpenClaw E2E containers, shares provider CLI tool caches between compatible lanes, and stops scheduling new pooled lanes after the first failure unless OPENCLAW_DOCKER_ALL_FAIL_FAST=0 is set. If one lane exceeds the effective weight/resource cap on a low-parallelism host, it can still start from an empty pool and run alone until it releases capacity. Per-lane logs, summary.json, failures.json, and phase timings write under .artifacts/docker-tests/<run-id>/; use pnpm test:docker:timings <summary.json> to inspect slow lanes and pnpm test:docker:rerun <run-id|summary.json|failures.json> to print cheap targeted rerun commands.
Notable Docker lanes
| Command | Verifies |
|---|---|
pnpm test:docker:browser-cdp-snapshot |
Chromium-backed source E2E container with raw CDP + isolated Gateway; browser doctor --deep CDP role snapshots include link URLs, cursor-promoted clickables, iframe refs, and frame metadata. |
pnpm test:docker:skill-install |
Installs the packed tarball in a bare Docker runner with skills.install.allowUploadedArchives: false, resolves a current skill slug from live ClawHub search, installs via openclaw skills install, and verifies SKILL.md, .clawhub/origin.json, .clawhub/lock.json, and skills info --json. |
pnpm test:docker:live-cli-backend:claude, :claude:resume, :claude:cache, :claude:mcp |
Focused CLI backend live probes; :claude:cache settles the no-tool prompt shape, then requires at least 90% prompt-cache reuse on the following dirty-workspace resume and on the steady resume after a thinking-level change. Gemini has matching :resume and :mcp aliases. |
pnpm test:docker:openwebui |
Dockerized OpenClaw + Open WebUI: sign in, check /api/models, run a real proxied chat through /api/chat/completions. Requires a usable live model key and pulls an external image; not expected to be CI-stable like the unit/e2e suites. |
pnpm test:docker:mcp-channels |
Seeded Gateway container plus a client container spawning openclaw mcp serve: routed conversation discovery, transcript reads, attachment metadata, live event queue behavior, outbound send routing, and Claude-style channel + permission notifications over the real stdio bridge (assertion reads raw stdio MCP frames directly). |
pnpm test:docker:upgrade-survivor |
Installs the packed tarball over a dirty old-user fixture, runs package update plus non-interactive doctor without live provider/channel keys, starts a loopback Gateway, checks agents/channel config/plugin allowlists/workspace/session state/stale legacy plugin dependency state/startup/RPC status survive. |
pnpm test:docker:published-upgrade-survivor |
Installs openclaw@latest by default, seeds realistic existing-user files, configures via a baked openclaw config set recipe, updates to the packed tarball, runs non-interactive doctor, writes .artifacts/upgrade-survivor/summary.json, checks /healthz, /readyz, RPC status. Override with OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPEC, expand a matrix with OPENCLAW_UPGRADE_SURVIVOR_BASELINE_SPECS, or add scenario fixtures with OPENCLAW_UPGRADE_SURVIVOR_SCENARIOS=reported-issues (includes configured-plugin-installs and stale-source-plugin-shadow). Package Acceptance exposes these as published_upgrade_survivor_baseline(s) / _scenarios and resolves meta tokens like last-stable-4 or all-since-2026.4.23. |
pnpm test:docker:update-migration |
Published-upgrade survivor harness in the plugin-deps-cleanup scenario, starting at the latest stable release by default. The Update Migration workflow pins that baseline before fanout; pass baselines=all-since-2026.4.23 for an explicit historical cleanup replay. |
pnpm test:docker:plugins |
Install/update smoke for local path, file:, npm registry packages with hoisted dependencies, git moving refs, ClawHub fixtures, marketplace updates, and Claude-bundle enable/inspect. |
Sandbox compatibility lanes
| Command | Verifies |
|---|---|
pnpm test:e2e:openshell |
Real OpenShell gateway, isolated control-plane workspace, custom image, remote and mirrored filesystems, eight-way mixed exec/file stress, exact host/remote inventories, failure recovery, SSH cleanup, protected host metadata, and deny/allow network policies. |
pnpm test:docker:package-install |
Packed OpenClaw npm artifact installation into a clean global prefix, then CLI version and help startup from the installed package. |
pnpm test:docker:openai-web-search-minimal |
Mocked TLS endpoint with a private test CA, isolated Gateway startup, and web-search request handling through the configured certificate trust path. |
pnpm test:docker:browser-cdp-snapshot |
Chromium startup, raw CDP connectivity, isolated Gateway browser commands, doctor output, and accessibility snapshot roles. |
pnpm test:docker:kitchen-sink-rpc |
Installed plugin commands and catalog tools, read-only Gateway RPC traversal, authentication boundaries, channel lifecycle, and resource ceilings. |
pnpm test:docker:kitchen-sink-plugin |
Packaged and registry plugin install flows, plugin execution, expected unsupported-version failures, ClawHub fallback, and npm-to-ClawHub migration. |
Onboarding E2E (Docker)
Optional; only needed for containerized onboarding smoke tests. Full cold-start flow in a clean Linux container:
scripts/e2e/onboard-docker.shDrives the interactive wizard via a pseudo-tty, verifies config/workspace/session state, then starts the gateway and runs openclaw health.
QR import smoke (Docker)
Ensures the maintained QR runtime helper loads under the default Docker Node 24 runtime:
pnpm test:docker:qrWas this useful?
