On this page
On this page
Testing and CI
ACP bind and Codex app-server lanes
Live: ACP bind smoke (/acp spawn ... --bind here)
- Test:
src/gateway/gateway-acp-bind.live.test.ts - Goal: validate the real ACP conversation-bind flow with a live ACP agent:
- send
/acp spawn <agent> --bind here - bind a synthetic message-channel conversation in place
- send a normal follow-up on that same conversation
- verify the follow-up lands in the bound ACP session transcript
- send
- Enable:
pnpm test:live src/gateway/gateway-acp-bind.live.test.tsOPENCLAW_LIVE_ACP_BIND=1
- Defaults:
- ACP agents in Docker:
claude,codex,gemini - ACP agent for direct
pnpm test:live ...:claude - Synthetic channel: Slack DM-style conversation context
- ACP backend:
acpx
- ACP agents in Docker:
- Overrides:
OPENCLAW_LIVE_ACP_BIND_AGENT=claudeOPENCLAW_LIVE_ACP_BIND_AGENT=codexOPENCLAW_LIVE_ACP_BIND_AGENT=droidOPENCLAW_LIVE_ACP_BIND_AGENT=geminiOPENCLAW_LIVE_ACP_BIND_AGENT=grok-buildOPENCLAW_LIVE_ACP_BIND_AGENT=opencodeOPENCLAW_LIVE_ACP_BIND_AGENTS=claude,codex,geminiOPENCLAW_LIVE_ACP_BIND_AGENT_COMMAND='npx -y @agentclientprotocol/claude-agent-acp@<version>'OPENCLAW_LIVE_ACP_BIND_CODEX_MODEL=gpt-5.6-lunaOPENCLAW_LIVE_ACP_BIND_OPENCODE_MODEL=opencode/kimi-k2.6OPENCLAW_LIVE_ACP_BIND_IMAGE_PROBE=1(oron/true/yes) to force the image check on; any other value forces it off. Runs by default for every agent exceptopencode.OPENCLAW_LIVE_ACP_BIND_REQUIRE_CRON=1OPENCLAW_LIVE_ACP_BIND_PARENT_MODEL=openai/gpt-5.6-luna
- Notes:
- This lane uses the gateway
chat.sendsurface with admin-only synthetic originating-route fields so tests can attach message-channel context without pretending to deliver externally. - When
OPENCLAW_LIVE_ACP_BIND_AGENT_COMMANDis unset, the test uses the embeddedacpxplugin's built-in agent registry for the selected ACP harness agent. - Bound-session cron MCP creation is best-effort by default because external ACP harnesses can cancel MCP calls after the bind/image proof has passed; set
OPENCLAW_LIVE_ACP_BIND_REQUIRE_CRON=1to make that post-bind cron check strict.
- This lane uses the gateway
Example:
Docker recipe:
Single-agent Docker recipes:
Docker notes:
- The Docker runner lives at
scripts/test-live-acp-bind-docker.sh. - By default, it runs the ACP bind smoke against the aggregate live CLI agents in sequence:
claude,codex, thengemini. - Use
OPENCLAW_LIVE_ACP_BIND_AGENTS=claude,OPENCLAW_LIVE_ACP_BIND_AGENTS=codex,OPENCLAW_LIVE_ACP_BIND_AGENTS=droid,OPENCLAW_LIVE_ACP_BIND_AGENTS=gemini, orOPENCLAW_LIVE_ACP_BIND_AGENTS=opencodeto narrow the matrix. - It stages the matching CLI auth material into the container, then installs the requested live CLI (
@anthropic-ai/claude-code,@openai/codex, Factory Droid viahttps://app.factory.ai/cli,@google/gemini-cli, oropencode-ai) if missing. The ACP backend itself is the embeddedacpx/runtimepackage from the officialacpxplugin. - The Droid Docker variant stages
~/.factoryfor settings, forwardsFACTORY_API_KEY, and requires that API key because local Factory OAuth/keyring auth is not portable into the container. It uses ACPX's built-indroid exec --output-format acpregistry entry. - The OpenCode Docker variant is a strict single-agent regression lane. It writes a temporary
OPENCODE_CONFIG_CONTENTdefault model fromOPENCLAW_LIVE_ACP_BIND_OPENCODE_MODEL(defaultopencode/kimi-k2.6). - Direct
acpxCLI calls are only a manual/workaround path for comparing behavior outside the Gateway. The Docker ACP bind smoke exercises OpenClaw's embeddedacpxruntime backend.
Live: Codex app-server harness smoke
- Goal: validate the plugin-owned Codex harness through the normal gateway
agentmethod:- load the bundled
codexplugin - select an OpenAI model through
/model <ref> --runtime codex - send a first gateway agent turn with the requested thinking level
- send a second turn to the same OpenClaw session and verify the app-server thread can resume
- run
/codex statusand/codex modelsthrough the same gateway command path - optionally run two Guardian-reviewed escalated shell checks: one benign command that should be approved and one fake-secret upload that should be denied so the agent asks back
- load the bundled
- Test:
src/gateway/gateway-codex-harness.live.test.ts - Enable:
OPENCLAW_LIVE_CODEX_HARNESS=1 - Harness baseline model:
openai/gpt-5.6-luna - Fresh OpenAI API-key selection default:
openai/gpt-6-astra - Default thinking:
low - Model override:
OPENCLAW_LIVE_CODEX_HARNESS_MODEL=openai/<model> - Thinking override:
OPENCLAW_LIVE_CODEX_HARNESS_THINKING=<level> - Non-default model effort assertion:
OPENCLAW_LIVE_CODEX_HARNESS_EXPECTED_EFFORT=<level> - Matrix override:
OPENCLAW_LIVE_CODEX_HARNESS_TARGETS=<model>=<thinking>,... - Auth mode:
OPENCLAW_LIVE_CODEX_HARNESS_AUTH=codex-auth(default) uses the copied Codex login;api-keyusesOPENAI_API_KEYthrough Codex app-server. - Optional image check:
OPENCLAW_LIVE_CODEX_HARNESS_IMAGE_PROBE=1 - Optional MCP/tool check:
OPENCLAW_LIVE_CODEX_HARNESS_MCP_PROBE=1 - Optional Guardian check:
OPENCLAW_LIVE_CODEX_HARNESS_GUARDIAN_PROBE=1 - Optional resume stress:
OPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS=1adds four history turns, then closes and restarts the Gateway and Codex app-server three times while requiring the same native thread id and conversation history. Override the bounded counts withOPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS_HISTORY_TURNS(1-20) andOPENCLAW_LIVE_CODEX_HARNESS_RESUME_STRESS_RESTARTS(1-10). - Optional fan-out stress: set
OPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_PROBE=1andOPENCLAW_LIVE_CODEX_HARNESS_SUBAGENT_COUNT(1-12). The harness starts every child concurrently, waits for every terminal run, and verifies each unique child reply and native thread identity. - Optional compaction stress:
OPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS=1generates bounded native tool output, requires automatic compaction events, verifies the persisted compaction count and hidden-marker recall, restarts the Gateway and physical Codex app-server, then repeats the output and compaction wave. Tune the bounded work withOPENCLAW_LIVE_CODEX_HARNESS_COMPACTION_STRESS_TURNS(1-8) andOPENCLAW_LIVE_CODEX_HARNESS_LARGE_OUTPUT_BYTES(100000-800000). - Full direct-API context:
OPENCLAW_LIVE_CODEX_HARNESS_FULL_CONTEXT=1applies the922000context and700000total automatic-compaction limits, sends dense bounded user turns without/compactor another manual checkpoint, and requires a later small turn to trigger native automatic compaction. It requiresOPENCLAW_LIVE_CODEX_HARNESS_AUTH=api-keyplus an absoluteOPENCLAW_LIVE_CODEX_HARNESS_MODEL_CATALOGpath. The catalog must expose the exact selected model withcontext_window: 922000,max_context_window: 922000, andauto_compact_token_limit: 700000so Codex does not clamp the override back to its normal catalog window. The ordinary reduced-threshold stress above keeps the stricter automatic-compaction and hidden-marker retention assertions. - Optional loop-relay opt-out check:
OPENCLAW_LIVE_CODEX_HARNESS_DISABLE_LOOP_RELAY=1 - The requested thinking preference may map to the nearest effort advertised
by Codex for that model. For example, Luna maps
minimaltolow. - Known Codex catalog models derive that exact native effort automatically. Unknown model overrides must state the expected mapped effort.
- The smoke forces provider/model
agentRuntime.id: "codex"so a broken Codex harness cannot pass by silently falling back to OpenClaw. - Auth: Codex app-server auth from the local Codex subscription login, or
OPENAI_API_KEYwhenOPENCLAW_LIVE_CODEX_HARNESS_AUTH=api-key. Docker can copy~/.codex/auth.jsonand~/.codex/config.tomlfor subscription runs.
Local recipe:
Docker recipe:
The Docker lane uses a seccomp profile based on Docker's default policy with
the namespace setup operations required by Codex's native Bubblewrap sandbox.
It keeps no-new-privileges enabled and checks native sandbox startup before
making provider requests. The runner's kernel and AppArmor policy must also
permit nested sandbox setup. On disposable CI/Testbox runners with AppArmor,
the lane loads a temporary profile for its own container and removes it after
the container exits. It preserves Docker's other restrictions and does not
change the default profile or kernel settings. Loading requires an available
apparmor_parser and noninteractive permission to load profiles.
The subagent check also enables Codex's V2 native children by default. Explicit
OPENCLAW_CODEX_APP_SERVER_ARGS overrides are forwarded unchanged; disabling the
subagent check leaves the native argument defaults unchanged. Manual compaction
configurations supply explicit native arguments that take precedence over this
environment default.
Restart and history stress:
Fan-out, large-output, compaction, and restart stress:
Full native Codex 922000 input-budget compaction stress:
GPT-5.6 native Codex matrix: