OpenClaw

​

Parameters

​

Parameter reference

​

Install strategy

• install_if_missing (default) — installs only when openclaw is not present in the container. The install runs npm, so an image without openclaw must provide a Node.js that OpenClaw supports plus npm, or set setup_capsule_tag.
• preinstalled — uses what’s already installed on the image; errors if absent.
• auto — picks a strategy by environment (not a probing auto-install): host_process uses install_if_missing, other environments like docker use preinstalled.
• install_always — reinstalls on every run (some benchmarks pin install_if_missing, in which case the benchmark’s setting takes precedence).
• upload — instead of installing from npm, uploads an existing openclaw executable from your local machine (the one running AgentCompass) into the container: it uploads from upload_src to upload_dst, adds the executable bit, and uses that file at runtime (binary automatically points to upload_dst, no need to set it separately). Suited to containers that can’t reach npm, or when you want a self-compiled / pinned binary; upload_src is required in this case.

​

Context and token limits

• context_window — the model’s total context length (corresponds to vLLM’s --max-model-len). Setting it explicitly stops OpenClaw from falling back to an over-small default window when it can’t recognize the model-under-test, which would trigger context compaction prematurely. When the value is 0, AgentCompass does not write the field into openclaw.json and the window is left to OpenClaw: if OpenClaw has a built-in context window for the model, it uses that built-in value; if it’s an unrecognized custom model, it falls back to a smaller default window and may compact the context before it has actually exceeded the limit, losing information and hurting long-task performance. Passing the model-under-test’s real context length explicitly during evaluation is therefore recommended.
• max_tokens — the per-turn completion budget; it should be noticeably smaller than context_window so that “input tokens + max_tokens” doesn’t exceed the served context length. When the value is 0 it is likewise not written into openclaw.json, and the model / server’s own default output limit applies.

​

Run examples

agentcompass run <benchmark> openclaw <model>

agentcompass run \
  pinchbench \
  openclaw \
  "$MODEL_NAME" \
  --env docker \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

agentcompass run \
  gdpval_ac \
  openclaw \
  "$MODEL_NAME" \
  --env docker \
  --harness-params '{"install_strategy": "install_if_missing", "openclaw_version": "2026.5.7"}' \
  --execution-params '{"run_timeout_seconds": 14400}' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat

agentcompass run \
  wildclawbench \
  openclaw \
  "$MODEL_NAME" \
  --env docker \
  --harness-params '{
    "provider_id": "vllm",
    "context_window": 262144,
    "max_tokens": 32768
  }' \
  --model-base-url "$MODEL_BASE_URL" \
  --model-api-key "$MODEL_API_KEY" \
  --model-api-protocol openai-chat