OpenClawDocs
View as MarkdownView this page as plain textOpen in ChatGPTAsk questions about this pageOpen in ClaudeAsk questions about this page

Providers

llama-server Provider

llama-server is llama.cpp's standalone HTTP server. The llama-server provider connects OpenClaw to a process that you already run and manage. OpenClaw discovers the models and capabilities exposed by the server and sends chat requests through its OpenAI-compatible API.

The same llama-cpp plugin also provides the managed llama-cpp provider. The managed provider installs and runs a verified server. New external setups never install, start, stop, download, or reconfigure anything. Existing configurations with an explicit localService block retain their previous supervisor behavior for compatibility.

Property Value
Provider ID llama-server
API openai-completions
Default base URL http://127.0.0.1:8080/v1
Authentication Optional LLAMA_SERVER_API_KEY
Model discovery Model-list and property endpoints
Process owner User or external supervisor

Quick start

  • Start llama-server

    Start an official llama.cpp server with a stable model alias:

    bash
    llama-server \  --model /path/to/model.gguf \  --alias my-model \  --host 127.0.0.1 \  --port 8080

    Choose the context, GPU, slot, batching, and chat-template flags for your deployment. OpenClaw does not change them.

  • Run OpenClaw setup

    bash
    openclaw onboard

    In the Local llama.cpp group, choose Existing llama-server. Accept the default URL or enter another local or private endpoint. Leave API-key authentication disabled unless the server or its reverse proxy requires it.

  • Select the discovered model

    bash
    openclaw models list --provider llama-serveropenclaw models set llama-server/my-model

    • A stable --alias keeps the OpenClaw model reference independent of the GGUF file path.

    Managed and external providers

    Install the llama-cpp plugin once for either provider:

    bash
    openclaw plugins install @openclaw/llama-cpp-provider
    Model reference Server owner Setup behavior
    llama-cpp/<model> OpenClaw Installs and manages the runtime
    llama-server/<model> User Connects to an existing endpoint

    Both providers can be configured at the same time. Use llama-cpp when you want OpenClaw to manage the process. Use llama-server when another terminal, container, host manager, or machine owns the process.

    Existing manual llama-server configurations that use localService continue to work. New setups should use the managed llama-cpp provider when OpenClaw must own the server process.

    Discovery

    Discovery reads these endpoints:

    • /health
    • /models, with /v1/models as a compatibility fallback
    • /props for a single loaded model
    • /props?model=<id>&autoload=false in router mode

    Discovery does not load, wake, unload, download, or reload a model. Router property requests set autoload=false, and model-list requests do not set reload=1.

    For each model, OpenClaw uses server metadata for the active context, maximum output, input types, slot count, build information, status, and chat-template capabilities. It enables tools only when the server reports both supports_tools and supports_tool_calls.

    Explicit model entries in openclaw.json take priority over discovered rows with the same model ID.

    Authentication

    Set LLAMA_SERVER_API_KEY when llama-server or its reverse proxy requires a bearer token:

    bash
    export LLAMA_SERVER_API_KEY="your-key"openclaw onboard

    Guided setup can save the key in an OpenClaw auth profile. Provider API keys, SecretRef values, and explicit authorization headers are also supported. An explicit authorization header takes priority over bearer-key resolution.

    Do not put credentials in the endpoint URL. OpenClaw rejects URLs that contain a username or password.

    Non-interactive setup

    An unauthenticated local server needs its URL:

    bash
    openclaw onboard \  --non-interactive \  --accept-risk \  --auth-choice llama-server \  --custom-base-url http://127.0.0.1:8080/v1

    Select an advertised model explicitly when needed:

    bash
    openclaw onboard \  --non-interactive \  --accept-risk \  --auth-choice llama-server \  --custom-base-url http://127.0.0.1:8080/v1 \  --custom-model-id my-model

    Pass --llama-server-api-key or set LLAMA_SERVER_API_KEY for an authenticated endpoint. When you replace an existing endpoint, pass --llama-server-api-key explicitly. OpenClaw does not reuse the previous endpoint's environment, profile, header, or configured credentials. Non-interactive setup verifies the endpoint and selected model before it writes configuration.

    Manual configuration

    Guided setup is recommended because it discovers and verifies model metadata. A minimal manual provider has this shape:

    json5
    {  models: {    mode: "merge",    providers: {      "llama-server": {        baseUrl: "http://127.0.0.1:8080/v1",        api: "openai-completions",        request: { allowPrivateNetwork: true },        models: [],      },    },  },}

    The private-network option is required for loopback and private endpoints. OpenClaw still pins requests to the configured origin and blocks cloud metadata and unsafe redirects.

    Troubleshooting

    Server is unavailable

    Check the public health and model endpoints:

    bash
    curl http://127.0.0.1:8080/healthcurl http://127.0.0.1:8080/models

    HTTP 503 from /health means the model is still loading. Wait for the external process to become ready. OpenClaw does not restart it.

    Tools are disabled

    Inspect the active model properties:

    bash
    curl http://127.0.0.1:8080/props

    Check chat_template_caps.supports_tools and chat_template_caps.supports_tool_calls. Start llama-server with Jinja enabled and a tool-capable template. OpenClaw does not guess tool support from the model name.

    Router discovery loaded a model

    OpenClaw property requests include autoload=false, and model-list requests do not include reload=1. Check other clients and the server's --models-autoload setting if a model loads outside an inference request.

    Authentication fails during inference

    Health and model-list endpoints can remain public while chat inference requires a key. Set LLAMA_SERVER_API_KEY to the value expected by llama-server or its reverse proxy, then rerun setup or restart the Gateway so it can read the new environment value.

    Structured output fails

    Use a current official llama.cpp release. OpenClaw maps JSON Schema requests to llama-server's json_object schema shape so structured output also works with older external server builds.

    Was this useful?
    On this page

    On this page