Self-Host Laya: Jev-Compatible HTTP API, CLI, Docker & MCP Server

The upstream laya Python package can do more than run inside your own script. It ships a self-hosted HTTP server that speaks the same wire protocol as TypeSafe's hosted Jev API, a command-line tool for quick tests, and an optional MCP server for agent clients.

This guide collects those options in one place. Primary source: Laya on GitHub.

Option 1: the Jev-compatible HTTP server (`laya-serve`)

laya.serve exposes the Router over HTTP at POST /v1/systemone. Upstream states that Laya's answer payload is already schema-identical to what Jev returns (choice / score / noul answers plus an {input_tokens, output_tokens} usage block), so an existing Jev client only needs its base URL changed.

pip install "laya[serve]"                     # adds fastapi, uvicorn, python-multipart
LAYA_DEVICE=cuda LAYA_PRELOAD=1 laya-serve    # binds 0.0.0.0:8000, preloads all 3 checkpoints

Send a request:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": {"body": "billed twice, refund please or we cancel"},
  "questions": {"dept": {"type": "choice", "instructions": "which team?",
                "criteria": {"billing": "refunds", "tech": "bugs"}}}
}'

Configuration

Variable	What it does
`LAYA_HOST`, `LAYA_PORT`	Bind address and port (default `0.0.0.0:8000`)
`LAYA_DEVICE`	`cuda`, `cpu` or `mps`, passed straight to torch
`LAYA_PRELOAD`	Build checkpoints at startup instead of on first request
`LAYA_MODELS`	Comma list of checkpoints to preload
`LAYA_THREADS`	Cap torch intra-op threads on CPU (keep at or below physical cores)
`LAYA_AUTO_TASK`	Enable automatic task detection
`LAYA_API_KEY`	When set, clients must send `Authorization: Bearer <key>`

A client's model field is honoured when it names a Laya checkpoint (english, multilingual or typed-decisions); otherwise the router selects one by script and language.

Security defaults

Since 0.3.11 the server uses timing-safe API key checks, enforces request size and question limits (HTTP 413), returns 400 for malformed JSON and avoids leaking paths in errors. 0.3.12 also enforces the body-size limit on chunked uploads. If you expose the server beyond localhost, set LAYA_API_KEY and put it behind your usual TLS proxy.

Option 2: Docker

Upstream maintains a Docker Compose quickstart in docs/docker.md. It runs a sample request on CPU and keeps downloaded models between runs. The Compose file binds laya-serve to loopback by default and adds a healthcheck. Native ARM64 and DGX Spark container builds are also provided.

Option 3: NixOS

The repository is a Nix flake. On a machine with an NVIDIA GPU, nix run .#laya-serve builds and serves. For a NixOS host, import laya.nixosModules.default and enable services.laya-serve, which runs a hardened DynamicUser systemd unit, caches weights under /var/lib/laya-serve and reads an optional bearer token through LoadCredential.

Option 4: the local web GUI

For manual testing without writing a client, examples/server.py in the upstream repository is a self-contained FastAPI app with a request builder that renders choice / score / noul answers as bars, plus /predict and /predict/batch JSON endpoints:

pip install "laya[serve]"
python examples/server.py        # http://127.0.0.1:8000

--no-preload loads checkpoints lazily and --device cuda|cpu|mps pins the device.

Option 5: the `laya` command

Installing the package also installs a laya command:

laya "I was charged twice, please refund"            # routing decision only; offline, no download
laya "Refactor this service" --predict               # full answers (downloads the checkpoint once)
laya "Mein Konto wurde zweimal belastet" --lang de   # force a language
laya "My payment failed twice" --preset triage       # presets: triage, email, guard, moderation, router

Routing alone never downloads a checkpoint, so it returns in milliseconds.

Option 6: MCP server for agent clients

Laya can run as an MCP stdio server so MCP clients such as Claude Desktop or Cursor can call typed decisions as tools (laya_predict, laya_route, laya_preset, laya_status):

pip install "laya[mcp]"
laya-mcp-server

{
  "mcpServers": {
    "laya": {
      "command": "laya-mcp-server",
      "env": { "LAYA_DEVICE": "cpu" }
    }
  }
}

The MCP server preloads english,multilingual by default and leaves typed-decisions lazy. As with the SDK, use it for structured decisions, not open-ended Q&A.

Sizing: latency and memory

Upstream measurements with Router(preload=True): 32.8 ms per request on a T4 GPU and 193–464 ms on CPU. A lazy Router() keeps two checkpoints resident; max_loaded=1 reloads on every language switch, measured at a 7.4 s median on CPU and 10.3 s on a T4. For a server, preload what you serve.

Download sizes from the model card: about 808 MB for the English checkpoint and 647 MB for the multilingual one.

Community serving layers

If you want more than the upstream server, see Arbiter (checkpoint routing, batching, metrics and a Playground) or the macOS MPS runtime. The Runtimes overview compares all options.

Last verified: September 24, 2026.