Laya MPS on macOS

git clone https://github.com/afshinm/laya-mps.git
cd laya-mps
./scripts/serve.sh

laya-mps runs Laya on your Mac's GPU through PyTorch's Metal Performance Shaders (MPS) backend, as a local HTTP server with a browser demo, and focuses on low memory use.

Note the default model: it serves laya-typed-decisions, the checkpoint fine-tuned for customer service, invoices, security incidents and agent traces (what that means). Use --checkpoint english for the general English model.

Requirements

• Apple Silicon Mac (M1 or newer), macOS 14+.
• Git and uv; uv installs Python 3.12 if needed.
• About 4 GB of free disk for the runtime, cache and ~843 MB model.

Start it

git clone https://github.com/afshinm/laya-mps.git
cd laya-mps
./scripts/serve.sh

The first run installs dependencies and downloads the model; later runs work offline. The server listens on 127.0.0.1:8000, with a Pong demo and a 60-second latency benchmark at /demo/.

Memory modes (project README, M5 Pro, 24 GiB)

All three settings run the full model in FP32 and matched exactly on 260/260 decisions. minimal streams layers from disk, which is why it is slower. Switch with ./scripts/serve.sh --memory minimal.

API

curl -sS http://127.0.0.1:8000/v1/decisions \
  -H 'Content-Type: application/json' \
  --data-raw '{
    "state": "The export page returns HTTP 500. Our team cannot finish its work.",
    "questions": {
      "owner": {"type": "choice",
                "instructions": "Which team should investigate this report?",
                "criteria": {"billing": "Payments and invoices",
                             "engineering": "Software failures"}}
    }
  }'

• Endpoints: GET /health, GET /v1/config, POST /v1/decisions, GET /docs, GET /demo/.
• 1–16 questions per request; choice takes 2–26 labels, score takes 2–10 levels.
• Add ?metrics=true for timing and memory, or ?diagnostics=true for tokens, temperatures and logits.
• One inference runs at a time; a busy server answers 503 with Retry-After: 1.
• Not a drop-in Jev endpoint: it uses its own path, a levels field for scores and its own limits. For a Jev-compatible API see Arbiter or upstream's laya-serve.

CLI

uv run --locked laya-mps decide request.json evaluates a request without starting the server; laya-mps doctor checks GPU availability and model readiness.

Source: afshinm/laya-mps · MIT (model weights Apache 2.0).

Last verified against the project README: September 24, 2026.