Laya quickstart

Install Laya AI Locally: Python Quickstart

Install Laya with pip, preload checkpoints with Router, run your first typed decision, and choose the right Laya model for local Python inference.

Install
pip install laya

Python 3.10+. The required checkpoint is downloaded when you first load it. RAM, GPU and disk requirements

Latest release: v0.4.27· Nov 6, 2026
PyPI GitHub
Python
from laya import Router

router = Router(preload=True)

state = {
    "body": "I was charged twice. Please refund the duplicate."
}

questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this?",
        "criteria": {
            "billing": "payments and refunds",
            "technical": "bugs and outages"
        }
    },
    "refund_requested": {
        "type": "noul",
        "instructions": "Does the user explicitly request a refund?"
    }
}

result = router.predict(state, questions)
print(result)
Start from a recipe

Copy complete examples for triage, model routing, guardrails, RAG, and more.

Choose a runtime

Beyond Python, explore MLX, Node.js / ONNX, MPS, and local serving.

Hosted Laya API · Use Laya in production without a server: Jev-compatible API, pay as you go, 20 free calls.

Get an API key

Why preload?

Laya checkpoints are local models. Building or loading a cold model can take seconds, so the upstream project recommends preloading for server workloads.

router = Router(preload=True)

If memory is constrained, the Router also exposes checkpoint-loading controls. Check the upstream documentation before hard-coding operational assumptions because this API is still changing quickly.

Direct checkpoint loading

For a dedicated single-model pipeline:

import laya

agent = laya.load("convaiinnovations/laya")
result = agent.predict(state, questions)

Which checkpoint should I load?

  • laya — start here for English.
  • laya-multilingual — evaluate for non-English workloads.
  • laya-typed-decisions — evaluate when your task resembles the typed-decision workflows used for its fine-tuning.

See Laya Models for details.

Run Laya locally: CPU, NVIDIA GPU or Apple Silicon

Laya runs entirely on your own machine. Choose the device when you load it:

import laya
from laya import Router

agent = laya.load("convaiinnovations/laya", device="cuda")  # NVIDIA GPU
router = Router(preload=True, device="mps")                # Apple Silicon

Use "cpu" when there is no GPU. On CPU, thread settings matter more than you might expect; see Laya on CPU for measured latency and the settings to use.

The first run downloads the weights from Hugging Face: about 808 MB for the English checkpoint and 647 MB for the multilingual one. They are cached, so later runs load from disk. To run without network access, pass laya.load() a local directory that contains the checkpoint files, or set HF_HUB_OFFLINE=1 once the files are cached.

Want an HTTP endpoint instead of Python calls? See Self-host Laya.

Production checklist

Before using Laya for automated actions:

  1. Evaluate on your own labeled data.
  2. Measure warm and cold latency on your hardware.
  3. Test confidence calibration for your domain.
  4. Inspect failures on large choice sets.
  5. Add a fallback or human-review path for low-confidence decisions.
  6. Pin package and model versions.

Next steps

Last verified against the upstream repository: September 25, 2026.