Laya Python / PyTorch

pip install laya

Requirements

• Python 3.10 or newer. The floor comes from the dependencies: huggingface_hub 1.x, transformers 5.x and torch 2.14.
• CPU, NVIDIA CUDA, Apple MPS and Intel XPU devices are supported by PyTorch; pick a specific PyTorch build first if you need CPU-only or a particular CUDA version.
• Disk: about 808 MB for the English checkpoint and 647 MB for the multilingual one. Only the checkpoint you request is downloaded.

Install into a virtual environment and verify without loading a model:

python3 -m venv .venv
.venv/bin/python -m pip install laya
.venv/bin/python -I -c "import laya; print(laya.__version__)"

Two ways to call it

Router (recommended). Detects the script and language of each state in under a millisecond and sends it to the right checkpoint:

from laya import Router

router = Router(preload=True)            # keep checkpoints resident
result = router.predict(state, questions)
print(result["routing"])                 # which checkpoint answered, and why

Single checkpoint. For a dedicated pipeline:

import laya

agent = laya.load("convaiinnovations/laya")                              # English
agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")  # 100+ languages
result = agent.predict(state, questions)

Measured speed (upstream, Tesla T4)

On CPU with Router(preload=True) upstream reports 193–464 ms per request.

Production settings that matter

• Preload in servers. Router() keeps two checkpoints resident; max_loaded=1 reloads on every language switch (7.4 s median on CPU).
• Batch when you can. agent.predict_batch(states, questions) shares forward passes. On a GPU upstream measured roughly 10 ms → 1 ms per decision; on CPU batching alone may not help, and sort_by_length=True can reduce padding.
• Opt-in speedups: laya.load(..., fast=True) (TileLang GPU path), Agent(compile=True) (torch.compile), and ONNXAgent for ONNX Runtime.
• Schema-driven calls: agent.decide(state, schema=...) takes a JSON schema or pydantic model (0.3.12+).
• Hooks for audit logging, PII redaction, caching and metrics, on Agent, Router and ONNXAgent.
• If laya.load() hangs, TensorFlow may be installed; run with USE_TF=0.

What this runtime is for

It is the reference: the other runtimes on this site port or wrap it, and fine-tuning (guide) only exists here. Start here unless you specifically need Node.js, MLX or a separate HTTP server. To expose it over HTTP, see Self-host Laya.

Source: NandhaKishorM/laya · Apache 2.0.

Last verified against the project README: September 24, 2026.