Laya in Rust: Official SDK, sys1 and ollaya

cargo add laya-onnx --git https://github.com/NandhaKishorM/laya

let engine = laya::LayaEngine::from_directory("path/to/laya-multilingual")?;
let result = engine.predict(state, &questions)?;
// As an HTTP server instead: cargo install sys1 --features cpu

There are two ways to run Laya in Rust, and neither needs Python on the serving machine:

• Inside your own program, with the official Rust SDK from the Laya repository. A decision is a function call, with no server and no network hop.
• As an HTTP server, with one of two community projects, sys1 or ollaya. Both answer the Jev-compatible POST /v1/systemone, so any Jev or Laya client works against them.

Official Rust SDK: laya-onnx

Since Laya 0.4.11 the upstream repository ships laya-rust, a port of the Python package that runs the English, multilingual and typed-decisions checkpoints on ONNX Runtime through the ort crate. It reproduces Python's tokenization, calibration and answers, and is tested against outputs recorded from the Python code. CPU is the default; cuda and directml are cargo features.

The package is named laya-onnx and imported as laya. On November 4, 2026 it was not on crates.io yet, so add it from the repository:

cargo add laya-onnx --git https://github.com/NandhaKishorM/laya

The SDK reads ONNX files, and the ONNX export is not on Hugging Face yet. Export it once with the bundled script; this one step needs Python, serving does not:

git clone https://github.com/NandhaKishorM/laya && cd laya
pip install "torch>=2.9" onnx onnxscript onnxruntime transformers safetensors huggingface_hub numpy
pip install -e .
python laya-rust/tools/export_onnx.py --model all

Point LayaEngine::from_directory (or LAYA_ONNX_ROOT) at a checkpoint folder from the export, then ask typed questions:

use laya::{LayaEngine, Question, QuestionSet};
use serde_json::json;

let engine = LayaEngine::from_directory("path/to/laya-multilingual")?;
let questions = QuestionSet::new().with("department", Question::choice(
    "Which department should handle this request?",
    [
        ("billing", "invoices, payments, refunds"),
        ("technical", "bugs, outages, system errors"),
        ("other", "everything else"),
    ],
)?)?;
let result = engine.predict(json!({"body": "We were billed twice for March."}), &questions)?;
println!("{}", result["department"].as_choice().unwrap().choice); // billing

The SDK covers predict, predict_batch, predict_long for states longer than the model window, the min_confidence abstention gate, per-language temperatures, routing, language detection, email cleaning and the embedding shortlist. Histogram-binning calibration and per-call hooks are not ported yet. Calls are synchronous: create one LayaEngine, share it behind an Arc, and wrap calls in spawn_blocking from async code.

sys1: a Rust server built on candle

sys1 is a System One API server built with tokio, axum and serde, running Laya's ModernBERT encoder and decision heads on candle. Pick the backend when you install it:

cargo install sys1 --features cpu
cargo install sys1 --no-default-features --features metal   # Apple Silicon
cargo install sys1 --no-default-features --features cuda    # NVIDIA GPU

Start it with a Laya checkpoint:

sys1 --model-id convaiinnovations/laya --dtype auto

It listens on port 3000 and answers POST /v1/systemone and POST /v1/decide. Requests are batched dynamically by token count, and the project reports about 14 ms per query on an NVIDIA RTX Pro 6000. The README lists the English Laya checkpoint today, with more models planned.

ollaya: Ollama-style commands for decision models

ollaya works like Ollama: a daemon plus a CLI that pulls and runs decision models by name. It runs on ONNX Runtime on CPU, CUDA on NVIDIA GPUs, and natively on Apple Silicon.

curl -fsSL https://ollaya.dev/install.sh | sh
ollaya serve
ollaya run laya --preset triage "My payment failed twice"

laya routes between laya:en (ModernBERT-large, 421M) and laya:multilingual (mmBERT-base, 322M). ollaya also serves other decision models, such as Decider and NLI classifiers. It listens on port 11435, and the official TypeSafe SDK works unchanged with TYPESAFE_BASE_URL=http://localhost:11435. ollaya ships small ONNX graphs and reads the original weights from Hugging Face, checked by sha256, instead of re-hosting them.

Which one to use

Choose the SDK when the decision runs inside a Rust service; choose sys1 or ollaya when several programs, or programs in other languages, share one Laya server. sys1 and ollaya are community projects, not part of the upstream package; check each repository for its license and current model support before you deploy.

Related

• Laya and Ollama: the Ollama-style workflow with ollaya, compared with other local options
• Self-host Laya with the upstream Python server
• Jev alternatives
• Laya runtimes overview

Last verified: laya-rust against the upstream repository on November 4, 2026; sys1 and ollaya on September 25, 2026.