Laya and Ollama: Run Laya Locally with ollaya
Can you run Laya in Ollama? Not directly. Run Laya locally with ollaya, an Ollama-style CLI and daemon, and compare pip, laya-serve, MLX and Rust options.
Can you run Laya in Ollama? Not in Ollama itself. Ollama runs generative language models, and Laya is a different kind of model: an encoder with decision heads that returns probabilities instead of text. There is no Laya model in the Ollama library.
What you can use instead is ollaya, a community project that runs decision models such as Laya the way Ollama runs LLMs: one binary, pull and run commands, a local daemon and a Modelfile. This page shows how to run Laya locally with ollaya, and how it compares with the other ways to run Laya on your own machine.
Run Laya with ollaya
ollaya (Apache 2.0, v0.7.1 at the time of writing) installs with one command on macOS (Apple silicon) and Linux:
curl -fsSL https://ollaya.dev/install.sh | sh
ollaya run laya --preset triage "I was charged twice for my subscription this month and want a refund."
On Windows, run irm https://ollaya.dev/install.ps1 | iex in PowerShell. A desktop app and Docker images (ghcr.io/ollaya-dev/ollaya, with a :cuda tag for NVIDIA GPUs) are also available.
The triage preset asks several questions at once and prints each answer with its probability, for example the intent (refund), whether the message is urgent and how frustrated the customer is.
Commands that work like Ollama
| ollaya | Does |
|---|---|
ollaya serve | Start the local daemon (the CLI starts it automatically if needed) |
ollaya pull laya | Download a model |
ollaya run laya "…" | Ask questions about a piece of text |
ollaya list, ps, show, rm, stop | Manage models, as in Ollama |
ollaya create my-model -f Modelfile | Bake your own question set into a model |
A Modelfile uses the same idea as Ollama's:
FROM laya
QUESTIONS ./triage.json
PARAMETER precision fp32
Laya models in ollaya
| Model | What it is |
|---|---|
laya | Router: picks the English or multilingual checkpoint from each request's language |
laya:en | English checkpoint (ModernBERT-large, 421M) |
laya:multilingual | 100+ languages (mmBERT-base, 322M) |
laya:typed-decisions | Fine-tuned for the typed-decisions workflows |
ollaya also serves other decision models, including Kev, Von and Decider. It does not re-host weights: it publishes small ONNX graphs that read the original weight files from each author's Hugging Face repository, pinned to a commit and checked by hash. It runs on ONNX Runtime on CPU and CUDA; the project reports that its fp32 exports give the same decision as the PyTorch reference on all 2,383 test questions per checkpoint.
Use it as a local Jev API
The ollaya daemon listens on port 11435 and serves POST /v1/systemone in the same format as TypeSafe's Jev API. Code written for Jev works by changing one variable:
export TYPESAFE_BASE_URL=http://localhost:11435
ollaya mcp also exposes the models to Claude Code, Claude Desktop, Cursor and other MCP clients.
Other ways to run Laya locally
ollaya is the closest to the Ollama experience, but it is not the only option:
| Option | Best for | Install | Runs on |
|---|---|---|---|
| ollaya | Ollama-style CLI, daemon and desktop app | One-line installer | CPU, NVIDIA CUDA |
Python package (pip install laya) | Python code and notebooks; the reference implementation | pip | CPU, CUDA, Apple MPS |
| laya-serve | A Jev-compatible HTTP API from the official package | pip install "laya[serve]" | CPU, CUDA, MPS |
| laya-mlx | Native Apple Silicon inference | pip | Apple MLX |
| sys1 | A Rust server on candle with token batching | cargo | CPU, CUDA, Metal |
Guides: Install Laya, Self-host Laya, Laya on MLX, Laya in Rust (sys1 and ollaya) and all runtimes.
Which one should you pick?
- You like Ollama and want the same workflow: ollaya.
- You write Python or want the official, most up-to-date code:
pip install laya. - You have existing Jev client code: ollaya or laya-serve; both speak the Jev API.
- You are on an Apple Silicon Mac and want native speed: laya-mlx, or ollaya.
- You only have a CPU: any of them works. See Laya on CPU for measured latency.
Try Laya without installing anything
The Laya Playground runs the model in your browser, so you can test your own text and questions before you pick a local setup.
FAQ
Is Laya available on Ollama?
No. Ollama runs generative models, and Laya is an encoder-based decision model that does not generate text. Use ollaya for the same kind of workflow.
Is ollaya made by the Laya team?
No. ollaya is an independent community project (ollaya-dev on GitHub). Laya itself comes from ConvAI Innovations.
Does ollaya need a GPU?
No. It runs on CPU with ONNX Runtime and uses CUDA automatically on NVIDIA GPUs when the driver is present.
Can I use the TypeSafe SDK with a local Laya?
Yes. Point TYPESAFE_BASE_URL at ollaya (http://localhost:11435) or at laya-serve, and keep the rest of your Jev code.
Related
- Install Laya: the Python quickstart
- How to use Laya: write questions and read the results
- Self-host Laya: laya-serve, Docker and MCP
- Jev alternatives: open models that accept Jev requests
Last verified against the ollaya repository: September 26, 2026.