Laya Release Notes: What's New in 0.4.22

Laya moves fast: the laya package went from 0.2.1 to 0.4.22 on PyPI between September 19 and November 3, 2026. This page summarises what changed in each release that matters for developers, linked to the upstream sources.

Install or upgrade:

pip install -U laya

Sources: PyPI release history and the upstream README. The checkpoints themselves have not changed across these releases.

0.4.4 to 0.4.22 — October 12 to November 3, 2026

Nineteen releases in three weeks. None changes the checkpoints, and default answers from the Python package stay the same. The work went into ports to other languages, new tools around the HTTP server, and a long run of fixes that turn bad input into a clear error instead of a crash. Sources: the GitHub releases for each version.

Laya outside Python

Official Rust SDK (0.4.11). laya-rust runs the three checkpoints in-process on ONNX Runtime, with no Python process at runtime, tested against outputs recorded from the Python package. predict_long followed in 0.4.13, predict_batch, the min_confidence gate and jev_confidence in 0.4.16, and per-language temperatures in 0.4.17. The package is named laya-onnx and imported as laya; on November 4 it was not on crates.io yet, so add it from the repository and export the ONNX model once with the bundled script. See Laya in Rust.
Go runs the model (0.4.8 onward). laya-go gained Predict and PredictLong over ONNX Runtime (0.4.8), the min_confidence gate and per-language temperatures (0.4.9), presets, the embedding shortlist and email cleaning (0.4.14), and is porting the router in stages (0.4.14, 0.4.17). Running the model needs its cgo onnx module.
.NET catches up. Laya.Onnx added PredictLong (0.4.12), batch prediction and router batches plus the min_confidence gate (0.4.16), and calibration files and per-language temperatures (0.4.17).
C++ is early work (0.4.17, 0.4.18). A new laya-cpp port got its build setup and then an ONNX Runtime backend. Its README still describes it as a skeleton, so do not plan on it yet.
TypeScript. decideBatch and answerConfidence (0.4.11), local option-stability diagnostics (0.4.13), predictLong on Agent and Router (0.4.21), and many fixes that make it match Python on language detection, option order and token budgets.

Server, HTTP client and MCP

Large choices in batches (0.4.7). /v1/systemone/batch now honours LAYA_TOURNAMENT_GROUP as /v1/systemone has since 0.4.3: a choice with more options than the group is answered by elimination, state by state.
Routing check over HTTP (0.4.19). POST /v1/route returns the checkpoint the server would use (model, repo, reason) without running a prediction; explain: true adds the routing diagnostic.
Thin Python HTTP client (0.4.9, 0.4.13). laya-client, in sdk/python, calls a self-hosted laya-serve using only the standard library, so the application installs no PyTorch, Transformers or NumPy. predict_batch arrived in 0.4.13. It is not on PyPI: install it with pip install ./sdk/python from a checkout.
MCP tools. laya_tournament (0.4.7) picks from large option lists by elimination, laya_predict_long (0.4.11) scans states longer than the model window, and opt-in routing diagnostics arrived in 0.4.18.
Server hardening (0.4.15). A stalled upload no longer holds a request slot, a caller's batch_size can only shrink a pass, and the docs and schema pages are hidden when a bearer key is required.

Python package

OptionStabilityHook (0.4.12). An opt-in hook that asks each choice again with the options shuffled, reversed and relabelled, in the same forward pass, and reports how stable the answer is: a reliability signal next to answer_confidence.
Agent frameworks (0.4.11). laya[agno] routes and guards Agno agents, and LayaToolSelector (in laya.integrations.tools) picks the tool or function for a query in one forward pass, with a confidence gate and a fallback.
Language routing (0.4.14, 0.4.17). Question words and auxiliaries now count as English evidence, and Vietnamese no longer goes to the English checkpoint.
Everything else is fixes and speed-ups: malformed questions, non-finite numbers, files with a byte-order mark and out-of-range settings get a named error instead of a traceback, and batch encoding and language detection got faster.

0.4.3 — October 11, 2026

A smaller release: one new server option, two additions to the TypeScript SDK, and documentation that now matches the 0.4.0 routing default. Source: the 0.4.3 release notes.

Large choices on the server: LAYA_TOURNAMENT_GROUP. /v1/systemone refused a choice question with more than 100 options. Set LAYA_TOURNAMENT_GROUP=16 (any value from 2 to 255; unset means off) and a choice with more options than that is answered by elimination with predict_tournament, and the cap rises to 255, the same as Jev's. Smaller choices and other question types take the usual path. The answer keeps Jev's shape: every option appears in probabilities, an option knocked out in an early round gets 0.0, and x_tournament lists the finalists and rounds (left out under LAYA_JEV_STRICT). Upstream's one measurement, on CLINC150 with all 150 intents and the English checkpoint on CPU: 86.7% accuracy, median 0.77 s per request; a single pass over 50 options scored 82.7%, the tournament over the same 50 scored 94.0%. Treat it as one dataset and measure on your own labels.
TypeScript SDK catches up. predictTournament comes to Node and browser callers, and cached(...) wraps an embedder so a fixed option list (a catalog, an intent list) is embedded once instead of on every shortlistChoice call.
laya.jev_confidence(answer). The formula behind the served x_jev_confidence field is now importable, so in-process callers get the same Jev-style number without going through the HTTP server.
Docs now say multilingual. The HTTP API page and the server's own help still said the fallback for undecided text was english. 0.4.3 corrects them: when LAYA_DEFAULT_MODEL is unset, a 0.4.x server uses multilingual, as described under 0.4.0 below. Set english only if most of your traffic is English.
Fixes. The MCP server reports a dropped option_order instead of refusing the call; the TypeScript SDK's confidence fields and error text now match the HTTP responses and the Python SDK.

0.3.28 to 0.4.2 — October 5–10, 2026

Five releases in six days. The one change that can alter answers without any code change on your side is in 0.4.0: text whose language Laya cannot identify now goes to the multilingual checkpoint. The rest adds a one-command fine-tuning tool, a way past the option limit and a Jev-style confidence field. Sources: the release notes for 0.3.28, 0.3.29, 0.4.0, 0.4.1 and 0.4.2, and the new upstream documentation site, nandhakishorm.github.io/laya.

Undecided text now defaults to the multilingual checkpoint (0.4.0). When language detection cannot place a text (too short, or only content words such as "Quero cancelar"), Router used to fall back to english; it now falls back to multilingual. Text detected as English still goes to the English checkpoint. Upstream's reason: on its 51-language sweep the multilingual checkpoint leads on 50 languages and trails only on English (0.710 against 0.820), while the English checkpoint fails on other languages and stays confident, so a min_confidence gate cannot catch it. If most of your traffic is English (upstream puts the break-even at about 62%), set Router(default="english"). laya-serve passes nothing to Router when LAYA_DEFAULT_MODEL is unset, so a server follows the new default too; set LAYA_DEFAULT_MODEL=english to keep the old behaviour.
laya-train: fine-tune in one command (0.3.28, 0.3.29). laya-train --data tickets.csv --text-column body --label-column department --base english --out ./tickets-checkpoint builds the training items, fine-tunes on one device (CUDA, MPS or CPU), fits calibration temperatures on a slice held out before training and saves a checkpoint that laya.load opens, with the question saved beside it as questions.json. A JSONL file with several questions per case works too. --dry-run reports rows, skipped questions and the number of optimizer updates without training; --eval takes a held-out file and prints accuracy, ECE and Brier score before and after, also written to train_report.json; --target-error fits abstention thresholds. The Python equivalent is laya.train.finetune. See our fine-tuning guide.
More options than fit in one pass: predict_tournament (0.3.28). laya.predict_tournament(agent, state, questions) answers the labels in groups of at most 16, all groups in one forward pass, then asks again over the group winners, so up to 256 labels take two calls and no embedding model. With the English checkpoint, upstream measured BANKING77 (77 labels) rising from 0.430 to 0.610 and CLINC150 (150 labels) from 0.625 to 0.876, at about twice the latency of a single question. The probabilities cover the finalists only.
Your own checkpoints by name (0.3.28, 0.4.1). Router(models={...}) and Router.register(name, source) add fine-tuned checkpoints beside the three built-in ones, selected with model= like a built-in. On the server, LAYA_EXTRA_MODELS does the same from a JSON object such as {"tickets": "/models/tickets-checkpoint"}; registering a name that re-points english serves a fine-tune without changing any client (0.4.1).
x_jev_confidence (0.4.2). choice and score answers from /v1/systemone now also carry the confidence Jev would report for the same probabilities, computed with TypeSafe's formulas. Laya's own confidence is unchanged, so compare a threshold carried over from Jev against x_jev_confidence, not confidence. The field is left out under LAYA_JEV_STRICT.
Fixes. laya-serve answers a path-like or unpublished model value with a 422 instead of quietly routing to a built-in (0.3.28); the inference backends are back in the wheels (0.3.28); fine-tuning warns when its predictions collapse to the most common label (0.3.29); single-option questions no longer fail in the ONNX runtime (0.4.1); the server serves again after its app lifespan restarts (0.4.1). The Java SDK gains Router.predictBatch and predictTournament (0.4.2).

0.3.24 to 0.3.27 — October 2–4, 2026

Four releases in three days: mostly fixes, plus better-calibrated abstention and two new language SDKs. Sources: the release notes for 0.3.24, 0.3.25, 0.3.26, 0.3.27.

Abstention thresholds per option count (0.3.24). One min_confidence value does not suit every question, because the confidence of a 2-option answer and of a 12-option answer sit on different scales. In upstream's test on the English checkpoint, a single cut let through 24% wrong answers on 2-option questions but only 5% on 12-option ones. Thresholds can now be set per option-count bucket (keyed like temperature_by_options, for example "choice:3-5"), and the evaluation tools add Brier score, AURC and selective accuracy to measure abstention.
Histogram-binning calibration (0.3.25, fixed in 0.3.27). agent.fit_binning(records) fits a binning map on top of the temperature calibration, and save_calibration stores it with the checkpoint. Once a map is installed, answer_confidence is reported on the binned scale, so a threshold fitted without the map no longer gates correctly. 0.3.27 adds fit_abstention_thresholds(..., binning_map=agent.binning_map): if you use a binning map, refit your thresholds with it.
laya-serve (0.3.24, 0.3.25). LAYA_IDLE_UNLOAD_SECONDS (off by default) unloads the checkpoints after a quiet period and frees the memory between bursts. The MCP server can forward to one shared laya-serve instead of loading its own model: set LAYA_BASE_URL, plus LAYA_API_KEY and LAYA_REMOTE_TIMEOUT. A large /v1/systemone/batch request (up to 64 states × 64 questions) is now split across several forward passes instead of being collated into one tensor, and its total_usage reports output tokens instead of 0.
.NET and Java (0.3.26, 0.3.27). laya-dotnet (package Laya.Onnx) and laya-java run the ONNX-exported checkpoints in-process, with no Python sidecar, and are tested against outputs recorded from the Python package. Neither is on NuGet or Maven Central yet, so build them from the repository. The TypeScript port (laya-ts) gains the per-bucket thresholds and histogram binning.
Fixes worth upgrading for. laya.load() no longer crashes the interpreter when TensorFlow is installed but broken (0.3.27); predict_batch and route_batch accept per-call hooks (0.3.27); load() resolves checkpoint names and aliases, and the LangChain wrapper's .batch() forwards lang and min_confidence like .invoke() (0.3.24); a CUDA device of auto is accepted (0.3.25).

0.3.23 — October 1, 2026

Mostly fixes, plus the per-request controls reaching every way of calling Laya. Source: the 0.3.23 release notes.

Batch requests get the same controls. POST /v1/systemone/batch now forwards the per-call controls that /v1/systemone already forwarded in 0.3.22 (such as task, lang and min_confidence). The MCP tools, the CLI, the Python SDK and the LangChain, CrewAI and LlamaIndex wrappers pass them through as well.
predict_long fix. It could skip part of a long document while reporting that it had read all of it. Upgrade if you use predict_long.
Abstention state on every answer. Each answer now reports the state of the abstention gate, so you can see when min_confidence made Laya decline to answer. Thresholds in the integrations now use answer_confidence, not the entropy-based confidence.
ONNX export. --quantize now defaults to per-tensor quantization; per-channel quantization broke the model. Export also works again on torch older than 2.9.
Languages and benchmarks. Swedish language detection, a Chinese reliability evaluation and a Spanish benchmark for phone-conversation turns with a fine-tuning recipe.
Docker. Images can bake the checkpoints in at build time from ModelScope, which helps where Hugging Face is slow or blocked.
Fixes. A GPU out-of-memory fallback no longer disturbs other requests in flight, and a truncated checkpoint download is repaired instead of trusted.

0.3.22 — September 29, 2026

A large merge of community pull requests, mostly for self-hosting and fine-tuning. Source: the 0.3.22 release notes.

Batch HTTP endpoint. laya-serve adds POST /v1/systemone/batch: send a list of states with one shared questions object and get one result per state plus a total_usage sum. Up to 64 states per request; more returns 413.
More controls reach /v1/systemone. Before 0.3.22 the server read task, lang, lang_guess and min_confidence from the request body and silently ignored them. They are now forwarded, next to model, max_len and head_max_len. Hook settings (hooks, hooks_raise, hooks_timeout and the like) are refused instead of dropped.
Behind a reverse proxy. Set LAYA_ROOT_PATH (for example /laya) when a proxy publishes the server under a URL prefix, so the OpenAPI docs use the public path.
Fine-tuning without two T4s. research/scripts/finetune_single_device.py runs the notebook's training loop on one GPU or on CPU, for any checkpoint including laya-multilingual, and notebooks/laya_finetune_typed_decisions_mps.py targets Apple Silicon.
agent.warmup(). With compile=True, call it once at startup so the compile time does not land on the first live requests. Nothing changes if you do not call it.
MCP. The decision tools accept min_confidence.
TypeScript. The in-repo TypeScript port gains predict_batch / route_batch, answer_confidence abstention, per-checkpoint SHA-256 checks and the same input validation as Python. A typed HTTP client (@laya/typescript-sdk) is in the repository but not yet published on npm.
Fixes. DecisionResult exposes answer_confidence, predict_long merges token usage across windows, and Docker Compose now forwards LAYA_REVISION so a pinned revision actually applies.

0.3.21 — September 27, 2026

The biggest release since 0.3.13. Source: the 0.3.21 release notes.

Long documents: predict_long. agent.predict_long(state, questions, window=256) splits a state that is longer than the context window into overlapping windows, scores them in shared batches and keeps the answer from the most decisive window. Each answer reports which window decided it (answer["window"]). The probability is that window's, not a calibrated number for the whole document. Router.predict_long works the same way.
Opt-in abstention: min_confidence. predict, predict_batch, decide and decide_batch accept min_confidence=. Answers whose answer_confidence falls below it are flagged low_confidence: True, and decide returns None for them. It is off by default.

res = agent.predict(state, questions, min_confidence=0.85)
if res["answers"]["department"].get("low_confidence"):
    send_to_human(state)

Evaluation CLI. laya-evals (also laya eval) scores a labelled JSONL file on a checkpoint, fails when accuracy or calibration error crosses a threshold, and compares a report to a baseline, so it can run as a CI quality gate. See the evals guide.
ONNX catches up with PyTorch. ONNXAgent gains predict_batch, predict_long and decide_batch, and scripts/export_onnx.py --quantize writes an INT8 copy for CPU.
Batch everywhere. decide_batch, laya --batch FILE, new MCP batch tools (laya_predict_batch, laya_route_batch, laya_decide), and LangChain batch() / abatch() run on shared forward passes. New LlamaIndex selectors (laya[llamaindex]) and CrewAI routing (laya[crewai]).
Per-request token budget. max_len and head_max_len now reach laya-serve (capped by LAYA_MAX_TOKEN_BUDGET), the CLI (--max-len, --head-max-len), MCP tools and LangChain.
Operations. LAYA_MAX_LOADED and LAYA_REVISION; /health reports the device each checkpoint really runs on; a busy 503 carries Retry-After; POST /v1/systemone returns Server-Timing and X-Inference-Time-Ms headers.
Stricter inputs. A null or duplicate choice label, a None state and non-dict questions are refused with an error that names the problem.
Fixes. Router.predict_batch now applies each request's lang (so lang_temperatures work in batches), and very long inputs no longer stall language detection or the TypeScript tokenizer.

0.3.19 and 0.3.20 — September 24, 2026

Long documents on laya-multilingual. The checkpoint ships with a 1,024-token limit that cuts long documents off, but its encoder reads up to 8,192 tokens. Upstream now documents max_len=8192 for long inputs and publishes a measured table (0.3.19), then recommends 8,192 outright (0.3.20):

result = router.predict(long_document, questions, model="multilingual", max_len=8192)

What the measurements show. With up to about 4,000 tokens of text before the question, 16 to 18 of 20 requests were answered correctly; beyond that results varied (8 to 17 of 20), so check long-document accuracy on your own data. Short inputs give identical answers with max_len=8192, and speed follows the real input length: a 4,000-token input took about 1.7 s on an Apple GPU.
Name the checkpoint. Pass model="multilingual" for long documents; long, mostly English text would otherwise route to the English checkpoint, which is limited to 512 tokens.
The benchmark is reproducible with research/scripts/bench_long_context.py.

0.3.14 to 0.3.18 — September 24, 2026

Documentation-only releases: a documentation site and DeepWiki badges, installation and quickstart moved to the top of the README, a multilingual quickstart, and a fine-tuning callout. No inference code changed.

0.3.13 — September 24, 2026

Hardened inference and runtime parity (#319):
- laya-serve enforces body limits while streaming and answers malformed non-ASCII bearer headers with 401.
- The ONNX runtime now matches the PyTorch Agent on empty questions, newest-turn truncation, tokenizer compatibility and CUDA-provider fallback.
- The GPU fast path handles one-option confidence correctly, serializes CUDA-graph calls, restores the stock forward before a CPU retry after out-of-memory, and rejects oversized sequences with a clear error.
- LangChain remote calls reject cross-origin and HTTPS-downgrade redirects; lazy Router initialization is thread-safe in the default Router and the MCP server.
- The TypeScript runtime moves closer to Python on nested tokenizer layouts, noul labels, truncation, JSON serialization and temperature handling; CLI choice probabilities render correctly.
TypeScript hooks. laya-ts gains BaseHook and process-wide default hooks, matching Python (#321).
Research: two definitions of confidence. A new script recomputes calibration under max(p) and an entropy-based confidence from the recorded Chinese workplace diagnostic (#327). On that diagnostic (64 AI-assisted synthetic Chinese cases, 3 repeats, the multilingual base checkpoint without fine-tuning), Laya answered 31% correctly against 99% for Jev. Upstream describes it as a diagnostic, not a general ranking; it does not cover the English or fine-tuned checkpoints.
Contributor rules. An AGENTS.md with machine-readable contribution rules.

0.3.12 — September 24, 2026

Correct routed batches. Router.predict_batch keeps two requests apart when their choice options are the same but in a different order, so every request gets the same answer as its own predict call (#166). Predict hooks now run for every request in a batch, so a redaction hook also covers batched traffic.
Schema-driven decisions. agent.decide(state, schema=...) accepts a JSON schema or a pydantic model and returns typed values in one forward pass:

schema = {
    "type": "object",
    "properties": {
        "department": {"type": "string", "enum": ["billing", "support", "sales"]},
        "urgency": {"type": "integer", "minimum": 0, "maximum": 2},
        "needs_human": {"type": "boolean"},
    },
}
agent.decide("I was charged twice, refund me.", schema=schema)
# {"department": "billing", "urgency": 2, "needs_human": True}

Faster, same answers. The state is tokenized once per call instead of once per question; predict_batch(..., sort_by_length=True) reduces padding on mixed-length batches; Apple GPUs use fp16 once a call has enough question rows.
More hooks. A BaseHook class to subclass, and process-wide default hooks for tracers and metrics.
Per-language calibration. lang_temperatures= applies your own per-language temperatures; the Router passes the detected language through. Nothing ships with values.
answer_confidence. Every answer also reports max(p), next to the unchanged confidence.
Fixes. A blank lang= falls through to detection; laya-serve enforces its body-size limit on chunked uploads.
CLI and TypeScript. laya "..." --preset triage; laya-ts gains hooks, one-pass state tokenization, and routing and e-mail parity with Python.

0.3.11 — September 23, 2026

Routed batches. Router.predict_batch(requests) routes each request, groups by checkpoint and question set, and scores each group in shared forward passes. Each request can set its own model, task, lang or lang_guess.
Prediction hooks. Opt-in hooks around every decision on Agent, Router and ONNXAgent — audit, trace, redact, cache or gate results.
transformers 4.x and Apple GPUs. Checkpoints re-saved by transformers 5 load with correct RoPE settings on 4.x; predict() no longer crashes on MPS builds without an autocast backend.
Stricter noul questions. A criteria dict keyed anything other than true/false is rejected; use labels to change the wording.
Safer HTTP server. Timing-safe API key checks, request size and question limits (413), 400 for malformed JSON, no path leaks. Docker Compose binds to loopback by default.
More hardware. Native ARM64 and DGX Spark container builds, and a local web GUI demo.

0.3.10 (and 0.3.9) — September 23, 2026

0.3.10 changed only the README; its code equals 0.3.9. Everything new since 0.3.6:

About 10x faster loading. laya.load() drops from about 22 s to about 2 s on CPU with bit-identical answers.
import laya no longer loads torch, so routing, language detection and e-mail cleaning run in lightweight processes.
Batch scoring with agent.predict_batch(states, questions).
Opt-in fast paths: a TileLang GPU path (laya.load(..., fast=True)), torch.compile, and ONNXAgent on ONNX Runtime.
Local serving: the Jev-compatible laya-serve, the laya CLI, an MCP server, LangChain/LangGraph integrations and the laya-ts TypeScript package. See Self-host Laya.
Better routing for plain-ASCII Spanish, Italian, Portuguese and French, CJK text containing Latin brand names, romanized Bangla and Azerbaijani.
Fine-tuning notebook fixes: calibration is fitted on a held-out slice rather than training data (#186).

Earlier releases

Version	Released
0.3.6	September 22, 2026
0.3.5	September 21, 2026
0.3.4	September 20, 2026
0.2.1 – 0.3.3	September 19, 2026

Upgrade notes

Python 3.10 or newer is required.
Re-check confidence thresholds after upgrading if you use batching or mixed precision; upstream notes that changed batch shapes can cause small floating-point differences near decision thresholds.
For long documents on laya-multilingual, pass max_len=8192 and model="multilingual" (0.3.19+).

Last verified against the upstream releases and PyPI: November 4, 2026.