laya-typed-decisions: The Fine-Tuned Laya Checkpoint (0.766 Accuracy)

convaiinnovations/laya-typed-decisions is the base English Laya checkpoint fine-tuned on the typed-decisions workflows. It is the source of Laya's most-quoted number — 0.766 accuracy — and it is a specialist.

Hugging Face	convaiinnovations/laya-typed-decisions
Fine-tuned from	laya (ModernBERT-large)
Parameters	421M
Default context	1,024 tokens (256 for question and options)
Languages	English only
Training data	1,200-case training split (6,000 decisions) of the typed-decisions benchmark
License	Apache 2.0

Source: model card. Figures are upstream measurements unless noted.

Benchmark

400 test cases, 2,000 decisions, official test split:

Model	Accuracy	Soft acc	Brier	ECE	Score MAE
laya-typed-decisions	0.766	0.471	0.062	0.213	0.242
TypeSafe Jev 1.13.0 (published)	0.727	0.580	0.148	0.144	0.391
Teacher self-agreement ceiling	0.735
ModernBERT-base specialist (published)	0.646
laya (not fine-tuned)	0.362	0.332	0.316	0.175	0.694
Random guess	0.318

Upstream notes that the Jev figures are third-party published, not measured by the Laya project, and that sample sizes and prompts differ. Treat the comparison as indicative.

By workflow

Workflow	Accuracy
Invoice processing	0.804
Security incidents	0.766
Customer service	0.764
Agent-trace observability	0.730

By primitive

Type	Accuracy	ECE	n
noul	0.857	0.192	600
choice	0.733	0.255	600
score	0.723	0.199	800

How to read the 0.766

• It is a fine-tuned result on the benchmark's own training split, not base-Laya zero-shot accuracy. The base checkpoints score 0.342–0.362 on the same test.
• It beats Jev on argmax accuracy, Brier and score MAE, but trails Jev on soft accuracy and ECE.
• It is evidence that Laya fine-tunes well, which is the main reason to read the fine-tuning guide.

Use it

import laya

agent = laya.load("convaiinnovations/laya-typed-decisions")
result = agent.predict(state, questions)

Or route to it explicitly:

from laya import Router

router = Router()
router.predict(state, questions, model="typed-decisions")

The Router will not pick this checkpoint automatically unless you construct it with auto_task_detection=True; it should not be a silent default. If it is on a hot path, keep it resident with router.preload(["typed-decisions"]).

Limits

• It is a specialist. Outside the four workflows, expect it to behave like the base laya checkpoint, or worse.
• Treat its confidence as uncalibrated. Its inherited temperature_by_options overrides the per-type temperatures fitted for it, and those per-type temperatures were fitted on training items (#186). Refit on held-out data before thresholding.
• English only; use laya-multilingual for other languages.
• Keep choice questions under ~20 options.

Compare the family

Checkpoint	Encoder	Params	Context	Use it for
laya	ModernBERT-large	421M	512	English
laya-multilingual	mmBERT-base	322M	1024	100+ languages
laya-typed-decisions	ModernBERT-large	421M	1024	four typed-decisions workflows

See also Laya vs Jev and the benchmark tracker.

Last verified: September 24, 2026.