laya (English Checkpoint): ModernBERT-large, 421M, Benchmarks & Limits

convaiinnovations/laya is the default English checkpoint of the Laya family and the root of the family's Hugging Face repository.

Hugging Face	convaiinnovations/laya
Encoder	ModernBERT-large
Parameters	421M
Default context	512 tokens (192 for question and options, ~320 for the state)
Download size	~808 MB
Languages	English
License	Apache 2.0

Sources: model card and upstream README. All figures below are upstream measurements unless noted.

Load it

import laya

agent = laya.load("convaiinnovations/laya")
result = agent.predict(state, questions)

Or let the Router send English input here automatically:

from laya import Router

router = Router(preload=True)
result = router.predict(state, questions)   # English text -> "english" checkpoint

Measured performance

Speed (Tesla T4): 39.5 ms for one question, 84.5 ms for 5, 158.6 ms for 10 (15.9 ms per question) and 771 ms for 50.

English tasks:

Task	Accuracy	Note
AG News	0.947	in training mix
BoolQ	0.830	in training mix
XNLI (English)	0.860
MASSIVE intent (English, 20 options)	0.783
DAIR Emotion	0.573	held out
prompt-injections	0.698	held out, n=116
SST-5 (ordinal)	0.372	held out

Typed-decisions benchmark, zero-shot: 0.362 — above the 0.318 random baseline but below the 0.461 majority-class baseline. That benchmark's strong result belongs to the fine-tuned laya-typed-decisions checkpoint.

When not to use it

• Anything that is not English. Across 51 languages this checkpoint macro-averages 0.227 on MASSIVE intent, and only 23 of 51 languages clear three times random. On Khmer it scores 0.000 accuracy at 0.952 confidence — it stays confident while wrong, so confidence gating cannot catch it. Route non-English text to laya-multilingual.
• Large label sets. With a 192-token head budget, a 77-option question leaves only a few tokens per label (Banking77: 0.425). Keep choice questions under ~20 options, raise head_max_len, or use predict_shortlist.
• noul on sensitive decisions without checking. Upstream documents that on this checkpoint noul can follow its false: / true: labels instead of the state (#156). Validate on your own data, or use the labels override or a two-option choice with neutral keys.

Calibration

The checkpoint is over-confident as shipped. Refitting one temperature per (question type, option count) on held-out data moves its mean ECE from 0.466 to 0.081. See Fine-tune Laya.

Compare the family

Checkpoint	Encoder	Params	Context	Use it for
laya	ModernBERT-large	421M	512	English
laya-multilingual	mmBERT-base	322M	1024	100+ languages
laya-typed-decisions	ModernBERT-large	421M	1024	four typed-decisions workflows

Back to the model overview.

Last verified: September 24, 2026.