laya-multilingual: 100+ Languages, mmBERT-base 322M, Benchmarks & Limits
Laya's multilingual checkpoint on Hugging Face: mmBERT-base, 322M parameters, 1024-token context, 100+ languages, faster than the English model, plus calibration and known limits.
convaiinnovations/laya-multilingual is the Laya checkpoint for anything that is not English. It is also the faster of the two general checkpoints.
| Hugging Face | convaiinnovations/laya-multilingual |
| Encoder | mmBERT-base (22 layers, hidden 768, 256k vocabulary) |
| Parameters | 322M (307M encoder + decision head) |
| Default context | 1,024 tokens (256 for question and options); encoder supports up to 8,192 |
| Download size | ~647 MB |
| Languages | 100+ (model card lists 51 evaluated languages, including Chinese, Japanese, Korean, Arabic, Hindi) |
| License | Apache 2.0 |
Sources: model card and upstream README. Figures are upstream measurements unless noted.
Load it
import laya
agent = laya.load("convaiinnovations/laya-multilingual")
result = agent.predict(
{"body": "二重に請求されました。返金してください。"},
{"department": {"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {"billing": "invoices, payments, refunds",
"technical": "bugs and outages"}}},
)
With the Router, non-English input reaches this checkpoint automatically. The routing decision is made from the script and language before the forward pass. If most of your traffic is not English, set Router(default="multilingual") so very short, ambiguous Latin-script text also lands here, or pass your own language detector with lang_guess=.
Why it exists
Across all 51 MASSIVE languages (intent classification, 20 options, random = 0.050):
laya (English) | laya-multilingual | |
|---|---|---|
| Macro accuracy | 0.227 | 0.366 |
| Macro ECE (lower is better) | 0.733 | 0.387 |
| Languages clearing 3x random | 23 / 51 | 45 / 51 |
| XNLI, 14 non-English languages | 0.521 | 0.731 |
Per-language examples from the model card: Arabic 0.110 → 0.400, Hindi 0.100 → 0.387, Korean 0.110 → 0.490, Turkish 0.140 → 0.437.
Speed (Tesla T4)
| Questions per call | laya | laya-multilingual |
|---|---|---|
| 1 | 39.5 ms | 32.8 ms |
| 10 | 158.6 ms | 72.3 ms |
| 50 | 771 ms | 337 ms |
Upstream reports 103–332 questions per second batched on one T4.
Known limits
- Ships uncalibrated. Temperatures are all 1.0 and the model is systematically over-confident. Refitting on held-out data moves mean ECE from 0.314 to 0.106. Do this before trusting its probabilities; see Fine-tune Laya.
- Weaker on English than the English checkpoint (0.619 vs 0.684 macro across English suites). Route rather than replace.
- Low-resource languages are weak: Swahili 0.210, Tamil 0.250, Amharic 0.110.
- Position bias on
scorequestions: it rarely picks the first-listed level, in any language (#131). For Englishscorequestions use the English checkpoint; for others validate first. noulcan under-report "true" (#156). A two-optionchoicewith neutralA/Bkeys is a useful cross-check.- Near chance on typed-decisions zero-shot (0.342). Fine-tune for a specific workflow.
- Keep
choicequestions under ~20 options.
For Chinese-language workloads
The checkpoint's language list includes Chinese (zh), and CJK text containing Latin brand names is routed here. Upstream also links a community diagnostic of Chinese workplace decisions. As with every language, measure accuracy and calibration on your own data first.
Compare the family
| Checkpoint | Encoder | Params | Context | Use it for |
|---|---|---|---|---|
| laya | ModernBERT-large | 421M | 512 | English |
| laya-multilingual | mmBERT-base | 322M | 1024 | 100+ languages |
| laya-typed-decisions | ModernBERT-large | 421M | 1024 | four typed-decisions workflows |
Back to the model overview.
Last verified: September 24, 2026.