What Is RLCD? Reinforcement Learning for Calibrated Decisions
RLCD (Reinforcement Learning for Calibrated Decisions) explained: how Jev and Laya are trained, RLCD vs RLHF and RLVR, proper scoring rules and calibration.
RLCD stands for Reinforcement Learning for Calibrated Decisions. It is the training approach behind System One decision models such as TypeSafe's Jev and the open-source Laya. Instead of teaching a model to write text people like, RLCD teaches it to return a decision together with honest probabilities: when an RLCD-trained model says 80%, it should be right about 80% of the time.
This page explains what RLCD is, how it differs from RLHF and RLVR, how it works in Laya's open implementation, and what it does and does not guarantee in practice. Sources: TypeSafe's AI primer, which introduces the term, and the Laya model card, which documents Laya's training.
RLCD vs RLHF vs RLVR
TypeSafe places RLCD alongside the two post-training methods that shaped today's language models:
| Method | Full name | Trains the model to | Typical result |
|---|---|---|---|
| RLHF | Reinforcement learning from human feedback | Produce responses people prefer | Chat assistants |
| RLVR | Reinforcement learning with verifiable rewards | Reach answers that can be checked, such as math | Reasoning models |
| RLCD | Reinforcement learning for calibrated decisions | Return a decision with calibrated probabilities, no text | System One decision models |
TypeSafe's argument is that RLHF optimizes for what sounds good to a person. That suits chatbots, but it can reward confident-sounding mistakes, which is a problem when software acts on the answer without a human in the loop. RLCD optimizes for probabilities that match reality instead.
Why calibration matters
A model is calibrated when its stated probabilities match how often it is right. Across many predictions, outcomes given 0.2 should happen about 20% of the time, and outcomes given 0.8 about 80% of the time.
Calibration is what makes a probability usable by code. It lets you:
- Automate above a threshold. Act automatically when confidence is high and send the rest to a person.
- Compare answers. Rank options, documents or tickets by probability and trust the order.
- Combine answers. Multiply or weight probabilities from several questions in your own logic.
Accuracy alone does not give you this. A model that is right 90% of the time but always claims 99% gives you no way to find the 10% it gets wrong.
How RLCD works
TypeSafe has not published the details of how it trains Jev. Laya's model card describes its own RLCD setup:
- The model reports a distribution. For every question it outputs a probability for each option, not a single answer.
- Exploration. During training, zero-mean Gaussian noise is added to the model's logits, so it tries slightly different distributions.
- The reward is a strictly proper scoring rule. Laya uses the log score plus the spherical score, and adds the ranked probability score for ordinal
scorequestions. - Policy-gradient updates. The model is updated with REINFORCE and a group-mean baseline, in the style of GRPO. Multi-turn conversations use TD(λ=1.0) over prefix slices.
The key idea is the strictly proper scoring rule. Under such a rule, the only way to maximize expected reward is to report your true belief. With the log score, for example, the reward is the logarithm of the probability given to the correct answer. Overclaiming is punished hard when the model is wrong, and underclaiming wastes reward when it is right. The model therefore learns to state probabilities it actually "believes", not just to pick the right label.
RLCD in Jev and Laya
| Jev | Laya | |
|---|---|---|
| Maker | TypeSafe AI | ConvAI Innovations |
| Trained with RLCD | Yes, per TypeSafe | Yes, per the model card |
| Training details public | No | Yes: scoring rules, exploration and update rule |
| Train your own | No; one set of weights serves every account | Yes, with the fine-tuning notebook on free Kaggle GPUs |
Laya's fine-tuning notebook runs the whole RLCD loop: build a dataset, train with proper-scoring-rule rewards, fit calibration temperatures on held-out data, evaluate and publish the checkpoint.
Does RLCD guarantee calibrated probabilities?
No. RLCD trains a model toward calibration, but how well it transfers depends on the checkpoint, the task and the number of options. In our System One benchmark:
- Jev 1.13 was the best-calibrated model tested: expected calibration error (ECE) of 0.020 on SST-2 and 0.083 on the 77-intent Banking77 set.
- Laya was even better calibrated on two-label sentiment (ECE 0.015), but badly over-confident on Banking77 (ECE 0.506).
The Laya project itself says both base checkpoints are over-confident as shipped. Refitting one temperature per question type and option count on held-out data brought its mean ECE down from 0.466 to 0.081.
In practice:
- Measure calibration on your own labelled data before you trust a threshold.
- Fit temperatures on a held-out set if the probabilities are off (see Fine-tune Laya).
- Refit after any change of checkpoint, precision or question wording.
FAQ
What does RLCD stand for?
Reinforcement Learning for Calibrated Decisions: training a model to output decisions with honest, well-calibrated probabilities instead of generated text.
Who created RLCD?
The term comes from TypeSafe AI, the company behind Jev. TypeSafe notes that its cofounder Diogo Almeida co-invented RLHF. Laya, an independent open-source project, describes its own training as RLCD and publishes the details.
Is RLCD the same as RLHF?
No. RLHF rewards responses people prefer; RLCD rewards calibrated probabilities through a strictly proper scoring rule, and the model does not generate text at all.
Can I train my own model with RLCD?
Yes, starting from Laya. Its fine-tuning notebook trains with proper-scoring-rule rewards on two free Kaggle T4 GPUs; the project reports about 4 to 5 hours for 4 epochs over roughly 30,000 questions.
Which models use RLCD?
TypeSafe Jev and the Laya checkpoints (laya, laya-multilingual, laya-typed-decisions). Other System One models such as Kev and Von document their own training recipes and do not describe them as RLCD; see the System One models comparison.
Related
- What is Jev?: TypeSafe's hosted System One model
- What is Laya?: the open-source decision model
- Fine-tune Laya: the RLCD notebook and temperature fitting
- System One benchmark: accuracy and calibration measured side by side
Last verified against TypeSafe's documentation and the Laya model card: September 26, 2026.