How to Use Laya: Step-by-Step Tutorial for Typed Decisions

Learn how to use Laya step by step: write typed questions, read probabilities and confidence, set thresholds, batch requests and test on your own data.

Last updated: Sep 26, 2026

This tutorial shows how to use Laya from start to finish on one real task: routing customer support tickets. You will write typed questions, read the answers and probabilities, decide when to act automatically, process tickets in bulk and measure accuracy on your own data.

It assumes Laya is already installed (pip install laya, Python 3.10 or newer). If not, start with Install Laya.

Step 1: Make your first call

A Laya call takes a state (the text or JSON you want a decision about) and a set of typed questions. It returns one answer per question, with probabilities, in a single forward pass.

from laya import Router

router = Router()  # downloads a checkpoint on first use

state = "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
questions = {
    "department": {"type": "choice", "instructions": "Which department should handle this?",
                   "criteria": {"billing": "invoices, payments, refunds",
                                "technical": "bugs, outages, system errors",
                                "other": "everything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this?",
                "criteria": ["not urgent", "soon", "blocking"]},
    "churn_risk": {"type": "noul", "instructions": "Does the user threaten to cancel or leave?"},
}

result = router.predict(state, questions)
print(result["answers"]["department"]["choice"])  # billing
print(result["routing"]["model"])                 # english

The Router picks the checkpoint for you: English text goes to laya, other languages to laya-multilingual. For a server, create it once with Router(preload=True) so no request waits for a model to load.

Step 2: Write questions Laya can answer well

The wording of your questions matters more than any setting. Laya has three question types.

choice: pick one option. The keys of criteria are the labels you get back; the values are what the model reads. Describe each option in a few words instead of giving a bare label, and add an other option so unusual tickets have somewhere to go.

score: rate on an ordered scale. Give criteria as a list from lowest to highest. The answer is the expected level, so 1.8 on a three-level scale means "close to blocking".

noul: answer yes or no. The answer is the probability that the answer is yes. If you add criteria, the keys must be true and false. On the English checkpoint, a noul answer can follow the option labels instead of the text, so check it on your own examples. Upstream suggests either overriding the labels the model sees, or asking a two-option choice instead:

{"type": "choice", "instructions": "Is this review positive?",
 "criteria": {"A": "yes, the review is positive", "B": "no, the review is negative"}}

Keep a choice question under about 20 options. All options share a fixed token budget (192 tokens on laya, 256 on the other checkpoints), so long lists get squeezed until similar labels look the same.

Step 3: Read the result

Each answer is a small dictionary. For a choice question it looks like this (values are illustrative):

{
    "type": "choice",
    "choice": "billing",
    "probabilities": {"billing": 0.91, "technical": 0.03, "other": 0.06},
    "confidence": 0.62,
    "answer_confidence": 0.91,
}
  • choice, score or noul is the answer itself.
  • probabilities shows every option, which is useful for logging and debugging. A score answer also has a legend that maps level numbers to your labels.
  • answer_confidence is the probability of the answer Laya returned. This is the number to gate on. It is the quantity calibration is fitted on, and it has the same meaning for every question type.
  • confidence measures how concentrated the whole distribution is. It is not calibrated, so do not compare it against the same threshold.

Step 4: Act only when Laya is confident

The point of probabilities is that you can automate the easy cases and send the rest to a person or a larger model:

answer = result["answers"]["department"]

if answer["answer_confidence"] >= THRESHOLD:
    route_ticket(answer["choice"])
else:
    send_to_human(state, suggestion=answer["choice"])

There is no universal THRESHOLD. Upstream notes that the shipped checkpoints are over-confident and that laya-multilingual has no fitted temperatures yet, so pick the threshold from your own labelled data (Step 6). A threshold copied from Jev does not transfer either, because Jev defines confidence differently.

Step 5: Process many tickets at once

When you have a backlog, score all of it against the same questions with predict_batch. The results come back in the same order as the input:

import laya

agent = laya.load("convaiinnovations/laya")
states = [{"body": text} for text in ticket_texts]

results = agent.predict_batch(states, questions, batch_size=64)
for text, result in zip(ticket_texts, results):
    print(result["answers"]["department"]["choice"], text[:60])

Batching is a large speedup on a GPU. On a CPU it saves little, because the work grows with the number of states; see Laya on CPU for measured numbers.

Step 6: Check accuracy on your own data

Before you let Laya act on real tickets, label 50 to 200 examples from your own traffic and measure two things: overall accuracy, and accuracy on the decisions you would automate at your threshold.

labelled = [
    ("I was charged twice for my subscription", "billing"),
    ("The export button crashes the app", "technical"),
    # ... your own examples
]
THRESHOLD = 0.8
question = {"department": questions["department"]}

correct = kept = kept_correct = 0
for text, gold in labelled:
    answer = router.predict(text, question)["answers"]["department"]
    correct += answer["choice"] == gold
    if answer["answer_confidence"] >= THRESHOLD:
        kept += 1
        kept_correct += answer["choice"] == gold

print(f"accuracy: {correct / len(labelled):.2f}")
print(f"automated: {kept / len(labelled):.0%} of tickets, "
      f"accuracy on those: {kept_correct / max(kept, 1):.2f}")

Raise the threshold until the accuracy on automated decisions is one you can accept, and read the share of tickets that still get automated at that point. If the numbers are not good enough, reword the options (Step 2) or fine-tune Laya on your labelled data.

Common mistakes when using Laya

  • Too many options in one question. Above about 20, raise head_max_len, or narrow the candidates first with predict_shortlist.
  • Long text that gets cut off. The English checkpoint reads 512 tokens by default. See the context window for max_len and predict_long.
  • Sending non-English text to the English checkpoint. Use the Router, or load laya-multilingual directly.
  • Trusting score on the multilingual checkpoint without checking. Upstream reports a position bias there: it rarely picks the first level.
  • Gating on confidence instead of answer_confidence.

FAQ

Is Laya a chatbot or an LLM?

No. Laya answers typed questions about text you give it and returns probabilities. It does not write text, so there is nothing to parse and no output to hallucinate. For open-ended answers, use an LLM; for classification and routing, see Laya vs LLMs.

Do I need a GPU to use Laya?

No. Laya runs on a CPU, an NVIDIA GPU or Apple Silicon. A GPU is faster, especially for batches. See Run Laya locally.

How do I use Laya without Python?

Three ways: the laya command line (laya "My payment failed twice" --preset triage), the Node.js runtime for TypeScript apps, or an HTTP server that any language can call (Self-host Laya).

Can I try Laya without installing anything?

Yes. The Laya Playground runs typed questions in your browser, and each recipe comes with a ready-made example you can load there.

Next steps

Last verified against the upstream repository: September 26, 2026.