Move Beyond Text Generation With New AI Decision Models

Why Traditional Text Generation Falls Short

Large language models are exceptionally good at writing text, writing code, and reasoning through complex problems. However, many practical software applications do not actually require generating conversational replies. For example, a customer support tool needs to route incoming user messages to the correct department, determine issue urgency, and check for refund requests. Asking a model to generate text or format JSON can be slow and inefficient when the system only needs specific, predefined choices.

Understanding Decision Models

Decision models take an input context and evaluate predefined answers to return a probability distribution instead of generating text token by token. Initially introduced by TypeSafe AI as System One models, the developer community has converged on the term decision model to describe this computational role. This concept builds upon traditional classification and zero-shot classification methods. While older classifiers rely on rigid output categories baked directly into the model, decision models offer a flexible interface built around three main primitives.

The Three Core Primitives

The choice primitive lets the model select from a distinct set of options and outputs probabilities for each alternative, much like standard classification. The score primitive handles ordered scales by computing the expected value of the level index. Finally, the noul primitive evaluates binary questions using a Bernoulli distribution to return the probability that a statement is true. These models also provide a confidence metric to summarize how strongly the model believes in its top answer.