Ollaya: an Ollama for open source decision models

Today, to approve or reject an order with AI, a lot of people spin up a 7 billion parameter LLM, write a 40 line prompt and pray the JSON comes back valid. Ollaya shows up to tackle exactly that overkill. The pitch fits in one sentence: be the "Ollama of open source decision models". You download a model built to choose, run it locally and get a decision instead of text.

The name sounds like Ollama's cousin at Sunday lunch, and that's on purpose. The idea is to inherit the experience that made Ollama the standard: one command to download, one command to run, a local API to consume.

What Ollaya is, in practice

Ollama solved an annoying problem: running an LLM on your machine without fighting CUDA, quantization and weight formats. You type ollama run llama3 and you're done. Under the hood, it starts a server on localhost:11434 that any app can call.

Ollaya applies the same packaging to a different category of model. A decision model takes a context and returns a choice from a closed set of options. Approve or reject. Route to team A, B or C. Call tool X or call nothing. The output is structured by nature, not a paragraph you have to interpret afterwards.

The site describes these models as "Jev-style". That's the project's own jargon. Before adopting it, read the docs carefully to understand exactly what the term promises, because a nice name is no substitute for a spec.

Why decision models matter to developers

Think about how many small decisions your backend makes every day. Classifying a support ticket. Flagging a transaction as suspicious. Deciding whether a comment goes to moderation. Picking which of five internal endpoints an agent should call.

The trendy solution became "throw it at an LLM". It works, but look at what the code looks like with Ollama today:

const res = await fetch('http://localhost:11434/api/generate', {
  method: 'POST',
  body: JSON.stringify({
    model: 'llama3',
    prompt: `Classify the ticket as "billing", "bug" or "other".
Reply with JSON only: {"category": "..."}

Ticket: ${ticket.text}`,
    format: 'json',
    stream: false,
  }),
})

const { response } = await res.json()
const parsed = TicketSchema.safeParse(JSON.parse(response))

if (!parsed.success) {
  // retry, fallback, log, tears
}

Notice what's going on. You're using a text generator to produce a single word. You pay generation latency, you pay the memory of a huge model and you still have to validate with Zod because the model might invent a "finance" category that doesn't exist.

A decision model flips this around. The options are part of the contract, not the prompt. The model has no way to answer outside them.

If the answer fits in an enum, you don't need a model that writes essays.

What changes in your code and your workflow

If Ollaya delivers on its promise, the code above turns into something like this. The format below is illustrative, so check the real API in the project's docs:

type Decision = {
  choice: 'billing' | 'bug' | 'other'
  confidence: number
}

const decision: Decision = await ollaya.decide({
  model: 'ticket-router',
  input: ticket.text,
  options: ['billing', 'bug', 'other'],
})

switch (decision.choice) {
  case 'billing': return billingQueue.add(ticket)
  case 'bug': return engineeringQueue.add(ticket)
  default: return generalQueue.add(ticket)
}

Three concrete wins show up here:

• Real typing. TypeScript knows every possible output. The switch is exhaustive and the compiler complains if you forget a case.
• Confidence as a number. With a score, you can set a simple rule: below 0.7, send it to a human. With free text, that becomes guesswork.
• Predictable cost. Running locally means zero per token billing and no customer data leaving your server.

That last point matters more than it seems if you handle personal data. Sending a ticket's text with someone's ID number and address to an external API just to pick a queue is the kind of thing that gives your legal team headaches.

What are the risks of adopting it now

Here comes the honest part. A new project looks like a new project, and it's worth weighing a few points before getting excited.

The catalog is the product. Ollama got strong because it has hundreds of ready to use models. A runtime for decision models is only worth it if there are good models for your use cases. If the catalog has three generic models, you'll end up training your own, and then you have a different problem.

Benchmark or it didn't happen. The accuracy of a decision model is measured, not felt. Look for published numbers per model: accuracy, the dataset used, a confusion matrix. If there aren't any, you'll have to measure it yourself with your own data.

Each model's license. The runtime being open source doesn't guarantee the weights are free for commercial use. Read each model's license, one by one. I've seen a project go to production with an "open" model that banned commercial use in the fine print.

Automated decisions have rules. In Brazil, article 20 of the LGPD gives people the right to request a review of decisions made solely through automated processing. GDPR has a similar rule in Europe. If the model denies credit or blocks an account, you need to be able to explain why and offer human review. Store the input, model version, choice and confidence for every decision. This isn't optional.

How to test Ollaya without breaking production

My recommendation is shadow mode. You don't swap anything. You run the model in parallel with the current rule and only log when the two disagree.

const current = currentRule(ticket)

ollaya
  .decide({ model: 'ticket-router', input: ticket.text, options: CATEGORIES })
  .then((d) => {
    if (d.choice !== current) {
      log.info('divergence', { id: ticket.id, current, model: d.choice, confidence: d.confidence })
    }
  })
  .catch((err) => log.warn('ollaya failed', err))

return current

Let it run for two weeks. Then grab 100 divergences and review them by hand. If the model was right most of the times it disagreed with the rule, you have a real argument to migrate. If it was wrong, you just saved yourself an incident.

Notice that the call doesn't block the response and that a model failure doesn't take anything down. In shadow mode, the model can never be the reason for a 500 error.

One infrastructure detail: run the Ollaya server as a separate service, with a healthcheck, just like you would with Ollama. Don't load the model inside your Node API process. When it eats too much memory, you want to restart the model, not the whole backend.

Is it worth keeping an eye on?

Yes, and the reason isn't the model itself. Ollama didn't win because it had the best weights. It won because it turned "running AI locally" into something a dev does in an afternoon. A model nobody can run in five minutes, in practice, doesn't exist.

My take: we spent two years using giant LLMs as a hammer for every nail, including nails that were just an if with confidence. Small, local, typed decision models are the more sensible path for a lot of these cases. Whether Ollaya will be the tool that makes this mainstream is still impossible to say. It depends on the catalog, the community and benchmarks the project still needs to show.

For now, here's what I'd do: install it, run it on a low risk case in shadow mode and measure. If the numbers hold up, you get lower latency, zero cost per call and code that's easier to read. If they don't, you lost an afternoon and gained a dataset of divergences that's already worth gold.

If you want to see how I apply AI in real projects, without the hype, take a look at my projects.

Some devs are spinning up a 7 billion parameter LLM just to get back one word.

Classify a ticket, approve an order, pick a queue. If the answer fits in an enum, you don't need a model that writes essays.

Ollaya wants to be the "Ollama of decision models": download it, run it locally and get a typed choice with a confidence score, instead of JSON you pray comes back valid.

The real win: an exhaustive switch in TypeScript, zero cost per token and no personal data leaving your server.

But a new project calls for caution: the catalog, benchmarks, each model's license and the rules on automated decisions in data protection laws like GDPR and LGPD.

My suggestion is to test it in shadow mode for two weeks and let the numbers decide.

I wrote the full step by step on the blog. Have you ever used an LLM for something that was just an if?

#ArtificialIntelligence #TypeScript #OpenSource #DataPrivacy #SoftwareDevelopment