Turn a Vision LLM into a Jev-Like Decision Engine

A tiny output changes the whole job

Picture a webcam aimed at a doorway. A normal assistant might reply with a paragraph about the room, but an automation loop usually wants something smaller: indoors, outdoors, or unclear; true or false; perhaps a brightness score from 0 to 2. A common search query is: how do you make a normal large language model (LLM) return a decision instead of a paragraph?

Force the next output to be one of a few labels, then read the model's probability for each label. That is the heart of a Jev-like wrapper. A vision-language model (VLM), meaning an LLM that accepts images as well as text, fits the same pattern because the image changes what the model sees, not the shape of the answer.

Jev's public interface describes three useful question shapes: Choice, Score, and Noul, a yes/no-style value. A wrapper around a general model can provide similar shapes without pretending to reproduce Jev's training or calibration. (docs.typesafe.ai)

Read the next token, not a paragraph

A token is a small piece of text from the model's vocabulary. It might be a whole letter, a word fragment, or a word with a leading space. A log probability is the natural logarithm of the model's probability for that token; lower values mean the token was less favored.

The prompt can make the candidate tokens explicit:

State:
My order arrived broken and I want a refund.

Question: Which team should handle this?
[A] billing
[B] shipping
[C] returns

Answer with one letter only.

Ask for at most one generated token and request log probabilities. The model may return A as the visible answer, but the more valuable part is the list of alternatives at that same position. Convert each returned log probability l with exp(l - max_l) and normalize the weights. Subtracting the maximum keeps the exponentials numerically stable; the final probabilities still sum to one over the supplied options.

The current OpenAI Chat Completions reference exposes logprobs and top_logprobs, with top_logprobs documented from 0 through 20. That limit is one reason short, single-token labels such as A through T are friendlier than arbitrary option names. (platform.openai.com)

One function, three decision types

Here is the core of the wrapper. It uses an OpenAI-compatible chat endpoint, accepts an optional image URL or base64 data URL, and builds the typed result itself instead of asking the model to generate JSON.

import math
import string
import requests

def decide(base_url, model, state, questions, image_url=None):
 answers = {}

 for name, question in questions.items:
 kind = question['type']

 if kind == 'choice':
 options = list(question['criteria'].items)
 elif kind == 'score':
 options = [(str(i), text)
 for i, text in enumerate(question['criteria'])]
 elif kind == 'noul':
 options = [('true', None), ('false', None)]
 else:
 raise ValueError(f'Unknown question type: {kind}')

 if not 2 <= len(options) <= 20:
 raise ValueError('Use between 2 and 20 options')

 labels = string.ascii_uppercase[:len(options)]
 prefix = f'State:\n{state}\n'
 instruction = question['instructions']
 suffix = [f'Question: {instruction}', 'Options:']

 for label, (key, description) in zip(labels, options):
 line = f'[{label}] {key}'
 if description is not None:
 line += f': {description}'
 suffix.append(line)

 suffix.append('Answer with one letter only.')

 content = [{'type': 'text', 'text': prefix}]
 if image_url:
 content.append({
 'type': 'image_url',
 'image_url': {'url': image_url},
 })
 content.append({'type': 'text', 'text': '\\n'.join(suffix)})

 body = {
 'model': model,
 'messages': [{'role': 'user', 'content': content}],
 'max_completion_tokens': 1,
 'temperature': 0,
 'top_p': 1,
 'logprobs': True,
 'top_logprobs': 20,
 }

 response = requests.post(
 base_url.rstrip('/') + '/chat/completions',
 json=body,
 timeout=60,
 )
 response.raise_for_status
 result = response.json

 candidates = (
 result['choices'][0]['logprobs']['content'][0]['top_logprobs']
 )

 scores = {}
 for item in candidates:
 label = item['token'].strip.strip('[]')
 if label in labels:
 scores[label] = item['logprob']

 if set(scores)!= set(labels):
 raise ValueError('The endpoint did not return every option label')

 peak = max(scores.values)
 weights = {
 label: math.exp(value - peak)
 for label, value in scores.items
 }
 total = sum(weights.values)
 probabilities = {
 key: weights[label] / total
 for label, (key, _) in zip(labels, options)
 }

 if kind == 'choice':
 answers[name] = {
 'choice': max(probabilities, key=probabilities.get),
 'probabilities': probabilities,
 }
 elif kind == 'noul':
 answers[name] = {'noul': probabilities['true']}
 else:
 answers[name] = {
 'score': sum(
 int(key) * probability
 for key, probability in probabilities.items
 ),
 'probabilities': probabilities,
 }

 return answers

The function treats noul as true versus false, returns the most likely key for choice, and calculates a weighted average for score. Notice the deliberate failure when one candidate label is missing from the returned alternatives. Quietly assigning a missing label probability of zero can make a truncated response look more certain than it really is.

This is also why the label alphabet matters. If the tokenizer splits a candidate into several tokens, the first-token probability is not the probability of the whole word. Multi-token scoring is possible by multiplying conditional probabilities token by token, but letters keep the first version understandable. For reasoning models, use a direct-answer mode where supported; some APIs count hidden reasoning inside the completion-token budget. (platform.openai.com)

Images fit the same contract

For a webcam, encode each JPEG as a data URL such as data:image/jpeg;base64,... and pass it as image_url. A local llama-server can load a text model plus a multimodal projector with a command like:

llama-server -m model.gguf --mmproj mmproj.gguf --port 8080

Its documentation describes multimodal input through the OpenAI-compatible chat endpoint, including image URLs and base64 data. Hosted APIs may use a different request envelope—for example, the Responses API represents an image as input_image and exposes output log probabilities through an include setting—but the scoring loop remains the same: identify candidate tokens, exponentiate, and normalize. (github.com)

The expensive part is often before generation

One output token is cheap to decode, but the state and image still have to pass through the model. For a live camera, sending every captured frame into a growing queue is a good way to analyze yesterday's scene. A better loop keeps only the newest frame, submits one evaluation at a time, and drops stale work.

Put the stable state and image before the question, as the function does above. A key-value cache, usually called a KV cache, stores intermediate attention data from an already processed prefix. Local llama.cpp servers can reuse a common prompt prefix with prompt caching; when several questions share the same state and frame, a backend may avoid repeating part of the work. Hosted services may or may not expose equivalent reuse.

Treat the numbers as evidence, not truth

The wrapper's probabilities are normalized model preferences over your candidate set. They are not automatically calibrated probabilities of correctness. A value of 0.92 means the model strongly preferred one label in this prompt; it does not guarantee that 92 out of 100 similar decisions will be right.

Before using the result to open a door, reject a payment, or route a customer, test it on examples with known answers. Shuffle the option order and measure how often the decision changes. Add an unclear option for genuinely ambiguous frames, and make the application abstain when the top two choices are too close. If the numbers need to mean something operational, calibrate them against held-out data rather than trusting the raw normalization.

There is a second subtlety with images: the model may confidently infer details that are not visible. Tell it to judge only observable evidence, keep questions narrow, and separate independent checks. Is a person visible?, Is the scene indoors?, and How bright is it? are easier to inspect and debug than one broad prompt asking for an overall interpretation.

A Jev-like wrapper is not a new model architecture hidden inside a clever function. It is a disciplined interface: constrain the answer space, read the next-token distribution, and turn it into a data structure your program can use. Add a vision input and the same idea reaches webcam frames, screenshots, and photographs without giving up the flexibility of natural-language criteria.