Jev vs LLM: We tested TypeSafe’s Jev against Opus 5.5—The results were better than we expected

The AI market produces more tools than most teams can evaluate properly. Many look promising in a demo, work for a week, and then quietly disappear from the workflow.

TypeSafe AI’s Jev looked different. After hearing Jordan Crawford, founder of Blueprint GTM, discuss using it at Clay Sculpt, I wanted to test it against a model we already trusted: Claude Opus 5.5. Crawford’s Sculpt session focused on using Jev for low-cost judgment inside go-to-market workflows, which is exactly the kind of high-volume screening problem mid-to-large enterprises and fast-scaling teams run into when building AI-powered agent systems.

We ran two internal tests using real FullStack data. Jev performed slightly better than Opus 5.5 on one classification task, completed the work faster, and cost far less—an important tradeoff when repeated workflow decisions make the computational and financial cost of frontier models add up quickly. Along the way, the comparison also exposed the limits of our own screening criteria, which turned out to be as useful as the performance results.

These are internal tests on our own data, not controlled benchmarks on public datasets. They reflect our data, prompts, definitions, and workflows. What follows looks at Jev vs. frontier LLMs across classification and screening tasks, speed, and cost. We also go over where each model fits, and how to route uncertain decisions to an LLM or a human reviewer.

The results show where a specialized decision model fits best as part of a hybrid system with large language models and humans, not as a direct replacement for them. LLMs remain better for open-ended reasoning and creative work, while Jev-style models fit repeated structured decisions.

The hidden cost in GTM engineering is judgment

FullStack runs an outbound system that ranks the companies and people our team should contact based on buying signals. Those signals might include open engineering roles, recent funding, or a LinkedIn post from a newly hired AI leader.

Most of the system is data plumbing. Scattered throughout it, however, are small judgment calls:

• Is this LinkedIn post actually about AI engineering, or does it only mention the term?
• Is this company a plausible fit for FullStack?
• Is the account showing signs that it may buy soon?

The same pattern shows up outside GTM workflows, like deciding whether a support ticket is urgent, or running intent classification on customer messages. A common example is deciding whether a support ticket is urgent enough for immediate attention is just a simple repeated classification.

We had been using Claude Opus 5.5 to make those decisions. It performed well, but it was also the slowest and most expensive part of the pipeline. An LLM generates text one token at a time, which is overkill when the workflow only needs one answer from a fixed set of possible answers and can still yield plausible-sounding but incorrect information.

Jev-style models also return a confidence score that software can use directly. That forced us to sample records rather than evaluate everything. For a handful of decisions, that cost may not matter. At thousands of repeated classification tasks per day, however, cheaper decision-making changes how the system can be designed.

What is Jev?

Jev, launched on September 15, 2026, is TypeSafe AI’s first public “System One” model, a category the company describes as models built to make fast, structured decisions that software can use directly. Unlike a general-purpose LLM, it isn’t built to produce natural language, free-form text, or dialogue.

TypeSafe trains Jev with a method it calls reinforcement learning for calibrated decisions, in place of the human feedback used to tune chat models. OpenRouter reported that Jev achieved a Brier score of 0.043 on its 88-item faithfulness evaluation, compared with 0.054 for the LLM judge. The metric measures the average squared difference between predicted probabilities and actual binary outcomes, with lower scores indicating better probabilistic predictions.

Depending on the question, Jev can return:

• The probability that a yes-or-no statement is true
• A choice from a predefined list in discrete candidate sets, similar to a multiple-choice question rather than free text
• A score on an ordered scale with levels defined by the developer

The application then decides what to do with that response. A high-confidence result might move forward automatically, while an uncertain one can be sent to a frontier LLM or human reviewer. TypeSafe’s documentation recommends combining Jev’s answers with deterministic checks in code and routing cases for action or review. Its probability distribution and confidence score can be consumed directly by software systems without parsing malformed outputs or doing the secondary parsing text-generation systems usually need for structured data.

TypeSafe lists Jev at $0.042 per million input tokens, with no charge for output tokens. In a benchmark published by MotherDuck, Jev classified 100,000 articles sampled from the AG News training set in 40 seconds for $0.50 at 89% accuracy, while GPT-5.6 Terra took 31 minutes and 59 seconds, cost $37.58, and achieved 88% accuracy.

Across the testing and production runs described below, our total model spend was approximately $1.50: much less than what I spent on the coffee I drank while reading the results.

Test one: Classification tasks with LinkedIn posts

Our first test asked a practical question: Does a LinkedIn post from a technology leader qualify as a buying signal for FullStack?

We gave the same definition and the same 135 real posts to Claude Opus 5.5 and Jev. When the models disagreed, a human reviewer selected the better answer without seeing which model had produced it, effectively creating a judge setup that compared each output against a human-selected correct answer.

The sample is too small to prove a broad accuracy edge, but the trade-off in latency and cost is still operationally meaningful. These 135 posts are a small set of labeled examples, not a broad benchmark. In this test, Jev’s empirical accuracy was slightly higher. It agreed with the human reviewer on two more posts than Opus 5.5. With only 10 disagreements between the models, that isn’t enough to establish a broader accuracy advantage. Jev did, however, cost approximately 1/180th as much and complete the run about eight times faster, with a median response latency of 171 ms.

In a benchmark published by MotherDuck, the same task costs $37.58 with a comparable LLM, while Jev costs $0.0034.

TypeSafe reports headline gains of 193.6x faster and 444.6x cheaper in its workflow evaluations, while cautioning that these figures are likely on the higher end of real-world improvements.

The error patterns were more useful than the headline result. Opus 5.5’s six errors were false positives: posts that mentioned “AI” or “engineering” without being meaningfully about either. Jev’s four errors were close to the decision boundary, with probabilities ranging from 0.41 to 0.55, and storing those scores makes calibration possible when each response can hold up a workflow that must wait on the result. Those lower-confidence misses could also become useful training data for fine-tuning a smaller model later, but adaptation is only genuinely required when baseline calibration or task structure is not good enough.

That suggests a possible hybrid workflow: let Jev handle high-confidence decisions and send uncertain cases to Opus 5.5 or a person. We haven’t tested that design yet, so it should be treated as a next step rather than a result.

Test two: Screening 33,561 companies with our own data

The second test examined whether Jev could screen a much larger list before we paid for additional research or enrichment.

We started with roughly 41,000 mid-sized North American companies from a data vendor. After preparing the available records, 33,561 companies were sent to Jev. In one screening pass, Jev could evaluate multiple questions at once. The screening questions asked whether each company appeared to build software, maintain an internal engineering team, and fit our target market without being a direct competitor.

Jev received only the company name, homepage title and description, and a small set of firmographic fields. We didn’t purchase additional enrichment for the test. This was one example of structured screening with limited firmographic and website context.

The full run took four minutes and cost $1.13. Jev returned:

• 8,461 yes
• 2,289 review
• 22,811 no

We checked the results in three ways.

Two things stood out.

First, the “yes” list was accurate enough to justify further investment. In our blind review, 37 of 40 companies classified as “yes” were genuine prospects, giving us enough confidence to focus paid signal research and enrichment on that group.

Second, the client check showed that our definition—not the model—was the problem. Jev applied the criteria we gave it, but those criteria were narrower than our actual market. That was a useful result: it showed us where the question needed to change. Because we stored the raw probabilities, we could loosen the threshold with a one-line code change rather than run the model again, which is especially useful when you want to adjust criteria without collecting new task-specific training data.

Where decision models fit

Our tests suggest that Jev is best suited to calibrated decisions over predefined options inside software systems or AI agents, whereas embedding models excel at semantic search and document retrieval across large datasets. Examples include:

• Signal classification: Determining whether a post, job opening, or news item actually matches the signal being tracked
• Initial screening: Evaluating a large list before paying for enrichment or deeper research
• Triage and routing: Sending support tickets to research, nurture, partner, or review queues as a concrete example of workflow-based decisioning
• Repeated scoring: Re-evaluating an account whenever a new signal appears
• Content and agent checks: Flagging unsupported claims or factual consistency issues in AI-generated text, or deciding whether an agent’s proposed command is high risk

These tasks share a common structure: the input may be unstructured, but the required answer is constrained by the context state. As an example of where other model types fit, embedding models can rapidly match queries to relevant documents, often through vector databases, while LLMs may offer stronger contextual understanding and logic but are typically slower for large-scale retrieval. The workflow needs a probability, label, choice, or score—not an explanation written for a person.

Where Jev doesn’t fit

Jev isn’t a replacement for a frontier LLM or other large language models: those systems are better at planning, language work such as translation and summarization, and open-ended reasoning than constrained classification, and because they can adapt to new tasks through natural language prompts and track long-range dependencies and semantic nuances, they remain useful here. Claude Opus 5.5 is designed for complex knowledge work, long-running tasks, research, and agentic workflows.

Jev is designed for typed decisions inside software, so it cannot fill free text fields or provide explanations, while LLMs can; for structured decisions, though, those outputs often require secondary parsing before software can rely on them, unlike Jev.

We would continue using a frontier LLM or human reviewer for:

• Writing emails, briefs, hooks, and other prose with a generative model
• Research tasks that need to write code or support code generation across multiple sources
• Reasoning-heavy work that requires explanation, where text generation is actually needed rather than a typed answer
• Cases near the decision threshold
• Defining the question and evaluation criteria for complex tasks

Jev can apply a narrow definition quickly and consistently, but it can’t determine whether that definition reflects the business problem you meant to solve.

Four habits we would recommend

1. Store the probabilities. Set thresholds in code so you can adjust them without rerunning the model.
2. Route uncertain cases to people. Don’t force the model to decide when the answer is unclear.
3. Validate repeatedly. Test the output thoroughly before relying on it.
4. Design with cost in mind. Jev is most useful when low-cost decisions make a workflow practical at scale.

Concluding thoughts (and shameless plug)

If judgment becomes this inexpensive, more GTM workflows can move from sampling to evaluating every record. We plan to keep testing where that approach works.

If you’re building something similar, we’d be glad to compare notes.

^{Contact FullStack to discuss your AI-powered GTM workflow.}