GTM Lead Scoring with a System One Model

Every go-to-market (GTM) team faces the same problem: a list of accounts and no idea which ones to contact first. You know who your ideal customer is, but turning that into a ranked list of people to reach out to, without reading every profile by hand, is the hard part. This case study walks through a system that does exactly that.

The system combines a business data API with Jev, TypeSafe's first System One model, a model that makes fast structured decisions instead of writing text. It turns a title search into a sorted list of buyers, with the pipeline, the questions, the numbers before and after each iteration, and the honest limits we hit, laid out end to end.

The task is a decision, not a sentence. For every candidate we want the same five answers: is this a buyer, is the industry right, does the company build software, is it a competitor, what use case is this. A chat model asked for those five answers will give you five sentences, and then you parse them, and sometimes it hallucinates a field or changes the shape, and a retry loop is born. The cost and latency of that loop, repeated across every candidate, is the thing we wanted to remove.

A System One model does the opposite. It never generates prose. You send one state and a set of typed questions, and it returns a probability for each, in parallel, in a single call. The output is a number, which means the code around it is a threshold and a sort, not a parser.

A System One model is not a smaller LLM. It is trained with reinforcement learning for calibrated decisions, not for human-pleasing text, which is the difference between a model that says what sounds right and a model that says how sure it is.

There are three primitives. A noul question returns the probability that a statement is true, from zero to one. A score question rates the state across ordered levels and returns a probability-weighted position. A choice question picks one option from a set you define and returns the full distribution. Every answer carries a confidence value derived from how concentrated that distribution is.

Three steps. First, we query our own public search API for candidates, people whose title and seniority match the buyer roles we care about. Second, we send each candidate to Jev as typed questions against a small state, the person's title, company, industry, size and headline. Third, we combine Jev's answers with deterministic signals in code and rank the results.

The state is deliberately tiny. Here is an entire request, everything the model is allowed to see about one candidate:

The heart of the work was deciding what to ask. The scoring quality is entirely determined by the questions, because the model returns exactly what you ask for and nothing more. The questions have to be atomic: one narrow judgment each. A question that mixes three factors cannot be corrected by the others, because every question is evaluated in isolation against the same state.

We settled on four judgments that feed the score, plus a fifth kept only for segmentation:

The system is only as good as the profile it targets. Before writing a single question, we wrote down who we were looking for. The buyers are technical leads and go-to-market (GTM) leads: CTO, Founder, VP of Engineering, Director of Data Engineering, Head of Data, and on the revenue side VP of Sales, VP of Marketing, Head of RevOps and Head of Sales.

The industries are Software Development, IT Services and IT Consulting, Staffing and Recruiting, Marketing and Advertising, Sales Tech, and Venture Capital and Private Equity. The company size is eleven to five hundred employees, which is where a self-serve data API is bought without a procurement process. That profile, written down before any code, is what the questions below encode.

One dimension we kept out of the model entirely. Company size is a fact our API already returns as a staff range, and the sweet spot of eleven to five hundred employees is a deterministic rule, not a judgment. Asking the model to judge what a string comparison can compute is waste, and worse, it adds noise:

The principle generalizes. Known rules, calculations and exact lookups belong in code. Semantic understanding, the part where you need a judgment about language, belongs in the model. Keeping those two apart is what made the whole pipeline reliable.

Each noul question returns a probability from zero to one. We combine the signals with weights chosen by how strongly each predicts a real customer. The competitor signal is inverted, because being a competitor is a penalty, not a plus:

That is the system as it stands. The first version did not look like this. It asked only two things: is this person a technical decision-maker, and is the company a good fit? The result was technically correct and practically useless.

The top of the list was full of banks and insurers. A global head of data at a large financial institution is, by any reasonable reading, a technical decision-maker. But that person buys through procurement, on an annual contract, and would never touch a self-serve API priced at $499 a month. The model had no way to know that, because we never asked it.

The combined scores also landed in a narrow band, roughly 0.57 to 0.81 with an average around two thirds. A narrow band is a bad sign for a ranking: it means the best lead and the worst lead are barely distinguishable, which defeats the point of sorting.

The model was not wrong. It answered the questions we asked. The questions were wrong. A technical buyer is not the same thing as our customer. Our customer is a technical or go-to-market buyer at a product-led software company, in a target industry, of the right size, who is unhappy with per-credit vendors.

That sentence has four clauses, and each clause had to become its own question. The first version had collapsed all of it into two vague questions, and the result was a list of confident, irrelevant answers. This is the single most important lesson of the whole project: garbage questions produce garbage rankings, and the model will not save you from them.

Three changes. We broadened the competitor question from naming a few vendors to asking whether the company sells or resells any kind of B2B data as its own product. We sharpened the build question to separate software products from consulting and agency services. And we dropped a question entirely.

The dropped question asked whether the person's company needed our kind of data. It was noisy in both directions. It scored a commercial risk-intelligence provider as needing our data, when that provider harvests its own corporate records, and it scored obvious customers as not needing it. A model cannot reliably infer buy versus build from a title, a company name and an industry string. We removed the question and let the build and industry signals carry that weight instead.

The first run had also exposed a subtle failure in our own thinking. We assumed any technical decision-maker was a lead. The data showed otherwise: a technical decision-maker at a data provider is a competitor, and a technical decision-maker at a bank is unreachable. The competitor and size signals are what corrected that assumption.

The results moved in exactly the direction the fixes predicted. The data provider that had ranked near the top fell out of the top fifty, and the go-to-market consulting agency fell from near the top to the middle. The top of the list became what it should have been from the start: heads of engineering and data at product companies, and vice presidents of sales and revenue operations.

The score band widened to roughly 0.70 to 0.92, and the average rose. More distance between the top and the bottom, which is what a ranking is for. The accuracy of the top results went from roughly two thirds to roughly nine in ten, judged by hand-checking the top of the list against what each company actually does.

What remains is a small residue of borderline cases, companies that both consume and collect data, that no simple rule settles. We accept those as the cost of a fast, cheap first pass, and they are the ones we would read by hand before sending a message.

A ranking is only useful if the scores separate the good from the bad. A narrow band means the top and the bottom are interchangeable; a wide band means the model is making real distinctions. The iteration widened the band noticeably, which is the clearest single signal that the questions got better.

Speed and cost first. A request answers in tens of milliseconds to half a second, at a few cents per million input tokens with output effectively free. Because every question in a request is evaluated in parallel and in isolation, asking twenty questions about one candidate costs barely more than asking one. A chat model answering the same questions costs orders of magnitude more and takes orders of magnitude longer, and you still have to parse its answer.

Typed output second. Probabilities and a confidence value, not a paragraph. There is no JSON parsing, no retry loop waiting for the model to emit valid JSON, no hallucinated field. The answer is a number you branch on, which means the surrounding code can threshold it, sort it, or route it without reading prose.

Calibration third. Because the model is trained to report uncertainty honestly, a low-confidence answer is a real signal. In our first title-scoring test, a borderline title came back with a confidence around a quarter, and the model split its probability almost evenly between two levels. That is the model telling you to look yourself, which is exactly the behavior you want from a machine you are about to trust with a lead list.

The model cannot reason over evidence it does not have. The buy versus build distinction failed because the state only carried a title and an industry string, and that is not enough to know whether a company buys data or harvests it. No amount of clever prompting fixes a missing input.

The correction is architectural, not prompt-level. Where a fact is deterministic, compute it in code, as we did with company size. Where the model needs richer evidence, fetch it first and put it in the state, for example a company description or a recent job posting. The model judges what you give it; it does not go looking.

Candidates scored on four model judgments plus a deterministic size signal, ranked into a working list. The pipeline is a few hundred lines of Python and a single HTTP call to the model per batch. The whole thing runs from a button in a browser, streams its progress, and renders a sortable table with a profile link per lead.

It is a tool, not a product, and that is the point. The intelligence is one step in a workflow the code owns. The model decides, the code thresholds, sorts and routes, and a human reads the shortlist. No single piece is doing more than it should.

For this exact job, the choice is not a matter of preference. It is whether you need a decision or a sentence. Here is the comparison, measured on the same task:

The biggest mistake is treating a System One model as a replacement for a chat model. It is a complement. The two are good at different steps, and the best results come from putting them in the same pipeline and letting each do the job the other is bad at.

A System One model is the fast, repeated decision in the hot path: score every candidate, route every message, flag every anomaly. An LLM is the slow, one-off reasoning around it: read a company's website, fetch a job posting, write the outreach message, decide what evidence to gather next.

In our pipeline the split was clean. The model scored, the code ranked, and a human wrote the message. Scaled up, the natural next step is an LLM that fetches richer evidence for the borderline cases, feeds it back into the state, and lets the System One model re-score. Two models in a loop, each covering the other's weakness.

Write the questions first, and write them atomically. Keep deterministic logic in code. Iterate on the questions, not on the model. Treat confidence as a signal to review, not a verdict. And validate against a handful of real examples before trusting a ranked list.

A System One model is a good tool for this exact job: a structured judgment that runs in volume, in production, where hand-written rules are too brittle and a chat model is too slow and expensive. It is not magic. It is a very fast, very cheap way to ask the same narrow question again and again and get a probability back each time. The quality of the answer is yours to control, and it lives in the questions.