LLM evals: a test that can fail

Turboscore answers plain-language questions about used cars in Norway and Sweden. Most questions never reach a language model: a pattern extractor reads brand, model, fuel, budget, year and mileage from the text. Only when it finds nothing usable, or the question leans on a cultural reference, does the search ask a local model, running through Ollama on the same server, for search parameters.

The quality of that search is gated by a baseline suite: twenty queries, each with the intent it should produce (search, clarify, or advise) and the parameters it should extract. The suite is the product’s main quality metric.

What was wrong

For six weeks in mid-2026 the suite reported green while asserting nothing. The pass/fail thresholds lived in the script’s __main__ block, which the test runner never executes. Under pytest the test passed at any accuracy, and it sat at 85 % while reporting PASSED. A headline metric was decorative, and nobody noticed, because nothing about a green build invites a second look.

The fix, and its rule

The test now asserts 100 % intent accuracy and the response-time budget, and honours a known_issue field that the report had always printed but never excluded. Before the fix was accepted, an expectation was broken on purpose to see a red build. That is now the rule for every test here: a test that has never been seen to fail is not yet a test.

The first catch

With the assert in place, one query failed at once: “What car should I buy for towing a small boat?” There is nothing to filter on, because the listings carry no towing capacity, so the right answer is a clarifying question. The local model (gemma3:4b) answered anyway, with make = Ford, Chevrolet, Tesla, Jeep, Cadillac, GMC, Dodge: its US-market priors, applied to a Norwegian and Swedish market, narrowing 216 000 cars to 180 and presenting them as the answer.

It had been intermittent for infrastructure reasons, which is why no one saw it. When Ollama was cold or slow, the query timed out and fell back to a clarifying question, which was correct. It only answered wrongly when the model was warm.

Why the guard did not guard

A validator already existed whose first stated purpose was to drop brands the model invented. It accepted a brand if it appeared in the query or existed in the database. The second clause is true for every one of the roughly two hundred brands in the database, so the check never bit; only names that are not makes at all (Mustang, Camaro) were ever caught. A brand is now trusted only if it is named in the query. Being real is not grounding.

That broke one legitimate case: “What car did James Bond drive?” → Aston Martin, where recalling an unnamed brand is the whole job. The exception is a short list of cultural-reference words, and the same list decides whether to call the model at all. If the two lists drifted apart the search would either call the model and discard its answer, or trust a hallucination, so there is one list.

What is still open

Only brands are grounded. The towing question still yields fuel_type = gasoline, engine_power_min = 250, seats_min = 5 from nowhere and returns about 1 100 cars. It is recorded as a known issue in the suite, reported on every run, and does not fail the build. The suite is not loosened to make it pass, and the gap is not hidden.

Two lessons

First, a quality metric needs to be shown failing before it is trusted; the cheapest way is to break it on purpose. Second, a guard that accepts anything plausible is not a guard. The same class of error had been removed from the rule-based side in July 2026, when a “blind spot monitoring” filter turned out to rest on equipment data for 30 of 130 260 cars (0.02 %) and was deleted rather than left to return plausible-looking results. The model-based side needed the same treatment.

← all notes