What a model swap exposed
Faults in four places around Turboscore's local model, from the times it was added, changed or moved. None is a verdict on the model.
Turboscore’s search asks a local model, through Ollama, only when its pattern extractor cannot read a question on its own. That model has been changed and moved more than once. None of the moves went from a hosted API to a local one; every one was Ollama to Ollama, a different model or a different machine. Each time the model started running, changed, or moved, something turned up: in our prompt, our code, our configuration, or a job that did not do what its comment said.
The figures are from the commit messages of the fixes, written at the time. They have not been re-run for this note.
1. A few-shot example became the answer
In January 2026 the model was added to search-parameter extraction for questions like “a petrol V8 like Elvis”. It
first shipped naming llama3.2, a model that was not installed; that was fixed three days later. The
prompt showed one worked example with concrete values: a Cadillac, year_max 1980. For questions that matched no
cultural reference, the model copied the example. “Reliable car” returned 0 cars; after the example was changed to
placeholders, 7 319. “Recommend a car”: 0, then 20 505. “Fast car”: 8, then 970.
The same week, the model added fuel_type = gasoline to 8 of 8 neutral queries, steering vague searches away from
electric and hybrid cars; an explicit rule took that to 0 of 8. And the code that pulled JSON out of the reply used a
pattern that cannot match a nested object, and a reply it could not parse failed silently.
The prompt fixes stayed, and the model is now asked less: when the pattern extractor already has a body type or a time constraint and the question has no cultural reference, the model is not asked at all. Values in a prompt’s example are, to the model, a possible answer.
2. A swap aimed at one feature changed another
In February the search’s model call was moved onto the shared Ollama client. The request names no model, so it takes whatever the configuration says. In July two commits on the same evening, a search fix that also changed the deployment template and a one-time migration of the production settings, moved the primary model from llama3.1:8b on a development machine to gemma3:4b on the production server. The reason both give is latency in a different feature, the advice answers, which had timed out while the development machine was busy.
Neither commit says that search’s model call reads the same setting. Through it, search extraction changed model too. It still takes whatever the configuration names. A model is a dependency; a configuration line that names one changes every caller that does not name its own.
3. The old model’s tolerance hid a language-detection bug
The same day brought a fix for Norwegian questions answered in Swedish or English. The language detector matched the Swedish vilken inside the Norwegian hvilken, and did not know the Norwegian hva at all, so the prompt told the model to answer in the wrong language. The fix commit records that llama3.1:8b had masked this by answering in the language of the question regardless; gemma3:4b followed the instruction it was given.
A more obedient model is a test of the instructions. The bug was in the detector, and it had been there before the swap.
4. Keep-warm that did not keep warm
On the production server the model unloaded when idle, and the next question paid a cold load of about 15 seconds,
timed out, and fell back to the other machine. A cron job was added to ping the model every 30 minutes. Its comment
said the model stayed loaded for 60 minutes. The request sent no keep_alive, so Ollama applied its default of five.
By that arithmetic, if no question came in between, the model was cold for 25 of every 30 minutes, and the first question after a quiet spell paid
exactly the delay the job existed to prevent. Ten days later, a ping with keep_alive set moved the model’s expiry from five minutes to sixty,
checked on the server.
The failure path had its own fault, found the same day as the swap: when the call to Ollama failed, the user was shown an English apology with the raw exception text appended. It now returns a short busy message in the user’s language.
Slow and cold paths hide things in both directions. A timeout that falls back to a safe answer also hid a model that answered wrongly when warm; that case is written up in A test that can fail.
What this note does not claim
It does not compare the two models. There is no current benchmark of llama3.1:8b against gemma3:4b on today’s search pipeline, so no claim here is about which one is better or faster. The point is narrower: a swap changes behaviour in places no one is looking, so the checks belong on every caller of the model, not only on the feature the swap was for.