The AI industry has spent three years in an arms race measured in billions of parameters, and somewhere along the way everyone agreed that the answer to every problem was a bigger model.
Bigger clusters, bigger bills, bigger teams to manage the complexity. The implicit assumption is that general intelligence at scale is the end state and everything else is just waiting to catch up.
Which is obviously great for the foundational teams pushing this agenda, were it not for the fact that the overwhelming majority of problems we throw AI at don’t need otherworldly intelligence and infinite compute to solve.
Llama, Phi, Qwen, Mistral: models that would have been considered frontier-grade two years ago are now freely available, fine-tuneable, and deployable on hardware you control. The performance gap that once justified the cost and lock-in of proprietary giants is now much slimmer, so the question is no longer “which frontier model should we use?” but rather “do we need a frontier model at all?”
For most tasks, the honest answer is no.
The Flamethrower Problem
Using a frontier model on every task is like using a flamethrower to light a candle. Technically works, but your eyebrows and your budget are gone.
The majority of subtasks in a deployed agentic system are repetitive, scoped, and non-conversational. Extract this field, classify this ticket, format this output, none of these require a model trained on the sum of human knowledge. They require a model that is fast, predictable, cheap to run, and tuned to behave exactly as expected in a well-defined context.
In fact, on well-defined, repetitive tasks, specialised SLMs do not just cost a fraction of what frontier models cost, they outperform them. A model built specifically for one thing, trained on real examples of that thing, is better at that thing than a model trained to do everything.
And this holds even as the tasks get harder. Qwen2.5-Math-7B, running Microsoft’s rStar-Math reasoning framework, scores 90% on the MATH benchmark, beating OpenAI’s o1-preview. John Snow Labs’ MedS, an 8B medical model, was preferred over GPT-4o by practising physicians in blind evaluation across every dimension: factuality, clinical relevance, and conciseness. LlaSMol, a 7B chemistry model, achieves 93% exact match on molecular prediction tasks where GPT-4 scores under 5%. On Berkeley’s function-calling leaderboard, the top spot is held by an 8B model beating GPT-4o and Claude 3.5 Sonnet.
What open weights actually unlocks
The capability trajectory of small open models has been striking, yet the capability of these models is not the most exciting piece, what open weights actually unlock is control.
A frontier model you do not own, cannot audit, and cannot modify is a dependency, not infrastructure and certainly not your ‘technical moat’. When it changes, your system changes. When it halts, your system halts. When the provider decides to deprecate a version or reprice the API, you absorb it. The case for specialised small models is not just that they are better. It is that they are yours.
The physical dimension matters too. Frontier models run in data centres, behind APIs, with latency measured in hundreds of milliseconds and costs that compound with every token. In an agentic system where a single workflow might invoke a model thirty or forty times, that compounds fast. Serving a 7B SLM is roughly 10 to 30x cheaper in latency, energy, and compute than serving a 70 to 175B LLM. Small models can run on-premise, inside the security perimeter, on the device generating the data. A fraud detection model running locally, fine-tuned on that institution’s own transaction patterns, makes faster decisions with no external dependency and no data crossing a network boundary. The performance is better, the risk profile and compliance conversation are entirely different.
Even more exciting still (if you can believe it) is that a model small enough to run on a phone can be fine-tuned not just to a specific workflow but to a specific person. An agent that lives on your device and learns from how you actually work is a categorically different thing to one calling to a data centre every time you ask it something. We are moving from general intelligence in the cloud to personal intelligence in your pocket. That shift is bigger than most people are currently treating it.
Production is the best training data you have
Every invocation of a model in an agentic workflow is a natural source of high-quality training data. The prompts are well-defined, the expected outputs are narrow, and whether the workflow succeeded or failed is a clean signal. A listener at the model call interface, logging inputs, outputs, and downstream outcomes, accumulates exactly the dataset you need to continuously fine-tune a specialist model for that task, as the system runs. The production environment is generating its own improvement data in real time.
This is where the SLM case becomes compelling beyond economics. You instrument the calls, cluster the patterns, fine-tune the specialists, and the system gets better with every run. What determines whether you actually close that loop is whether you have the infrastructure to capture it, label it, and act on it.
AI security has become a pretty bloated term, so stick with me here. The version of it that actually matters is not about firewalls or prompt injection. It is about knowing what your model is doing, and knowing when it is doing something it should not. Specialised models make that question answerable. When a model has a narrow, well-defined scope, you know what it should be doing, which means you know when it is doing something else. The output distribution is tighter, the failure modes are more predictable, and the feedback loop from production behaviour back to model improvement is shorter and more reliable.
General models are a different problem entirely. The failure surface is vast, constantly shifting, and practically impossible to enumerate in advance. Guardrails are the industry’s current answer, which is to say: a list of rules written by humans trying to statically account for an infinite and ever-changing set of ways a frontier model can go wrong.
The honest counter-argument
Now I would love to just dunk on frontier models for the rest of this post, but there is a real counter-argument worth addressing. Calling an API is easy. You write a prompt, you get an answer, you iterate, you ship. The feedback loop is fast and the overhead is minimal. Building SLM-first infrastructure is a different proposition entirely.
Fine-tuning a specialist model requires training data, which requires instrumentation, which requires someone to own it. You need pipelines to capture production traces, processes to curate and label them, and infrastructure to run fine-tuning jobs on a cadence that actually moves the needle. You need model versioning, evaluation harnesses, and a deployment setup that lets you swap models without taking down the system. None of this is impossibly hard, but none of it is free either.
Why this matters to us
Everything in this post is, admittedly, not a neutral observation. We built Overmind on a specific bet: that the teams who win the next phase of agentic AI will be the ones who own their models, not the ones who rent the most powerful ones.
The infrastructure to do that, fine-tuning pipelines, production observability, continuous improvement loops that do not require a dedicated ML team to run, that is what we are building. Not just because it is an interesting technical problem, but because without it the agentic future we are all so excited about, the one where AI runs diagnostics in hospitals or runs locally on your phone and adapts to you as an individual, simply cannot happen.
So we are building that future.
Overmind builds real-time supervision and fine-tuning infrastructure for autonomous AI systems. If you are deploying agentic AI, get in touch