Start with what an AI model is
An AI model is a file. A large one — tens of gigabytes — but a file, like a video or a database. To use it, a computer loads the file into memory and does an enormous amount of arithmetic every time you ask it something. That arithmetic is the reason AI has historically lived in data centres: it used to need equipment nobody could put under a desk.
That changed. Modern desktop chips — Apple's M-series, AMD's Ryzen AI range — put a large pool of fast memory right next to the processor, which is exactly what these models need. The open models released in the last two years are also much better per gigabyte than their predecessors. Put those together and a machine the size of a lunchbox now runs a model that would have needed a rack in 2022.
So what's in the box?
Four things, and none of them are exotic:
- A language model — the part that reads and writes. It answers questions, drafts letters, summarises documents, and follows instructions about your house style.
- A speech model — usually Whisper. It turns recorded speech into accurate text, including Australian accents and the medical or legal words that trip up phone dictation.
- A search index of your documents — your files, converted into a form the assistant can look things up in. Ask a question, it finds the relevant passages, and answers using them with the source shown.
- A web interface — what your staff actually see. It looks like any chat tool. They open it in a browser at an address on your network; nothing is installed on their computers.
The machine keeps some memory spare so it doesn't slow down when three people use it at once, and reserves a few gigabytes for its own operating system. We publish the exact memory budget for each machine, because "it's fast" is a claim and a memory table is a fact.
Why it doesn't need the internet
People find this the hardest part to believe, so it's worth being blunt: once the model file is on the machine, the internet plays no part in answering a question. The model isn't looking anything up. Everything it knows is in the file, and everything about your business is in your documents on the same machine.
We connect the machine to your network so staff can reach it. We do not need to connect it to the outside world at all, and in several installs we haven't. Where clients want us to provide remote support, that runs over an encrypted private link they can switch off.
What it's good at
Recorded speech into formatted text. First drafts of letters, notes and reports in your templates. Finding things across files that nobody has organised in fifteen years. Reading a long document and answering questions about it. Repetitive extraction work — invoices, forms, timesheets — where the same fields come out every time.
What it's not good at
It is not a decision-maker and shouldn't be sold as one. It will occasionally produce a confident sentence that's wrong, which is why every workflow we build has a person checking output that matters. It doesn't know anything about the world after its training date unless the answer is in your own documents. And it is not magic on bad data: if the file share is a mess, search results will reflect the mess, though it's still better at finding things in a mess than a person is.
One more limit worth naming. A machine in your office runs a model sized to that machine, and the very largest models in the world are bigger than any office box. In practice this matters far less than it sounds — the gap between a good local model and a frontier cloud model has narrowed to the point where most of our clients can't tell on the work they do. But if you need the absolute frontier for research-grade reasoning, an office machine isn't that.
Is "server" even the right word?
Barely. It suggests a rack, a cooling problem and a specialist. In practice it's a small silent box near your router, drawing about as much power as a light bulb, that everyone in the office shares. We call it a machine, or an appliance. The industry calls it on-premise or edge AI. It's the same thing.
If you want to see one working on your own documents, book a 20-minute chat.