Anyone using AI usually sends their data to a cloud service. It does not have to be that way. Open language models have become capable enough to handle many tasks on your own hardware: in the server room, on a workstation or right inside a device. For companies with sensitive data this is becoming a real alternative.
Why local models are interesting now
Two developments come together. Freely available models have improved markedly and cover tasks such as summarising, classifying, extracting data or answering questions from your own documents well. At the same time the hardware required has become smaller and cheaper. A model that once needed a data centre now runs on a single server with a graphics card.
What speaks for running it in-house
Data sovereignty. Customer data, contracts, design documents or patient data do not leave the company. That simplifies compliance with Swiss FADP and GDPR and answers many questions before the legal department asks them.
Predictable cost. Instead of paying per request, you pay for hardware, power and operations. With a high, steady volume that pays off. With a few requests a day, the cloud remains cheaper.
Independence. The model does not change overnight, there is no dependence on a provider’s pricing and product decisions, and the solution works without an internet connection.
Where the limits are
The largest cloud models remain superior for demanding tasks, such as long multi-step analyses or complex software development. Running locally also means responsibility: procuring hardware, updating models, securing access and monitoring the quality of the answers.
In practice a hybrid is therefore often the sensible choice. Sensitive data and routine tasks stay local; for particularly hard tasks without delicate data, a cloud model is added.
AI right inside the device
Edge AI goes one step further: the model runs not in the server room but on the electronics inside the device itself. A camera recognises objects, a machine reports deviations or a device understands speech without data leaving the device. Here the interplay of hardware, firmware and model determines response time, power consumption and unit cost.