Technical debate
Local AI vs API AI: not an ideology choice.
You pick the architecture with the lowest total cost able to meet the required quality, privacy and availability. This is the real map of when a local model adds up and when an API model is the more efficient path.
Six criteria
Criterion-by-criterion comparison, no strawman on either side
| Criterion | Local AI (SLM) | API AI |
|---|---|---|
| Marginal cost per call | Apparently zero, but CAPEX and dedicated staff are paid regardless. | Predictable per token. Lightweight models cost cents per thousand calls. |
| Latency | Very low on the same network. Great when the user is right there. | Good on lightweight models, medium on frontier. Depends on endpoint region. |
| Privacy and compliance | Data does not leave your network. Hard requirement in health, defense, government. | Major providers offer no-retention contracts and EU regions. |
| Quality on complex tasks | Limited. Models that fit on a reasonable server do not rival frontier. | State of the art on demand. No infrastructure. |
| Maintenance and staffing | Requires MLOps, model updates, monitoring, hardware, backups. | The vendor maintains the model. You maintain your orchestration. |
| Availability during network outages | Works without internet. Real edge on shop floor, field, offline. | Depends on the link. Cache and deterministic fallback are mandatory. |
Decision rule
When privacy is a hard requirement, when volume is massive and stable, or when you must operate offline: local AI usually pays off. Otherwise, an API model on a neutral architecture is cheaper to run and faster to evolve.
In SVP the model gateway lets you switch providers without rewriting the product
The local vs API debate is rarely a single answer. Often the answer is both, at different levels of the Semantic Scale. Architecture decides, not the model.