Field technicians use a Windows laptop app to summarize their repair notes. Many work in plants with no network access for whole shifts, and the summaries must be produced on the spot. The laptops have modest GPUs. The company also wants to avoid per-token cloud charges for this feature. What should the developer use?
Choose one.
Where a model runs is part of model choice: offline or on-device requirements point to a small language model run locally rather than any cloud deployment type.
Two constraints decide: the feature must work with no network, and it should not incur per-token cloud charges. Only a model running on the laptop satisfies both. Small language models are sized for modest local hardware, and Foundry Local serves them through an OpenAI-compatible local endpoint. Every cloud option, however cheap or well provisioned, still needs connectivity.
- Spot the deciding constraint: no network access during the task.
- Rule out every option that calls a cloud endpoint, regardless of deployment type.
- Choose a small model that fits local hardware and runs on the device.
Exam tip: Offline or on-device requirement: run a small language model locally, not a cloud deployment.
AI-103: Choose Foundry Models, Retrieval Methods and Agent Tools — the lesson that teaches this.