®Not every AI feature needs a call to a server. On-device machine learning runs a model directly on the user's phone, so the app can recognise images, understand text or process sound without sending data away. For the right features, this makes an app quicker, more private and able to work without a connection.
What on-device machine learning means
A machine learning model is a file that has learned patterns from data. In a cloud setup, the app sends input to a server, the server runs the model and sends back a result. On the device, the model ships with the app or downloads once, and the phone's own processor does the work.
Modern phones include hardware designed to speed up this kind of work, often called a neural engine or neural processing unit. This makes many tasks practical that would once have drained the battery or felt slow.
Common examples include:
- Camera features such as scanning documents, reading text, detecting faces or recognising objects
- Barcode and QR scanning that works instantly in a warehouse or shop
- Speech and sound such as wake words, voice commands or noise detection
- Text features like smart replies, language detection or translation
- Health and fitness tracking from motion sensors
- Personalisation that adapts to the user without sharing their habits
Cloud or device: how to choose
The decision comes down to a handful of trade-offs. Neither option is better in every case.
| Factor | On the device | In the cloud |
|---|---|---|
| Speed | No network round trip, often feels instant | Depends on connection quality |
| Privacy | Data can stay on the phone | Data leaves the device |
| Offline use | Works without signal | Needs a connection |
| Model size and power | Limited by phone memory and battery | Can use very large models |
| Running cost | No per-request server cost | Usage-based fees or server bills |
| Updating the model | Needs an app update or model download | Change it on the server at any time |
On-device is a strong fit when the task is narrow, needs to be fast, involves personal data or must work in places with poor coverage. The cloud is better for large language models, tasks that need up-to-date knowledge, and features where the model changes often.
Many good apps mix both. A camera app might detect a document on the device, then send only the cropped image to a server for detailed processing.
The main toolkits
There are three common routes for mobile developers.
TensorFlow Lite. Google's framework for running models on mobile and small devices, now also offered under the name LiteRT. It works on both Android and iOS, which suits cross-platform teams. Models trained in TensorFlow, and increasingly in other frameworks, can be converted to its format.
Core ML. Apple's framework for iPhone, iPad and Mac. It is tightly integrated with Apple hardware and works alongside higher-level frameworks such as Vision for images and Natural Language for text. Apple provides tools to convert models from other formats into Core ML.
ML Kit. A Google SDK with ready-made features such as text recognition, barcode scanning, face detection and language identification. It runs on Android and iOS, and many of its features run on the device. It is often the fastest way to add a standard feature without training anything.
Flutter and React Native apps can use all of these through plugins or native modules. For custom models, the work usually involves training on a desktop or cloud machine, converting the model, shrinking it, and testing it on real phones.
Making models fit on a phone
A model that works well on a powerful computer may be too large or too slow for a phone. Developers use several techniques to fix this.
- Quantisation stores numbers with less precision, which makes the model smaller and faster, sometimes with a small loss of accuracy.
- Pruning removes parts of the model that contribute little.
- Smaller architectures designed for mobile can replace large general models.
- Hardware delegates let the model run on the phone's graphics processor or neural engine instead of the main processor.
Always test on a range of devices, including older and lower-cost phones. A feature that runs smoothly on a new flagship can struggle on a budget handset, and many of your users may own the latter.
Checklist for an on-device AI feature
- Is the task narrow enough for a small model?
- Does it need to work offline or respond instantly?
- Does it handle personal data that users would prefer stays on the phone?
- Is there a ready-made option in ML Kit or Apple's frameworks before you build a custom model?
- How will you update the model after launch?
- Have you tested speed, battery use and accuracy on older devices?
- What happens in the app if the model is unsure or fails?
Frequently asked questions
Can large language models run on a phone?
Smaller language models can run on recent phones, and both Apple and Google are adding on-device AI features to their platforms. Larger models still generally need a server, so many apps combine the two.
Does on-device AI make my app larger?
It can. Models add to the download size. You can reduce this by compressing the model or downloading it after install, only for users who need the feature.
Is on-device machine learning more secure?
It reduces the amount of data that leaves the phone, which helps with privacy. The model file itself can still be extracted from the app, so do not treat it as a place to hide secrets.
If you want to add camera, text or voice intelligence to your app, our mobile app development team can help you decide what should run on the phone and what belongs on the server.