You're building a mobile app - or upgrading one - and AI features are on the roadmap. Smart search. Personalized recommendations. Document scanning. Voice interaction. Image recognition. The question isn't whether to add AI anymore. It's where that AI runs.
This decision has real consequences for your budget, your user experience, your privacy posture, and your ability to scale. Get it wrong and you're either burning money on cloud inference at scale or shipping underwhelming AI that frustrates users.
Here's how to think about it.
The Three Options
On-device AI runs models directly on the user's phone - using the CPU, GPU, or dedicated neural processing unit (NPU). The model weights ship with your app or download once. After that, every inference is free. No server calls, no latency from network round-trips, no data leaving the device.
Cloud AI sends requests to a remote model (OpenAI, Anthropic, Google Vertex, etc.) via API. You get access to the most capable models available - frontier intelligence with broad world knowledge and complex reasoning - but every query costs money and requires a network connection.
Hybrid AI routes tasks intelligently between local and cloud models. Simple, frequent tasks run on-device. Complex, occasional tasks go to the cloud. You get the economics of edge inference with the ceiling of frontier capability.
The Cost Math That Changes Everything
This is where most teams get surprised.
Cloud AI charges per token. A single API call might cost a fraction of a cent. But mobile apps serve thousands or millions of users, and AI features get used constantly once they work well. At scale, per-query costs compound fast.
Consider a document scanning feature that uses AI to extract and categorize text. If you're making a cloud API call every time a user scans a receipt, and you have 50,000 active users averaging 10 scans per month, you're looking at 500,000 API calls monthly - potentially $500–$5,000/month depending on model and token count.
On-device? That same feature costs you nothing per inference after the initial model is deployed. The model ships inside your app bundle (or downloads at first launch), and the phone's NPU handles the work. Your cost scales with development and model updates, not with usage.
The rule of thumb: If your AI feature is high-frequency (used many times per session), runs on a task that a smaller model can handle well, or needs to work offline - on-device wins economically.
Latency: The User Experience Differentiator
Cloud API calls add 200–2,000 milliseconds of latency per request. For some features, that's fine - a user generating a summary of a long document can wait a second. For others, it's a dealbreaker.
Real-time features that demand on-device inference:
- Autocomplete and predictive text - Users expect instant responses. Even 300ms of lag feels broken.
- Camera-based recognition - Object detection, barcode scanning, real-time translation overlays need frame-rate processing.
- Voice interaction - Transcription and intent detection need sub-100ms response to feel natural.
- Gesture and motion detection - Fitness apps, AR experiences, and accessibility features process sensor data continuously.
If your AI feature needs to respond in under 200ms, or needs to process data continuously (not in discrete request/response cycles), cloud AI isn't viable. On-device is the only option.
Privacy and Compliance: The Non-Negotiable Cases
Every time your app sends data to a cloud AI provider, that data transits their infrastructure. For consumer social apps, that's usually acceptable. For healthcare, financial services, legal tech, or apps handling children's data, it often isn't.
On-device AI eliminates the data transmission question entirely. The data never leaves the phone. There's no third-party processor to evaluate under HIPAA, GDPR, CCPA, or COPPA.
If your app handles any of the following, on-device should be your default:
- Protected health information (PHI)
- Financial account data
- Biometric data
- Children's personal information
- Attorney-client communications
- Location history combined with identity
This isn't just about compliance checkboxes. It's a competitive advantage. "Your data never leaves your device" is a trust signal that converts privacy-conscious users - and that segment is growing fast.
The Capability Gap: What On-Device Can (and Can't) Do
Here's the honest tradeoff. On-device models are smaller - typically 1B to 7B parameters on current hardware - compared to frontier cloud models with hundreds of billions of parameters. That gap matters for some tasks.
On-device handles well today:
- Text classification and intent detection
- Named entity recognition
- Image classification and object detection
- Speech-to-text transcription
- Simple summarization
- Semantic search over local data
- Autocomplete and text prediction
Cloud AI still wins for:
- Complex multi-step reasoning
- Long-document analysis (50K+ tokens)
- Code generation across large codebases
- Tasks requiring current world knowledge
- High-quality image or video generation
- Multi-document synthesis and comparison
The gap is narrowing quickly. Quantization techniques now let 4-bit models run in 4GB of RAM with minimal quality loss. Google's Gemma and Meta's Llama both ship edge-optimized variants specifically designed for phones. Apple's Neural Engine on the iPhone 16 delivers roughly 35 TOPS (trillion operations per second) - enough to run a 4B parameter model at conversational speed.
What's Coming: Apple Core AI and the Platform Shift
This decision is about to get easier - and more important.
Apple is expected to unveil "Core AI" at WWDC in June 2026, replacing the Core ML framework that has powered on-device inference since 2017. According to Bloomberg's Mark Gurman, the new framework will add standardized third-party model integration (potentially via the Model Context Protocol), ship with Apple's own Foundation Models, and support agentic capabilities for the first time.
Google's ML Kit already provides robust on-device inference on Android with NNAPI support. Their AI Edge Gallery lets developers run LLMs offline directly on phones.
The platform vendors are making massive bets on on-device AI. That means better tooling, better optimization, and more capable local models every year. If you architect for on-device today, you'll benefit from those improvements automatically.
The Decision Framework
For each AI feature in your app, ask these four questions:
1. How often will users trigger this feature?
- Multiple times per session → On-device (cost scales linearly with cloud; stays flat on-device)
- A few times per week → Cloud is viable if capability demands it
- Once at setup or occasionally → Cloud is fine
2. What latency does the UX require?
- Under 200ms (real-time) → On-device only
- 200ms–2s (interactive) → Either, depending on other factors
- 2s+ acceptable (background processing) → Cloud is fine
3. What data does the feature process?
- Sensitive/regulated data → On-device strongly preferred
- User-generated content with privacy expectations → On-device preferred
- Non-sensitive, public, or anonymized data → Cloud is acceptable
4. How complex is the AI task?
- Classification, detection, transcription, simple NLP → On-device capable today
- Multi-step reasoning, generation, broad knowledge → Cloud required (or hybrid)
- Both types intermixed → Hybrid architecture
Most apps end up hybrid. The high-frequency, latency-sensitive, privacy-critical features run locally. The complex, occasional, knowledge-intensive features call the cloud. A well-designed routing layer handles the split transparently.
Implementation Paths for React Native and Native Apps
If you're building cross-platform with React Native (which we do at Apptitude for most mobile projects), your options look like this:
On-device: Use platform-specific native modules that wrap Core ML (iOS) or ML Kit/TensorFlow Lite (Android). Libraries like react-native-fast-tflite and react-native-executorch bridge local model inference into your JS layer. You write the feature logic once but configure models per platform.
Cloud AI: Standard REST API calls to OpenAI, Anthropic, or Google from your backend. Your React Native app hits your own API, which proxies to the AI provider. Never call AI APIs directly from the client - you'd expose keys and lose control over rate limiting.
Hybrid: Build an abstraction layer in your backend that routes requests based on task type. Simple inference routes to a "suggest on-device" response that tells the client to use its local model. Complex requests get processed server-side. The client doesn't need to know which path was taken.
For native Swift or Kotlin apps, the integration is more direct - Core ML and ML Kit are first-party APIs with excellent documentation and optimization tools.
What This Means for Your Budget
Here's the planning reality:
On-device AI features cost more upfront - model selection, optimization, testing across device tiers, managing model updates - but have near-zero marginal cost at scale. Budget 20–40% more development time compared to a cloud API integration, but your ongoing costs are predictable and independent of user growth.
Cloud AI features are faster to ship initially - an API call is simpler than on-device model optimization - but carry ongoing inference costs that scale with usage. Budget for the API costs at 10x your current user base, not just today's numbers.
Hybrid features have the highest initial development cost (you're building both paths plus routing logic) but the best long-term economics. They're worth it when the feature is core to your app's value proposition and you expect significant scale.
Our Recommendation
If you're adding AI to a mobile app in 2026, start with this default:
- Default to on-device for anything real-time, privacy-sensitive, or high-frequency.
- Use cloud AI for complex reasoning, generation, or knowledge-intensive features where capability matters more than latency.
- Build hybrid for your most important AI feature - the one that defines your app's value. Invest in the routing logic and dual-path architecture.
- Architect for migration. Today's cloud-only feature might move on-device in 12 months as models improve. Don't hard-couple your feature logic to a specific model provider.
The teams that get this right build AI features that feel instant, respect user privacy, don't blow up their infrastructure budget at scale, and improve automatically as on-device models get more capable.
The teams that get it wrong either ship slow, expensive AI that bleeds margin - or underinvest in capability and deliver features that feel like gimmicks.
At Apptitude, we build mobile apps with AI baked in - not bolted on. If you're planning AI features for a new or existing app and need help navigating the on-device vs. cloud decision for your specific use case, let's talk.