The Three Places AI Integration Actually Pays Off
Across the mobile apps we've shipped from community feeds to legal research tools three categories of AI integration consistently earn their engineering cost: on-device recommendation ranking, retrieval-grounded in-app assistants, and background classification that improves a feature the user already relies on.
On-Device vs. Server-Side Inference
A recommendation model small enough to run on-device removes a network round-trip and a server bill entirely. We reserve server-side LLM calls for tasks that genuinely need a large model grounded question-answering, summarization, or anything where the app is a thin client over a knowledge base.
import { runOnDeviceModel } from "./ml/recommendation-model";
export async function getFeedRanking(userId: string, candidates: FeedItem[]) {
const scores = await runOnDeviceModel(candidates.map((c) => c.features));
return candidates
.map((item, i) => ({ item, score: scores[i] }))
.sort((a, b) => b.score - a.score)
.map((r) => r.item);
}Where AI Integration Backfires
The two failure patterns we see most: a chatbot bolted onto a support screen with no access to the user's actual account data, and an AI feature added purely for the app-store description rather than a measured user need. Both erode trust faster than having no AI feature at all.