Most AI pitch decks show a box labeled "AI/ML" floating next to the architecture diagram. In practice, that box has to connect to data pipelines, backend services, business logic, and an interface the end user should not perceive as "intelligent."
The distance between that diagram and a feature working in production is where months and budget disappear.
What integrating AI into a product actually means
Building AI into a product is integration work. Think of it like adding a payments system or biometric authentication: understand the domain, pick the right infrastructure, and measure impact on the experience.
An NLP model classifying support tickets needs an ingestion pipeline, preprocessing, an inference service, a fallback for low-confidence cases, and continuous drift monitoring. Every piece lives in the same repo, sprints, and CI/CD as the rest of the product.
When we worked on MyBotPal for Huawei, an AI voice assistant on HarmonyOS NEXT, the core challenge was exactly that: making inference work offline with GPS, weather, and multi-language support under strict privacy standards.
The AI model was one part of the system, but the real complexity lived in device integration and the seamless offline experience users expected, a pattern that also shows up in our work with AI agent skills.
Technical decisions that define the experience
The first fork in any AI feature is where inference runs. On-device gives p50 latency under 50ms and removes the network dependency, but limits you to models under 20-40MB for reasonable battery consumption.
Cloud inference lets you use bigger models and update them without a deploy, but adds network latency, with p95 climbing to 300-500ms depending on region, and opens privacy questions about data leaving the device.
Approach · Main advantage · Main limitation
On-device (LiteRT, ex TensorFlow Lite; ML Kit) · p50 latency under 50ms, no network dependency · Models limited to under 20-40MB to preserve battery
Cloud inference (Google Cloud AI, AWS SageMaker) · Larger models, updatable without a deploy · Network latency p95 of 300-500ms and open privacy questions
These decisions hit the roadmap directly. On-device means an MLOps pipeline that handles model quantization and automated accuracy testing.
A quantized model that goes from 0.3% to 1.2% false positive rate can make a computer vision feature start generating friction that hurts retention.
In cloud, the trade-off shifts to operating cost and privacy. Every inference request has a cost that scales with traffic, and sensitive data raises the question of whether it can leave the device.
In ProWallet, an instant-payments fintech for construction, the decision of what to process locally versus send to backend defined both architecture and the speed users expected on job sites with irregular connectivity.
The practical rule: if latency impacts a direct interaction, prioritize on-device. If the feature is asynchronous, cloud is usually more flexible and easier to maintain.
From strategy to a feature in production
An AI feature that reaches production predictably goes through four stages, the same staged discipline behind AI-driven development workflows.
- Discovery (2 to 4 weeks): defines whether AI solves the problem better than a heuristic, what data exists, and the success metric. Many features die here, and that is fine, since hardcoded rules solving 80% of the case saves months of complexity.
- Prototype (4 to 6 weeks): builds a working model with real data and sets accuracy baselines.
- Pilot (4 to 8 weeks): puts the prototype in front of real users, measuring latency and product metrics like conversion or retention.
- Production: integrates the feature with drift monitoring, degradation alerts, and a retraining pipeline that runs regression tests before promoting the model.
At Somnio, this folds into full product development, since AI enters as a layer only when the case justifies it, with the same team building the rest of the app.
With CAA Club Group, Canada's largest automobile association with 7M+ members, the roadside assistance and dispatch tools needed that level of integration to work at scale.
For startups without an in-house data science team, evaluating a partner comes down to three things: model ownership, MLOps support so you do not depend on the partner after handoff, and documented knowledge transfer.
When an AI feature is worth the investment
Before committing budget, a CTO needs to answer one question: does the business metric improvement justify the operational complexity you are adding?
A recommendation model that improves conversion by 0.5% but needs weekly retraining and a dedicated engineer probably does not close the equation for a product under 100K active users, the kind of judgment call in leading AI transformation.
- Inference latency (p50 and p95) measures whether the feature responds within the time UX requires.
- Production model size determines viability on resource-constrained mobile devices.
- False positive and false negative rate matter for classification features where errors carry visible user cost.
- Cost per request scales with traffic and can become prohibitive if not projected early.
- 30-day retention impact connects technical quality to business outcome.
If you cannot define these metrics before you start building, you have a hypothesis that needs more discovery, not an AI case yet.
For interviews with potential partners, a few checks filter fast. Ask them to show a CI/CD pipeline that includes model validation, and what happens when a model degrades, whether they have alerts, rollback, and a runbook.
A partner who can answer with concrete examples has the operational maturity production AI requires.
Frequently asked questions
When does on-device make more sense than cloud inference?
When the feature impacts a direct interaction, like a tap, voice, or camera, on-device's lower latency justifies the quantization pipeline's complexity. If the feature is asynchronous, cloud is usually more flexible and easier to maintain.
What happens if the AI model degrades in production?
A mature technical partner has degradation alerts, automatic rollback, and a documented runbook. Without those, a model degradation can stay invisible until the retention impact is significant.
How do I know if my use case actually needs AI or a heuristic is enough?
The discovery phase exists to answer that. If hardcoded rules solve 80% of the problem, that is usually the right call, since it avoids months of complexity. AI is justified when a heuristic cannot capture the real variability of the problem.
What artifacts should stay in my infrastructure after an AI project?
Trained model weights, datasets, training and inference pipelines, and architecture decision records should stay versioned in repositories you own. Depending on the partner to retrain after handoff limits your ability to evolve the product.
What user base size justifies investing in an AI feature?
There is no universal number, but a model requiring weekly retraining and a dedicated engineer needs an active base large enough that the improvement covers that cost. For products under 100K active users, exhaust simpler solutions first.
The gap between a brilliant prototype and an AI feature that survives production is exactly what discovery is built to close, it is what let MyBotPal run inference offline under strict privacy standards without the project collapsing under its own complexity.
If you're searching for a trusted software development partner, look no further. Contact us today to learn how we can help you turn your vision into reality with our tailored, high-quality solutions.
