The fundamental shift
Traditional software does what it is told. Given the same input, it returns the same output, and when it is wrong, an engineer can read the code and find the bug. A product with a machine-learned component behaves differently. Its outputs are predictions with a probability of being right. The same input can produce a different output after retraining. When it is wrong, there is no line of code to fix; there is data to examine and a model to improve. Managing this kind of product requires a different mindset about requirements, testing, launch, and success.
What this means for a product manager
- Requirements become targets, not specifications. You cannot write ‘the system shall classify every email correctly’. You write ‘the system shall classify at least 92 percent of emails correctly on the agreed test set, with precision above 95 percent on the urgent class’.
- Errors are a design surface. Since errors are guaranteed, the product must be designed around them: how they are shown, how users recover, how they are reported back.
- Data is a product input. The quality, quantity, and freshness of data determine what is possible, as much as engineering effort does.
- Launch is a beginning. Performance drifts as the world changes. The product needs monitoring, feedback loops, and a retraining plan from day one.
Step-by-step: reframing a feature
- Write the feature as a traditional requirement.
- Identify the component that is a prediction rather than a rule.
- Rewrite that component as a metric target on a defined dataset.
- Write the error-handling requirement: what the user sees when the prediction is wrong or uncertain.
- Write the feedback requirement: how the product learns that it was wrong.
Worked example
An expense-management product wanted to auto-categorize receipts. The first requirement read: ‘Receipts are automatically assigned to the correct expense category.’ Reframed: ‘On the labeled test set of 2,000 receipts, the model assigns the correct category at least 88 percent of the time. Predictions with confidence below the threshold are shown as a suggestion with a dropdown rather than pre-filled. Users can change any category with one tap; every change is logged as a correction. Weekly, the team reviews correction patterns.’ The reframed version gave engineering a clear target, gave design a clear error state, and gave the team a built-in improvement loop.
Common mistakes
- Promising accuracy before anyone has looked at the data.
- Designing only the happy path and leaving the error state to engineering.
- Treating the first model as the finished product.
Action Step: Take one feature in a product you know that uses or could use a prediction. Rewrite its requirement using the five-step reframing. Pay special attention to what the user sees when the prediction is uncertain.