How to Know If an AI Feature Actually Needs AI

Most feature requests that get pitched as "we need AI for this" don't actually need AI. They need a rule, a better default, or a cleaner workflow that was never built.

The way to tell the difference is simple: try to write the decision down as a short, plain rule. "If the order total is over $500, flag it for review." "If the user hasn't logged in for 30 days, send a re-engagement email." If a person could follow that rule by hand, correctly, most of the time, you don't have an AI problem. You have a rule nobody wrote down yet.

AI earns its place when that exercise breaks down — when the rule keeps growing exceptions, when the inputs are too varied to enumerate, or when the pattern you're trying to catch shifts over time faster than anyone could keep rewriting the logic. That's a specific, checkable condition, not a feeling.

Why does this distinction matter right now?

It matters because "AI feature" has become the default pitch for almost anything a product team wants to ship, regardless of whether a model actually improves the outcome.

Jason Fried, co-founder of 37signals, put this plainly in his own writing: today's productivity tools are packed with "overhyped AI features that fail to do what they promise and end up providing little in the way of practical value." That's not a rejection of AI. It's a specific complaint about features added because AI was available, not because the feature needed it.

The cost of getting this wrong isn't abstract. A model is slower to ship than a rule, harder to debug when it's wrong, and more expensive to run at scale. A feature built on a rule can be explained to a user in one sentence. A feature built on a model that didn't need to be one just adds all of that cost for no real gain.

What does Google's own machine learning guidance say about this?

Google's internal "Rules of Machine Learning" guide, written by machine learning engineer Martin Zinkevich, opens with exactly this warning as its very first rule: "Don't be afraid to launch a product without machine learning."

The reasoning is specific. Machine learning needs data to work well, and a model trained on borrowed or limited data will often underperform a basic heuristic. Zinkevich's own framing is direct: if you think machine learning will give you a 100% improvement, a well-built heuristic will usually get you 50% of the way there, for a fraction of the cost and time.

Zinkevich's guide isn't anti-AI. Later rules in the same document explicitly recommend choosing machine learning over a complex heuristic, once that heuristic becomes unmanageable to maintain by hand. The point is sequencing: earn the case for AI by hitting the limits of a simple rule first, rather than assuming the limit exists before you've tested it.

That sequencing matters more, not less, now that generative AI makes it trivially easy to bolt a model onto almost any feature. The ease of adding AI has nothing to do with whether the feature needs it.

What criteria actually separate a "needs AI" feature from a "needs a rule" feature?

Google's own People + AI Research (PAIR) team publishes a guidebook specifically for teams deciding where AI belongs in a product, built from research across more than a hundred Google employees, industry practitioners, and academic sources. It frames the decision around a small number of concrete situations where AI tends to add real value:

• Personalized recommendations — when the right output genuinely differs per user, based on signals too numerous to hand-code as rules.
• Pattern discovery across large amounts of data — when the useful pattern isn't something a person could spot by looking at a handful of examples.
• Automation of a task that's repetitive and high-volume — when the volume makes manual handling impractical, not just mildly annoying.

The same guidance is just as specific about where AI tends not to help: tasks people actually enjoy doing themselves, and situations where the required accuracy is higher than a model can reliably deliver. Both are easy to miss when a feature is being pitched on how impressive it sounds rather than on what it actually needs to do.

None of these criteria are about whether AI is technically capable of doing the task. Modern models can attempt almost anything. The question is narrower: does this specific task have the shape where a model's strengths (handling large, messy, shifting inputs) actually apply — or does it have the shape of a rule that's simply never been written down?

What's the practical test to run on a feature request?

Run the request through a short sequence of questions before any build decision gets made.

1. Can the decision be written as a short, stable rule? If the logic doesn't change often and the inputs are a handful of known fields, a rule will serve you fine. A useful rough guideline from practitioners working in this space: if the decision tree stays under about 20 branches and doesn't need rewriting every quarter, that's a rule-shaped problem, not a model-shaped one.
2. Are the inputs structured and limited, or messy and open-ended? If the decision depends on fields already sitting cleanly in a database — amounts, dates, status codes, boolean flags — a model isn't adding value by interpreting them. If the input is open-ended text, images, or behavior that varies too much to categorize by hand, that's where a model's flexibility starts to matter.
3. Does the underlying pattern change over time? Fraud techniques evolve. Customer language shifts. A catalog of products grows in ways nobody anticipated. Rules require a person to notice the drift and rewrite the logic by hand. A model can be retrained on new data instead. If nothing about the problem drifts, this advantage doesn't apply.
4. What does a wrong answer cost, and does the interface allow for graceful correction? A recommendation that's occasionally off is a minor annoyance a user scrolls past. A miscalculated invoice or a wrongly denied request is a different category of failure. High-stakes, low-tolerance decisions favor a rule you can verify and audit, not a probabilistic system that's right on average.

If the honest answers point to a stable, structured, low-drift, high-stakes decision, that's a rule. If they point to messy, shifting, high-volume inputs where being right on average is genuinely useful, that's a real case for AI.

Does the answer have to be one or the other?

No, and treating it as an either-or question is its own mistake. Andrew Bonham, Senior Distinguished Engineer at Capital One, laid out this exact point in the company's own published engineering writing: rules and machine learning aren't competitors, they're layers that can feed into each other.

One pattern Bonham describes has the model doing the harder work first — generating a probability or a prediction from messy, high-volume input — and a rule taking that prediction and turning it into a final, auditable decision. The model handles the part that genuinely benefits from pattern recognition. The rule handles the part that needs to stay explainable, consistent, and easy to check when something goes wrong.

That's a materially different design than either "add AI to everything" or "never use AI." It's the same underlying question from this whole framework, just applied inside a single feature instead of across a whole roadmap: which specific part of this decision benefits from a model, and which part should stay a rule regardless of what the model outputs?

The mistake isn't choosing wrong between AI and rules. It's not asking the question at all, and defaulting to whichever one sounds more impressive in a roadmap update.

What does this mean for how a feature actually gets built?

None of this means avoiding AI. It means not assuming AI first, and not building it because the pitch sounds more current than "we added a setting" or "we fixed the default."

The cheapest way to find out which one you're dealing with is to try building the rule-based version first, even a rough one. Zinkevich's own later guidance in the same Rules of Machine Learning document backs this directly: keep the first version simple, and get the surrounding infrastructure right, before reaching for a more complex model. If the rule handles the real cases well, you've saved the cost of a model nobody needed. If it clearly falls short on a specific, nameable class of input, you now have concrete evidence for exactly what a model needs to do, instead of a vague sense that "AI would help."

That evidence-first approach is the same discipline behind Ingenetic's AI Readiness Audit: before committing budget to a build, the question worth answering first isn't "can we add AI to this," it's "does this specific problem actually need it, and what happens if we start with something simpler." See How It Works for how that audit is structured.

Frequently asked questions

When should you not use AI?

Skip AI when the logic is stable and well understood, when the inputs are structured and small in number, when a wrong answer is costly and unacceptable, or when a simple default already solves the problem for most users. AI earns its place when the input space is too large or messy for rules, or when the product needs to improve as it sees more real usage.

What's the fastest way to test if a feature request actually needs AI?

Try to write the decision as a short, plain-language rule first: "if this, then that." If a person could follow that rule by hand and get it right almost every time, you don't need AI. If the rule keeps growing exceptions every time you try to write it down, that's a real signal the problem needs a model.

Isn't it safer to just add AI in case the simple version isn't good enough?

No. A rule-based version is faster to ship, easier to debug, and easier to explain to a user when it's wrong. Shipping AI first means shipping something harder to test, harder to explain, and more expensive to run, before you've confirmed a simple version couldn't have done the same job.

Does this mean AI is usually the wrong answer for a new feature?

Not usually wrong — just not usually the first answer. Google's own internal machine learning guidance opens with exactly this point: don't be afraid to launch a product without machine learning first. Many teams find a simple heuristic gets most of the value, and the case for a model becomes obvious once real usage data shows where the heuristic actually breaks.

What if the feature genuinely does need AI?

Then it usually shows a specific signal: the input variety is too large to enumerate as rules, the underlying pattern shifts over time in a way fixed logic can't track, or the value clearly comes from personalization or pattern discovery across large amounts of data. If you can name which of those is true for your feature, that's a real case for AI, not just an assumption.