skip to content

Search

Identifying AI Opportunities in Your Product

Less than one minute read Updated:

A practical filter for when AI earns its complexity, and when simpler tools win.

# product-management
# strategy
# leadership
# ai
# learn
# opportunity
# discovery
AI for Product Managers

The fastest way to burn a quarter is to fund a model before you fund a definition.

Every AI pitch I have watched, at Pixis and at InMobi, arrives the same way. A vendor demo lands. A competitor ships something with a chat icon. The team starts scoping a model before anyone has written down what “working” means.

I have owned both sides of that mistake. I led the AI-facing products, AdRoom and Prism, and I owned the plain reliability engineering underneath them, the Data Platform and Execution Engine at Pixis and the system of record under InMobi’s ad exchange. One thing decided whether the AI bet paid off, and it was rarely the model.

So that’s today’s post.

Today’s Post

A filter for killing the wrong AI ideas before they cost you a quarter, in seven sections.

  • Why rules beat AI for anything a human can already write down
  • The training-signal test that catches half of the pitches early
  • Pricing the cost of being wrong, before you ship
  • Human review as the feature, not the fallback
  • AI as a feature people see, versus AI as infrastructure nobody sees
  • The one-page brief that gates funding
  • Why the unglamorous bet usually wins

1. Rules first, always

If a task is fully specified by rules a human could write down, write the rules down. Do not fund a model to do it.

Rules-first test: eligibility checks, routing, formatting, deterministic validation. All of these are debuggable, predictable, and easy to explain to legal or a customer. AI earns its complexity when the mapping from input to output is too fuzzy, too large, or too expensive to hand-write, not when a team is too impatient to document the logic.

AdRoom, the AI creative product I led at Pixis, is the proof that rules and models sit inside the same product without conflict. AdRoom generates on-brand ads with a model. It enforces the brand guidelines, fonts, color, logo rules, voice, with a rule, checked at generation time rather than after. That rule is the reason the team could kill the manual QA gate entirely and still trust the output. The model does the fuzzy part. The rule does the part a junior analyst with a style guide could already do.

Rules first, always A task enters, one gate decides whether a human could already write the mapping down. Grey monospace text names the real AdRoom example of each branch. A task enters brand check on an ad font, colour, logo, voice Could a human write the mapping by hand? fully specified vs too fuzzy yes: fully specified Rules-based build deterministic, debuggable, explainable to legal AdRoom's guideline check no: too fuzzy to hand-write Model-based build learns the fuzzy part a rule cannot specify AdRoom's ad generation a task to automate the decision gate rules win here a model earns its complexity here AdRoom runs both at once: a model generates the ad, a rule enforces the brand guideline at generation time.
AdRoom runs both branches at once: a model drafts the ad, a rule checks the brand guideline at generation time. PNG ↓

2. The training-signal test catches half of the pitches early

Repetition is the first screen. If experts make the same judgment call thousands of times, with similar inputs and comparable stakes, you might have a pattern a model can learn.

Repetition alone is not enough. The task also needs a signal you can count, and two trusted humans need to agree often enough that the signal means something. At Pixis, before AdRoom’s guideline engine existed, brand reviewers judged creative by eye. If the disagreement rate is high, the problem is a missing policy, not a missing model, and no amount of labeling fixes it.

The instrumentation choice matters just as much as the agreement rate. When Pixis ran the two-week onboarding sprint for new AdRoom brands, the team picked one hard signal: a published, on-brand ad, live inside the window. Logins were ignored on purpose. Across four automotive and consumer-tech brands, that one signal produced $800K ARR in two months, and the full motion reached $0 to $10M ARR in five months with a 20-person team. Counting logins instead would have declared victory in week one and taught the team nothing about which wedge actually worked.

3. Price the cost of being wrong, before you ship

AI systems are rarely perfect, and the real question is whether “mostly right” is good enough for the specific decision you are automating.

Error economics: write down what a false positive costs and what a false negative costs, in dollars or in trust, before you pick a threshold. A wrong suggestion that a user ignores with one click is cheap. A wrong action that touches someone’s live ad spend or a customer’s account is not.

Prism, the AI co-pilot I led at Pixis, was built around that exact asymmetry. Prism watches campaigns, drafts creative through AdRoom, and surfaces one-click actions on its own recommendations. The team drew one line early: the AI proposes and explains, and a human commits. That line is not a UX nicety. It is what let a natural-language surface touch real ad budgets without betting the company on the model always being right, and it is part of why the launch moved trial-to-paid up 25% and onboarding time down 40%.

Price the cost of being wrong What kind of automation is safe depends on what a false positive and a false negative each cost. Grey monospace text names the real Prism example in each cell. FALSE POSITIVE FALSE NEGATIVE LOW COST HIGH COST One-click suggestion a wrong draft the user can ignore with one click Prism drafts a creative, user skips it, no harm safe to automate fully Missed suggestion a good draft never surfaced, nothing was risked Prism stays quiet, the human proceeds as before safe to automate fully Needs a human gate an action that touches live ad spend, wrongly triggered Prism proposes the spend move, a human commits it gate before it fires Needs monitoring a real problem missed, silent until someone checks a spend risk the model never flagged at all schedule a review, do not trust silence cheap to be wrong here, automate fully
Cheap to be wrong, automate it fully. Expensive to be wrong, gate it or monitor it. PNG ↓

4. Human review is the feature, not the fallback

The mature pattern across most AI products I have shipped is the model drafts and a person decides. That is not a compromise. It is usually the fastest path to value at an acceptable level of risk.

If nothing routes a low-confidence output back to a person, you are betting the model is always safe, and that bet ages badly. Prism’s propose-then-commit split is one version of the gate. The other version I have run is upstream of the model entirely: at Pixis, 15-plus voice-of-customer interviews on the Prism launch became exactly five roadmap features, not fifty, because the discipline was declining the other forty. Saying no with evidence is a form of review too, and it is cheaper than reviewing a shipped feature nobody asked for.

The InMobi side of my work runs the same discipline further downstream, in how a measurement system, not just a dashboard, decides whether a feature worked. The habit is the same one: someone has to look, on a schedule, and be allowed to say the output was wrong.

5. AI as a feature people see, versus AI as infrastructure nobody sees

Feature AI is what a user notices: a co-pilot, a drafting tool, a recommendation surfaced in the UI. Success there is adoption and task completion. AdRoom and Prism both live here, and both are judged on ARR, trial-to-paid, and onboarding time.

Infrastructure AI sits underneath: scoring, routing, forecasting, the pipeline that decides what the feature is allowed to trust. Success there is reliability, latency, and cost, and failure is silent until it is not. My own first mandate at Pixis was exactly this layer, the Data Platform and Execution Engine, with a 12-person pod and an unglamorous thesis: reliability is the product. That engine ended up processing 1M-plus keywords a day for 400-plus enterprise customers at 99.6% end-to-end sync reliability, and cut data availability delay from 4 hours to under 15 minutes. Advance, the performance-AI layer built on top of it, only earned the enterprise trust it has because that number came first.

Both tracks are legitimate, and they need different stakeholders. Feature AI needs a crisp UX promise and a visible fallback for when it is wrong. Infrastructure AI needs monitoring, an SLA, and a partnership with whoever owns compliance, because a silent failure there compounds for months before anyone notices.

Feature AI versus infrastructure AI Two legitimate tracks, matched row by row, with different stakeholders and different risk. Grey monospace text names each track's real system and number. FEATURE AI · WHAT A USER NOTICES INFRASTRUCTURE AI · WHAT SITS UNDERNEATH WHO NOTICES The user, in the product, every session AdRoom's drafting tool, Prism's co-pilot WHO NOTICES No one, until it fails the Data Platform's sync pipeline METRIC THAT PROVES IT WORKED Adoption and task completion trial-to-paid up 25%, onboarding down 40% METRIC THAT PROVES IT WORKED Reliability, latency, and cost 99.6% end-to-end sync reliability WHAT A FAILURE LOOKS LIKE A visible bad draft or suggestion, reported the same day a wrong ad caught in QA, loud WHAT A FAILURE LOOKS LIKE A silent sync delay that compounds for months before anyone checks 4-hour delay, before the 15-minute fix Needs a UX promise and a fallback AdRoom and Prism Needs an SLA and a compliance partner Data Platform, 400+ enterprise customers Both are legitimate discovery tracks. Neither substitutes for the other.
Feature AI is judged by adoption. Infrastructure AI is judged by whether anyone ever has to notice it. PNG ↓

6. The one-page brief that gates funding

Treat an AI proposal like any other expensive bet: require a short assessment before it reaches the backlog, lightweight, but mandatory for anything touching a model.

Six questions, in order, and a proposal has to clear each one before it moves to the next:

  1. Problem clarity: what user or business outcome improves, and through what mechanism?
  2. Baseline: what non-AI solution exists today, and what bar must the model beat?
  3. Data and labels: what evidence exists now, what has to be collected, and who owns its quality?
  4. Error economics: what happens when the system is wrong, and how fast can you detect it?
  5. Operational load: latency, cost per action, retraining, monitoring, and who owns the ongoing health of it?
  6. Governance: privacy, consent, bias risk, and third-party terms. What blocks launch?
The proposal moves through six gates One AI proposal, checked in order. It clears a gate in sage or stalls and turns coral at whichever one it fails. Scroll slowly: the token travels once, left to right, and settles at the gate it reaches. Problem clarity 1 Baseline 2 Data and labels 3 Error economics 4 Operational load 5 Governance 6 clears the gate, continues sage fails the gate, stops and turns coral Funding follows the gate it clears: a prototype, a pilot, or a pause.
This run clears every gate. A real proposal stalls, turns coral, and stops at whichever question it cannot answer. (animates as you scroll to it) PNG ↓

The output of that brief is not a go or no-go from one product manager alone. It is a shared picture that lets leadership fund the right phase, a prototype, a pilot, or a pause, with everyone looking at the same six answers.

7. Why the unglamorous bet usually wins

Competitive pressure is real. “A competitor shipped an assistant, we need one” can be a rational parity call. It is still not a problem statement until you name the job the assistant does, for whom, and what outcome improves, or you end up shipping a checkbox that drains support and disappoints the first user who tries it.

The best AI opportunities I have shipped rarely looked exciting on a slide. The Data Platform’s 4-hour-to-15-minute delay cut does not screenshot well for a board meeting. It shows up in the 99.6% reliability number that let 400-plus enterprise customers trust Advance’s recommendations with real spend, and that is the same pattern behind InMobi’s month-end revenue close moving from 5 days to 12 hours. Neither one is a demo. Both are the reason the visible feature on top gets to look good.

Ask engineering for the cheapest credible baseline before you fund a model. If an LLM cannot beat that baseline on cost, latency, and quality in a real eval, no amount of impressive-looking output should get it shipped. The rules-first test from section 1, the training-signal test from section 2, and the funding brief from section 6 all exist to catch that gap before it costs a quarter instead of a meeting.

Takeaways

A filter for the next AI pitch you get Seven sections on killing the wrong idea before it costs a quarter Hoshank Ailani · hoshank.vercel.app 1. Rules first, always Test: could a human write the mapping down by hand? Yes → rules. No → a model earns it. AdRoom: a model drafts the ad, a rule checks the brand guideline the manual QA gate: killed, not kept 2. The training-signal test Repetition alone is not enough Need a signal you can count Pick that signal before instrumenting AdRoom sprint: counted a published, on-brand ad, not a login $800K ARR in two months, four brands 3. Price the cost of being wrong LOW COST HIGH COST one-click suggestion, ignorable touches live spend, needs a human gate Prism: proposes and drafts, a human commits the action +25% trial-to-paid, -40% onboarding time 4. Feature AI vs infrastructure AI AdRoom, Prism what a user notices, judged on ARR Data Platform what sits underneath, judged on uptime 1M+ keywords/day, 400+ customers 99.6% sync reliability, silent otherwise 5. The six-question funding gate 1 Problem clarity 2 Baseline it must beat 3 Data and labels, who owns quality 4 Error economics, how fast detected 5 Operational load, who owns health 6 Governance, what blocks launch a shared picture: fund a prototype, a pilot, or a pause, on the same six answers 6 & 7. The unglamorous infrastructure bet usually wins A demo that screenshots well is not the same bet as the one that pays off. Data Platform: 4h → 15min sync delay · InMobi month-end close: 5 days → 12 hours Ask engineering for the cheapest credible baseline before you fund a model. If an LLM cannot beat it on cost, latency, and quality, it should not ship. gates 1, 2, and 6 exist to catch that gap before it costs a quarter Fund the definition before you fund the model. Run the last AI proposal your team funded through the six-question brief. If it stalls at gate two, you already know what to do differently. Full post: hoshank.vercel.app/posts/ai-track-identifying-opportunities Takeaways
The whole post on one card. Save it, share it, argue with it. PNG ↓

Seven sections, one filter. Run it on the next AI idea someone pitches you before it reaches a sprint.

Final Words

Good discovery does not mean never using AI. It means using it where the mapping is genuinely too fuzzy for a rule, where you can name the signal before you instrument it, and where you have already priced the cost of being wrong. Everywhere else, the rule, the baseline, or the review step wins, and it wins faster.

Pick the last AI proposal your team funded and run it through the six-question brief in section 6. If it would not have cleared gate two, you already know what to do differently next time.

Reply and tell me where the proposal stalled. I am collecting the misses for a follow-up.

Cheers, Hoshank