skip to content

Search

Data as a Product Requirement

Less than one minute read Updated:

How to treat data readiness as a product decision, not an engineering side quest.

# product-management
# strategy
# leadership
# ai
# learn
# data
AI for Product Managers

Most AI initiatives do not die in a training run. They die in a conference room, the day someone finally admits the data was never there.

At InMobi I own the system of record under a demand-side platform and an ad exchange that process 200M+ devices’ worth of ad events a day. I have also watched a pipeline that ingests billions of those events a day get poisoned by something as small as three spellings of one campaign: Facebook, facebook, and FaceBook, three rows to a dashboard that should have been one. Nobody caught it until the quarterly numbers refused to reconcile.

PMs are not paid to write the ETL. You are paid for clarity: what data has to exist, at what quality, flowing through what controls, before your team commits a quarter to a feature that needs it.

So that’s today’s post.

Today’s Post

Six sections on treating data readiness as a first-class product requirement, not an engineering side quest.

  • Why availability is a binary gate, not a line item in the roadmap
  • Quality as fitness for the decision you are about to automate
  • Pipelines and governance: the same promise, made to two different people
  • Why a label definition is product policy, and why leakage is a PM’s problem
  • Cold start and representativeness as product bets, with a real bootstrap
  • Data contracts, and the boring habits that keep a data-dependent roadmap honest

1. Availability is a binary gate, not a line item

Availability sounds mundane until it blocks you. Do you have the events, documents, labels, outcomes, and identifiers you need, where you need them, when you need them?

“We probably have logs” is not availability. Availability is provable: a schema, a retention window, a permission, and a lineage from the user’s action to the signal your model trains on.

Assumed and provable are not the same word "We probably have logs" checks nothing. Availability is a schema, a retention window, a permission, and a lineage, proven, not assumed. Grey monospace text names what each field means in practice. ASSUMED AVAILABLE PROVABLY AVAILABLE Schema "there's a field for that, probably" ✗ Schema named field, typed, versioned ✓ Retention "it's probably still there" ✗ Retention a written window, e.g. 90 days ✓ Permission "nobody's said no yet" ✗ Permission access grant on record, named owner ✓ Lineage "it comes from the app, I think" ✗ Lineage traced: user action to training signal ✓ Same four fields, either column. The right column is the acceptance criterion that belongs in the PRD.
Assumed and provable are not the same word. Check the same four fields either way. PNG ↓

Write the acceptance criterion the same way finance writes a close target. At InMobi, the reconciliation pipeline under the exchange did not need a new dashboard; finance could already see every line of revenue from those 200M+ devices’ worth of daily events. What it needed was a written number: days from month end to a signed figure. That sentence, joined to a deadline, is what turned the close from 5 days into 12 hours.

Product implication: put the same sentence in the PRD. “We can join support ticket text to resolution outcome within 24 hours for 95% of tickets.” If you cannot write that sentence, you do not yet know what you are building, no matter how good the model card looks.

2. Quality means fitness for the decision you are about to automate

Quality covers accuracy, completeness, timeliness, and consistency across systems. For a model, quality also means representativeness: does the data mirror the populations and scenarios the model will meet in production, or only the ones that were easiest to log?

A model trained on one region, one segment, or one quarter will fail confidently everywhere else. That is not a surprise. It is the training data doing exactly what it was shown.

Product implication: fund a data audit as discovery work, the way you would fund user research, because it is user research on your own information supply chain. Sample rows with the people who know the domain. Measure null rates and drift. Publish the findings the way the month-end close and the four-step measurement loop get published: with a number, a date, and a verdict, not a paragraph of reassurance.

3. Pipelines and governance are the same promise, made to two different people

A pipeline moves information from sources, app events, a CRM, documents, human annotation, to the place a model consumes it. Along the way it gets validated, transformed, aggregated, versioned, and often de-identified.

Four stages, one promise, two audiences A pipeline moves data. Governance is the same promise made to legal and security. The stage most roadmaps skip is the third. Grey monospace text names each stage's real check. Scroll slowly, this plays once. Raw event Whatever the app or device just logged click, install, ad event Validated, transformed Schema checked, joined, aggregated, versioned the part that looks like ETL Checked against a governance rule consent, retention, fairness the stage most roadmaps skip Available to the model Cleared for training or inference use what the PRD should have named a late governance surprise becomes a launch delay or a headline the outside world the happy path the judgement call, skipped last
Four stages, one promise. The governance check is the stage most roadmaps skip, and it is the one that turns into a headline. (animates as you scroll to it) PNG ↓

You do not have to pick the technology. You do have to know the shape of the promise: latency, reliability, and ownership, meaning who gets paged when the feed breaks. At Pixis, my first mandate was not the AI layer everyone wanted to demo. It was a Data Platform and Execution Engine, a 12-person pod with an unglamorous thesis: reliability is the product. We wrote SLAs, automated validation, and automated failure detection, and the engine ended up processing 1M+ keywords a day for 400+ enterprise customers at 99.6% sync reliability, with availability delays cut from 4 hours to under 15 minutes. None of that demoed well. All of it is the reason the optimization layer built on top of it was worth trusting with real ad spend.

Governance is the same promise made to a different audience: privacy, consent, retention, access control, and regulatory limits, plus bias and fairness, meaning which groups are underrepresented and what mitigation is required. It is not a review that happens at the end. It is a design constraint that shapes what you collect in the first place.

Product implication: bring legal and security in early, with specifics: not “we might use AI,” but “we propose these fields, for this purpose, with this retention and these user controls.” A late governance surprise becomes a launch delay or a headline, and both are more expensive than the conversation you skipped.

4. Why a label definition is product policy, and why leakage is a PM’s problem

A label is not a neutral fact. It is a definition somebody chose. “Churn risk,” “toxic,” and “qualified lead” mean different things on different teams, and when the definition drifts, your model learns the wrong objective.

The clearest version of this I have lived is not even a model label. It is a UTM value. Treat your taxonomy like a schema, I wrote after watching one campaign split into three rows because nobody enumerated the values or assigned an owner. You do not let engineers invent column names in production. Stop letting marketers invent UTM values, and stop letting anyone invent a label definition without a version number attached to it.

At Pixis, the version of this that worked was enforcing brand guidelines at generation time, not after. Fonts, color, logo rules, and voice became constraints the model could not violate, so the manual QA gate could disappear entirely and designer load dropped 80% without anything going off-brand. The definition of “on-brand” was written down and enforced upstream, exactly like a label definition should be.

Leakage is the quieter version of the same failure. It happens when information from the future, or from the label itself, sneaks into the training input. The model looks brilliant offline and collapses in production because it was never learning what you thought.

Product implication: require a plain-language narrative for how a dataset was split and which features were allowed in. If nobody on the team can say it out loud, pause the celebration.

5. Cold start and representativeness are product bets

Cold start is the empty room at the beginning: not enough signal to personalize, predict, or automate with any confidence.

The clean example I have of bootstrapping through it is not a modeling technique. It is a two-week onboarding sprint. Every new AdRoom brand at Pixis went through the same window with one hard goal: get the brand self-serving live, on-brand ads inside two weeks. We instrumented activation, meaning a published on-brand ad, and ignored logins entirely. Four automotive and consumer-tech brands later, the sprint had produced $800K ARR in two months, on the way to the full motion going from $0 to $10M ARR in five months with a 20-person team.

Zero signal on day 0, activated by day 14, all four brands A two-week onboarding sprint with one instrumented goal: a published on-brand ad. We ignored logins entirely. Mono day numbers mark when each brand crossed the activation line. day 0 day 7 day 14 Brand 1 day 6 Brand 2 day 9 Brand 3 day 11 Brand 4 day 13 zero signal, no published ad yet activated: self-serving, on-brand, live $800K ARR in two months four automotive and consumer-tech brands, one activation goal, zero brands measured by login
Zero signal on day 0, a published on-brand ad by day 14, for all four brands. That is the whole bet. PNG ↓

Had we measured logins that quarter instead, we would have called it a win in week one and learned nothing about whether the wedge was real.

Product implication: roadmap the cold start experience as carefully as the mature one. A mediocre first session is how a product loses a user before it ever collects the signal it needed from them.

6. Data contracts, and the boring habits that keep a data-dependent roadmap honest

Most roadmap slips are not model failures. They are upstream data that arrived late, changed shape, or never matched what the team assumed. A one-page data contract, owner, refresh cadence, required fields, acceptable null rate, and what happens when the feed breaks, turns “we should have logs” into an accountable dependency.

"We need churn data" is not a data contract A vague ask names no owner and sets no bar. A one-page contract answers all five before engineering commits a sprint. Grey monospace text is the concrete version of each field. VAGUE ASK DATA CONTRACT Owner "someone on the data team" ✗ Owner a named person, one Slack handle ✓ Refresh cadence "it updates regularly" ✗ Refresh cadence hourly, written down ✓ Required fields "the usual churn stuff" ✗ Required fields plan, last_active, cancel_reason, listed ✓ Null rate threshold "mostly complete, I think" ✗ Null rate threshold under 2%, checked at ingest ✓ A fifth field, escalation path, who gets paged when the feed breaks, closes the contract on the right; the left column never asks the question.
A vague ask has no owner. A data contract names one, on paper, before the sprint starts. PNG ↓

You do not have to write the pipeline. You do have to insist the dependency is named before engineering commits a sprint to it. Contracts also force the consumer-versus-producer conversation into the open: if sales operations owns CRM hygiene and your model needs clean stage history, that is a negotiation to have in week one, not a surprise in week ten.

The same discipline applies to data you do not own at all. When you rely on a third-party model or dataset, you still own the product outcome. At Pixis, evaluating measurement partners like AppsFlyer for the MMP integration meant treating a vendor’s claims the way ad-tech treats an ad network grading its own installs: as a claim to verify, not a fact to inherit. If the vendor cannot guarantee the isolation or the freshness you are about to promise a customer, you cannot promise it either, unless you architect around the gap yourself.

Two more habits round this out. If humans label your data, moderation queues, quality raters, reviewers checking model output, that workflow has throughput and fairness implications, and it deserves the same measurement discipline as a funnel. And “we’ll clean the data later” is how a quarter disappears: data work is unglamorous, but it is serializing, and a model cannot compensate for a missing join or a poisoned label at scale.

Product implication: for any AI bet tied to a system you do not control, put a one-page contract in the initiative brief before the kickoff. If nobody will sign it, you have found your real bottleneck, and it was not the model.

Takeaways

Data readiness is a product requirement Six sections on why most AI initiatives die in a conference room, not a training run Hoshank Ailani · hoshank.vercel.app 1. Availability is a gate ASSUMED PROVABLE "we probably have logs" schema, retention, permission no join written down lineage: action to signal a hope an acceptance criterion "join ticket text to resolution within 24h for 95%" write that sentence in the PRD, or you're not ready InMobi close: written deadline took 5 days to 12 hours 2. Quality is fitness, not cleanliness Accuracy, completeness, timeliness Representativeness: does it mirror production One region trains a model that fails everywhere Fund a data audit as user research grade the audit with a number, a date, a verdict, the same way a launch review is graded sample rows with the people who know the domain, measure null rates and drift before you trust it 3. Pipeline to governance 1 Raw event lands 2 Validated and transformed 3 Checked against a governance rule 4 Available to the model step 3 is the one most roadmaps skip, and it's the one that turns into a launch delay or a headline Pixis data platform: 4h to 15min delay, 99.6% sync, 400+ customers 4. Labels are policy, leakage is your problem "churn risk" defined per team refuses to reconcile Facebook / facebook / FaceBook three rows, one campaign brand rules enforced upstream 80% less QA load a label without a version number is a UTM value Leakage: the model looks brilliant offline and collapses in production, because it never learned what you thought it was learning 5. The cold start is a product bet $800K ARR in two months, four brands, one two-week sprint Instrumented goal: a published on-brand ad Ignored entirely: logins measuring logins that quarter would have called it a win in week one and proven nothing about the wedge 6. The one-page data contract Owner, named, one Slack handle Refresh cadence, written down Required fields, enumerated Null rate threshold and escalation path for data you don't own, verify the vendor's claim, never inherit it, the way ad-tech grades an MMP if nobody will sign the contract, that's your bottleneck Most AI initiatives don't die in a training run. They die in a conference room, the day someone finally admits the data was never there. Full post: hoshank.vercel.app/posts/ai-track-data-as-product-requirement Takeaways
The whole post on one card. Save it, share it, argue with it. PNG ↓

Final Words

Data readiness belongs on the same page as scope, not buried inside “model tuning” where it becomes a fiction on the roadmap and a burnout risk on the team. Take the last AI feature your team shipped, write the acceptance criterion it should have had, and check whether anyone could actually sign it.

Reply and tell me what you found. I am collecting the misses for a follow-up on how delivery changes once the spec is statistical: experiments, evaluation, build versus buy, and partnering well with ML engineers.

Cheers, Hoshank