skip to content

Search

Metrics and Measuring Success

Less than one minute read Updated:

Measure what changes decisions: a tight hierarchy from north star to inputs and guardrails, with hypotheses written before build and honest reviews after ship.

# product-management
# strategy
# leadership
# metrics
# outcomes
Product Management Foundations

If a number on your dashboard has never changed a decision, delete the tile.

I have sat in launch reviews at three companies. Supply-chain platforms at Dell. An AI creative platform at Pixis. The data layer under InMobi’s ad exchange. The same slide shows up every time. A dashboard, full of charts. Nobody in the room asks what the team would do differently if any one of those numbers moved.

At InMobi I own the system of record under a demand-side platform and exchange that measure ad events from 200M+ devices a day. Getting the numbers right there comes down to which four or five of them the team is actually allowed to act on, and that is a much shorter list than the dashboard suggests.

So that’s today’s post.

Today’s Post

Six pieces of a metric system that survives contact with a real launch review.

  • Why a metric that never changes a decision is decoration
  • Leading indicators move first, lagging indicators confirm the story weeks later
  • A north star metric needs inputs and guardrails, or it is a slogan
  • Write the hypothesis before you build, or you retrofit one after
  • Same word, different denominator: why definitions and segmentation matter
  • Kill the metrics nobody acts on, and give the survivors an owner

1. A metric that never changes a decision is decoration

Teams collect dashboards because dashboards look like discipline. Ask the harder question instead. What would you do differently if this number moved?

If the honest answer is “worry about it in Slack,” you have a mood ring.

Decision-linked metric: one that maps to a lever someone on the team can actually pull. Copy, flow, pricing, latency, a support policy. Start from the decision, then instrument. Most teams do it backward. They instrument first and hope a decision falls out later.

At InMobi, that discipline is the difference between a chart finance glances at and the close date finance actually holds the team to. Everything else on that dashboard is wallpaper.

2. Leading indicators move first, lagging indicators confirm weeks later

Lagging indicators are the outcomes you actually care about: revenue, retention, churn. They move slowly and blend a dozen causes at once. Judged daily, they lie.

Leading indicators are the behaviors that come before those outcomes: an activation step, a time-to-value number, how deep someone uses a core workflow. They are noisier, but they respond to what shipped this week.

At Pixis, the leading indicator I built the whole Data Platform around was freshness. How long between an ad event happening and it showing up in a report. We cut that from 4 hours to under 15 minutes for 400+ enterprise customers. The lagging indicator downstream, whether Advance’s bid and budget recommendations were worth acting on, could only be trusted once the leading one was fixed.

The lagging number confirms weeks later Freshness moves first in week 1. Trust in Advance's recommendations only catches up by week 6. Scroll slowly, this plays once. Grey monospace text is the concrete number at each step. week 0 week 1 week 6 week 7 Baseline Freshness lag before the fix event to report: 4h Leading indicator moves Data freshness fixed, week 1 4h → under 15min weeks pass Lagging indicator confirms Trust in Advance's bid and budget calls climbs week 6, 400+ customers Review fires Only now is the hypothesis judged week 7 decision leading indicator, moves this week lagging indicator, confirms it late Judging the lagging number in week 1 would have called the hypothesis wrong before it had time to be right.
Judge the lagging number in week 1 and you'd call the hypothesis wrong before it had time to be right. (animates as you scroll to it) PNG ↓

Name the pair before you ship. “If freshness drops under 15 minutes, recommendation trust climbs within six weeks” is a sentence you can be wrong about. That is the point of writing it down.

3. A north star metric needs inputs and guardrails, or it is a slogan

A north star metric is supposed to focus a team. In practice it often becomes the only thing anyone looks at, and a single number can’t carry a product’s whole health by itself.

Pair it with two things. Inputs are the handful of drivers the team believes move the north star, each with an owner who feels it when the number slips. Guardrails are what the team refuses to sacrifice for it.

At Pixis, AdRoom’s north star was brands self-serving live, on-brand creative. The input we actually instrumented was activation inside a two-week onboarding sprint, and we ignored logins entirely. Four brands through that sprint produced $800K ARR in two months, on the way to $0 to $10M ARR in five months with a 20-person team. The guardrail underneath the whole motion was brand-guideline enforcement at generation time, before anything published. Fast never got to mean off-brand.

A north star needs inputs and a guardrail AdRoom's stack: one number the team focuses on, the drivers that move it, and the line it won't cross. Widest at the bottom, on purpose: the guardrail applies to every layer above it. North star Brands self-serving live, on-brand creative $0 to $10M ARR in 5 months, 20-person team Input: onboarding activation Four brands through a two-week sprint owner: onboarding PM · $800K ARR in 2mo Ignored: logins Deliberately not instrumented as an input owner: none, by design Guardrail: brand-guideline enforcement at generation time checked before anything publishes, so fast never means off-brand
Drop the guardrail and the north star becomes a number you can win by cheating. Drop the inputs and it stays a slogan. PNG ↓

Drop the guardrail and the north star becomes a number you can win by cheating. Drop the inputs and it stays a slogan on a wall.

4. Write the hypothesis before you build, or you retrofit one after

The failure mode is familiar. Ship first, then hunt for a chart that went up. Anything found that way is a story you told yourself after the fact.

Write the hypothesis before design locks. Name the user, the behavior, the primary metric, the guardrail, the time horizon, and the minimum size of change that would actually change your next decision.

Suppose you own workspace admin tools on a Business plan. A hypothesis worth writing looks like this: inline role editing will cut time-to-first-invite by 20% within four weeks for admins on that plan, with no rise in permission-related support tickets. Five nouns, one sentence, and it can fail.

If you can’t write that paragraph, you are not ready to argue for the build. What happens after you ship it, the review, the record, the decision, is a separate discipline, and it only works if this paragraph existed first.

5. Same word, different denominator

“Retention” is not a metric until you name the event, the population, and the window. Two teams can both report “retention” and both be technically right, and still mean nothing to each other in the same meeting.

Averages make this worse. A flat conversion rate can hide a mobile funnel collapsing while desktop improves next to it. At Pixis, Advance served 400+ enterprise customers, each running its own segmentation logic on top of the platform. Treating an enterprise account and a self-serve trial as one activation curve would have buried real enterprise churn under the much larger self-serve numbers, because the two cohorts move at entirely different speeds.

One average, two very different stories Advance's activation curve reads flat blended together, and moves in opposite directions apart. Same six weeks, two panels. Left is what the dashboard showed; right is what it hid. Blended: 400+ accounts, one line activation rate reads flat, week 1 to week 6 no reason to look closer Segmented: enterprise vs self-serve enterprise: churning underneath self-serve: improving on top self-serve trial, moving up enterprise account, moving down, buried by volume
The average told a flat story. Split by segment, one line was climbing and the other was quietly churning underneath it. PNG ↓

Default your dashboards to the segments that match how the business actually runs, even when a different split would be easier to query.

6. Kill the metrics nobody acts on, and give the survivors an owner

Dashboards accrue cruft the same way a codebase does. A tile nobody has acted on in two quarters is either misunderstood, unowned, or dead. Retire it out loud so people stop arguing about a ghost.

Every metric that survives needs a name attached to it: who wakes up when it regresses, and which meeting has the power to reallocate work because of it. A number with no owner turns into wallpaper within a quarter, no matter how carefully someone defined it on the day it was built.

Takeaways

A metric earns its place by changing a decision Five panels on building a metric system a launch review can actually trust Hoshank Ailani · hoshank.vercel.app 1. North star, inputs, guardrail North star: brands live, on-brand Input: onboarding activation owner: onboarding PM Guardrail: brand check before publish fast never means off-brand $800K ARR in 2 months, 4 brands, one sprint 2. Leading moves, lagging confirms Leading: freshness, moves this week Lagging: recommendation trust, confirms late Name the pair before you ship Judging the lagging number early calls it wrong too soon 4h → 15min freshness fix, trust followed by week 6 3. Write the hypothesis first 1 User, behavior, metric, guardrail, horizon 2 Name the minimum change that moves a decision 3 Write it before design locks, not after ship a sentence you can be wrong about ship first, hunt for a chart later: a story, not a result 4. Segment before you average BLENDED SEGMENTED one flat line, 400+ accounts two lines, opposite ways enterprise churn buried underneath visible, named retention needs an event, a population, a window named 5. Give every survivor an owner A tile nobody has acted on in two quarters is dead Retire it out loud so people stop arguing about a ghost Every survivor names who wakes up when it regresses And which meeting can reallocate work because of it 1 quarter how long an unowned metric takes to turn into wallpaper If a number has never changed a decision, delete the tile. Pick one dashboard your team looks at weekly and write down the decision each tile informs. Full post: hoshank.vercel.app/posts/pm-track-metrics-and-measuring-success Takeaways
The whole post on one card. Save it, share it, argue with it. PNG ↓

Five panels, one per lever. Save the card, then check your own dashboard against it.

Final Words

Pick one dashboard your team looks at weekly. For every tile, write down the decision it is supposed to inform. Delete the ones you cannot answer for.

Reply and tell me what survived.

Cheers, Hoshank