The One Metric I Actually Check Before Deciding If a Product Is Worth Keeping Alive
I've written about the apps I've killed. This is the actual test I run before I get there, the single number that tells me more than revenue, downloads, or reviews combined.
I've written before about the apps I've quietly killed. What I haven't written about is the actual test I run before I get anywhere near that decision. It's not revenue, it's not download count, and it's not app store rating. It's a single number: what share of active users are still opening the app 90 days after their first session.
Why This Number and Not the Obvious Ones
Revenue tells you what's happening now, not what's coming. A product can have healthy revenue today from a shrinking base of long-time users while quietly failing to hold onto anyone new, and revenue alone won't show you that until it's already declining. Download count measures interest at the door, not whether anyone stayed. App store rating is skewed toward people motivated enough to leave a review, which is a small and unrepresentative slice of the actual user base.
90-day retention answers a more honest question: once the novelty wears off, does this thing keep earning a place on someone's phone. That's closer to the real test of whether a product is actually solving a problem, versus solving a problem well enough to get installed once.
What the Number Actually Tells Me
A high number means the core idea is validated, even if growth is slow. If people who try the product keep coming back three months later, the fundamental value proposition works. Growth might be a marketing problem, a distribution problem, a positioning problem, all fixable. But the hard part, does this actually deliver ongoing value, is already answered.
A low number means I have a much harder problem, regardless of what else looks good. A product can have decent downloads, decent short-term reviews, and even decent early revenue while still failing this test, because none of those measure whether the value holds up past the first impression. A low 90-day number means people are trying it and not finding a reason to stay, which no amount of top-of-funnel improvement actually fixes.
Why I Don't Use a Shorter Window
A 7-day or 30-day retention number mostly measures onboarding and initial impression, real things worth tracking, but not the actual question I'm asking. Plenty of products hold people for a month on momentum and habit before the actual value proposition gets tested. 90 days is long enough that novelty has fully worn off and habit alone isn't enough to explain someone still using it. What's left at that point is closer to the real signal.
What I Actually Do With the Number
Below a certain threshold, specific to what the product is trying to be, I don't immediately kill it, but I stop investing in growth and start investigating why. Sometimes that's a fixable retention problem: a feature that should exist and doesn't, a workflow that's more annoying than it needs to be. Sometimes the investigation confirms there isn't a fixable problem, the core idea doesn't hold up to sustained use, and that's when a product actually ends up on the quietly-killed list.
Above the threshold, I treat the product as validated at the retention level regardless of how the other metrics look, and the actual work becomes getting more people to it, not questioning whether it deserves to exist.
The Bottom Line
Revenue, downloads, and reviews all answer questions I care about, but none of them answer the one that actually determines whether a product deserves continued investment: does it hold up past the first impression. 90-day retention is the closest single number I've found to that answer, and it's the one I check before any of the others get to weigh in on a keep-or-kill decision.