How to prove a feature before you build it

I built Appray specifically to stop myself skipping this step, so I have thought about it more than is reasonable for a small app that only about a dozen people use every day. Here is what I have actually learned about proving a feature, as opposed to just saying the word "prove" and moving on.

The fake door I built and then didn't like

Early on I added a button to a test build that said "Sync across devices" and did nothing when you tapped it except log the tap. This is a real, well documented technique. You get a number without spending the three weeks a login system and a server would cost.

It worked, in the sense that it produced a number. Eleven taps out of nineteen testers over three weeks, which sounds like real interest until you remember that eleven of those people were tapping every button in a small test build because that is what testers do. What it did not produce was a single message afterwards from anyone asking where the sync feature had gone, or annoyed that it didn't work, or curious about it at all.

I stopped doing it after that round, not because it failed statistically but because it felt bad to use. Someone taps a button in good faith and gets nothing back but a silent log line. That's a small lie, told to a person who trusted the app enough to try it, and I did not want Appray to be built on small lies even in service of a genuinely useful number. Other people running fake door tests will tell you it's standard practice and mostly harmless, and they are probably right about the harm. I just didn't like doing it to someone, so I don't.

What I use instead

Two things, and neither of them is clever.

The first is counting unprompted asks over time rather than clicks in the moment. If three different people mention the same underlying want, in their own words, without me asking a leading question, that counts for more than fifty responses to a survey I wrote myself. A survey answer is shaped by the question. An unprompted email is shaped by nothing but the actual problem.

The second is asking for something that costs the person a little, right now, in exchange for the feature existing. Not money necessarily, though that's the strongest version. A real email address on a specific waiting list works. So does asking someone to describe the exact moment, in the past, when they needed the thing. If they can name a Tuesday and what they were doing, that's real. If they describe a general future where it would obviously come in handy, it isn't, no matter how sincerely they mean it. People are honest about the future in a way that has almost nothing to do with what they will actually do.

The two week rule

Prove has a failure mode that Kill doesn't have, which is that it can sit there forever looking like progress. Kill is at least final. Prove, left alone, quietly turns into "we're thinking about it" for a year, and a feature under permanent consideration is doing more damage to a roadmap than one that got killed cleanly, because it keeps getting brought up in every planning conversation without anyone having to defend keeping it there.

So I gave myself a rule: if a feature has sat in Prove for two weeks and I haven't actually done anything to gather evidence in that time, meaning I haven't sent an email, checked a number, or asked a single person a direct question, it moves to Kill by default. Not because it's necessarily a bad idea. Because a verdict I'm not acting on isn't a verdict, it's a feeling I've dressed up to look like a process, and the two week limit forces me to notice when that's happened.

The one that actually passed

The clearest Prove-to-Build I've had came from a PDF export request. Two people asked for it in the same week, in completely different wording. One wanted to "keep a copy" of their answers, the other wanted to "send this to my co-founder." Different words, same underlying fear: that the work would disappear or stay trapped on one phone. I built the export in three days. Nobody has asked for cloud sync since, which tells me the export solved the actual problem rather than a smaller version of it.

Compare that to the fake door number. Eleven taps told me nothing I could act on. Two independent emails, four days apart, told me exactly what to build and roughly how urgent it was.

Where this leaves "prove it"

None of this is rigorous. I'm not running controlled experiments on nineteen testers, and I'd distrust anyone at my size who claimed they were. What I have is a habit: don't fake interest to measure it, count the asks that arrive on their own, ask for something small and real in return for attention, and give myself a deadline before "prove it" quietly becomes "never mind." It has been right more often than my gut alone, which is the only claim I'm actually making for it.