Appray has no analytics. Here's how I know what to fix anyway.

The pledge that boxed me in

Appray's whole pitch is that nothing leaves your device. No sign in, no account, no server anywhere holding your answers. That line is on the App Store page and I mean it literally, not in the "we only collect anonymised, aggregated, de-identified usage data" sense that most privacy policies mean it. An analytics SDK, even a privacy-friendly one, is a thing that sends events off the phone. The moment I add one, the pledge becomes a sentence I have to start qualifying, and a decision-making tool that quietly tracks you while telling you to be honest with yourself is the kind of thing I'd mock in one of my own blog posts.

So the choice wasn't really "should I add analytics", it was "do I mean what I wrote on the App Store page or not". I meant it. Which left me building a product with genuinely no idea how it is used, and that part I underestimated going in.

What Apple still hands me, for free

It is not total darkness. App Store Connect gives me aggregate numbers without a single line of tracking code of mine running on anyone's phone: impressions, downloads, conversion rate, how many people updated, and crash counts bucketed by OS version. In a typical week that's something like three hundred to four hundred downloads and one or two crash reports, with zero way to connect either number to a person or a session.

That's enough to know the app is being found and isn't obviously on fire. It is nowhere near enough to know whether the thing I shipped last Tuesday is actually helping anyone decide anything, and that gap is where the real work is.

The reminder nobody could tell me was firing

When Appray gives you a Prove verdict, it offers to set a local notification two weeks out, a nudge to go check whether your cheap test actually happened. I built it, shipped it, and then had no way to find out if it worked. Local notifications on iOS are reliable in theory and occasionally silent in practice, especially after an OS update, and I had nothing watching for that failure mode because watching for it would have meant sending data somewhere.

For four months I genuinely did not know. What finally told me something wasn't a report, it was a batch of ordinary support emails. Whenever someone wrote in about anything else, a translation question, a receipt, a bug, I started adding one line at the end: did the two week reminder ever show up for you? I asked fifteen people that way over about six weeks. Nine said yes, it fired, two had notifications off entirely and never expected one, and the rest didn't answer. Nine out of fifteen on a question nobody set out to ask isn't a study. It was enough to stop assuming the feature was broken and move on to the next thing that actually needed me.

The three reviews that nearly cost me a round

Here's where I got it wrong in a way that worries me more than the reminder did. Three App Store reviews, out of around forty at the time, mentioned that the interview felt long. I read all three on the same evening, decided they were obviously right, and spent two days roughing out a version with one of the eight rounds removed.

I stopped because of a question that had nothing to do with those three people: who doesn't show up in a review at all? Someone who quits on round four never leaves a star rating and never emails support, because from where they're standing nothing went wrong, they just closed an app. The forty reviews I had were entirely people who finished, and three of forty finishers complaining about length tells me almost nothing about the people who never finished for a reason I'd be cutting blind. I had the vocal, self-selected minority confused for the whole picture, which is exactly the trap I'd written about doing to other people in an earlier post and then walked straight into myself three months later.

What I did instead, slower and less satisfying: I asked eleven people in a small iOS testing Slack to go through the interview once, on camera on their own machines, and just narrate out loud. Four of eleven stalled at the same round, not the one the reviews mentioned. I shortened that round's wording instead of removing anything, and left the length alone.

What I'd still choose

None of this is faster than a dashboard. A dashboard would have told me the reminder's fire rate in an afternoon, not six weeks of stray questions tacked onto unrelated emails. It would have shown me the real drop-off round directly instead of needing eleven volunteers and a shared screen. I know exactly what I'm giving up.

What I get back is that when someone emails asking whether Appray tracks anything, the answer has no footnote. Not a single event, not a single identifier, nothing. For an app whose entire job is helping you be honest with yourself about what you're building, that answer being completely true felt worth the extra six weeks, both times.