Complexities in Defining Retention Rate

Common terminology when measuring user retention in mobile is “day N” or dN retention. For example: d1 or d2 or d7 retention. N refers to the number of days after a user has installed the app. But this metric can be misleading, as the retention rate can mean different things:

  1. % of users that came back to your app within N days of installing.
  2. % of users that came back to your app on the Nth day.
  3. % of users that came back to your app on the Nth day or any day after.

Each of these definitions has its own interpretations. The important thing is to know exactly what you are measuring and what it means for you. Let us take Flurry’s way of measuring retention as defined here: Rolling Retention. That will likely be a common number quoted for mobile apps.

It uses the 3rd definition from the list above.

And that is almost as much of a vanity metric as the number of installs.

dN retention is not the same as engagement or activity. Using dN retention to answer “what % of users are active” is very, very wrong. For instance, a user may not have used the app at all after installing it, but if a new upgrade 10 days later makes the user open the app again, d1-d9 retention will include that user. This is by definition, as the user was active “day 1 or later”, “day 2 or later”, etc. However, they were not actually active on those days. This can be misleading, because now the d7 retention (for example) will be a higher number and will actually keep on increasing with time.

This metric can really tell you only ONE thing: how many users have not returned to the app at all after dN.

So should you use some other definition?

The general answer to this question is that the definition you should choose depends on what questions you want the metric to answer. Certainly don’t use retention when you mean to refer to some form of engagement.