TL;DR – Yes, by a minimum of 54.4%
The Hindu is a 144 year old English newspaper known for its editorial independence, integrity and a strict adherence to its code of conduct. Started in 1878, it is the oldest English language newspaper founded by Indians. All its contemporary English newspapers which continue to publish even today were previously owned by Britishers. The Hindu was integral to the Indian Independence movement and played a large role in the shaping up of India as an independent democratic country. Even today, The Hindu continues to be an indispensable pillar of Indian democracy by ensuring accountability of persons in positions of power through its in-depth reporting, editorial scrutiny, providing a platform for public discourse and fostering informed citizenry.
Even on the technology frontier, The Hindu Group has been at the forefront of digital adoption in the newspaper industry. For instance, The Hindu was the first newspaper in India to have launched a website in 1995. Again in 2018, The Hindu was the first major publication in India to go behind a paywall and transform itself from an advertising-led to a subscription-led business.
Naturally, I was excited to join such a prestigious organisation with a rich history as well as a forward looking culture. I joined The Hindu in March 2022 as a Product Manager for Subscriber Retention. In other words, it was my responsibility to ensure subscriber churn rate is reduced and users continue to renew their subscription. In this post, I want to outline one of the key experiments I conducted a few months ago in March which had a significant impact on The Hindu’s subscriber renewal rate. While there are numerous projects I have worked to impact this metric, in this article I want to talk about the primary driver which had the most profound impact on subscriber churn i.e. engagement.
There are numerous case studies on multiple news organisations which state that retention is a function of and directly correlated to engagement. The more engaged users are with your content, the more likely they are to renew their subscriptions. On these lines, I wanted to test the above hypothesis for The Hindu and also quantify its impact, if any. In other words, I wanted to know the answer to the question – Does engagement have an effect on subscriber renewal? If so, by how much?
The Experiment
Methodology
In order to find the answer to the above question, I decided to run an A/B test. In a nutshell, an A/B test involves running two (or more) different variants and determining which variant performs the best. This HBR article provides a great introduction for A/B Testing. For my A/B test, I decided to take all users whose subscription was due for expiry in the month of March as my total sample for the experiment. I then decided to split this sample randomly into 2 equal groups – Group A (50%) and Group B (50%). Group A received regular engagement communication from The Hindu while Group B received no engagement communication. This experiment ran for the entire month of March (31 days).
Group A was sent communication via emails. There are multiple ways in which a user can be reached out to and communicated with. However, for the purpose of this experiment I decided to use only email since that is the closest to a web reading experience and supports long form stories. The type of engagement communication included varied content such as daily news briefings, subscriber exclusive stories, topics from The Hindu Explains and manually curated digests of various trending topics.
Group B received no communication.
Hypothesis
Renewal rate of users with higher engagement would be higher than that of the users who did not receive engagement communication under this initiative.
Success Metrics
Comparing the renewal rate of subscribers in: Group A vs Group B
For the above hypothesis to be true, the subscriber renewal rate of Group A would have to be higher than that of group B.
Results
Since all engagement communication for this experiment was sent via emails, it is likely that not all users would have opened the emails. Thus, to have a fair comparison of the renewal rates in two groups, I decided to further segment Group A into users who have opened at least 1 email.
Following are the results of this experiment
| Group | Renewal Rate |
|---|---|
| Group A | |
| Opened ≥ 1 email | 24.7% |
| Opened ≥ 3 email | 29.8% |
| Opened ≥ 8 email | 35.3% |
| Group B | 16% |
Clearly, the group targeted with engagement communication (Group A) had higher renewal rates than the group that received no engagement communication (Group B).
Let us take Group B renewal rate (16%) as the base renewal rate. This is because the renewal rate of Group B represents those subscribers who are likely to renew their subscription even without any engagement communication. Using this, we can quantify the impact of engagement on subscriber renewal.
| Group A | Renewal Rate (1) | Base Renewal Rate (2) | Increase (3) [ (1) – (2) ] | % Increase (4) [ (3) ÷ (2) X 100 ] |
|---|---|---|---|---|
| Opened ≥ 1 email | 24.7% | 16% | 8.7% | 54.4% |
| Opened ≥ 3 email | 29.8% | 16% | 13.8% | 86.3% |
| Opened ≥ 8 email | 35.3% | 16% | 19.3% | 120.6% |
Findings
Three key findings emerge from this experiment:
- The hypothesis holds true that users who engage more are more likely to renew their subscription than the ones who do not.
- Even if a user viewed just one engagement email in their month of expiry, they were 54% more likely to renew than a user who was not exposed to any engagement communication.
- The renewal rate shows a direct positive correlation with the level engagement as evidenced in the above table. For instance, users who opened ≥ 8 emails had a much higher renewal rate than users who opened ≥ 1 email.
Next Steps
This experiment was a huge revelation in terms of the quantum of impact user engagement can have on a subscriber’s likelihood to renew. There are plenty of actionables that can be worked upon to maximise user engagement and its impact on retention. Here are the top 3 action items as immediate next steps which would help maximise the user engagement:
- Tap into other communication channels:
For the purpose of this experiment only email was used as the channel of communication. Email however, comes with its limitations such as messages being classified incorrectly by email service providers, being marked as spam or multiple other reasons which keep the user from being exposed to the content. Thus other mediums such as WhatsApp and Telegram must also be utilised especially in a country like India where the usage of these channels is the highest in the world.
- Tailor content offerings:
For the purpose of this experiment the engagement pieces chosen were rather generic and mass appealing which could be targeted to a wide range of users. However, the main advantage of the internet economy lies in offering the long tail content pieces based on user preferences. Thus, the next step would be to build more segmented groups with interest based content targeting.
- Integrating engagement into multiple customer journey touchpoints:
User engagement must be treated as an ongoing activity and in order to reap its maximum benefits should be well integrated at multiple touch points of the user journey such as onboarding, activation, reactivation of inactive users, users taking a free trial etc.