Evaluating Retention: Advice for (Early-Stage Start-up) Diligence

September 2024

Over the past quarter, I’ve reviewed 40+ SaaS start-up pitch decks and pitches as a mentor and in support of investors in initial diligence ahead of term sheet negotiation.

Founders, understandably, want to stand out by demonstrating early traction in a tough fundraising market. Many focus on showcasing 30-day customer retention, which can be calculated even for early-stage start-ups – as early as Beta users and even for freemium products. But there may be holes in the traction story!

Here are some questions to discuss with the founder to ensure you get a more accurate picture of what is actually happening with their customers and the business, which will help you make a more informed investment decision.

Statement of the Problem

Typically, a 30-day retention plot looks like the one above; it is based on simulated data. The plot shows that:

On day 1, the cohort has 100% of the customers.
Over time, day by day, some of those customers leave the product, and the retention drops off.

In this example, after a week, we’ve retained about 80% of the cohort; at 30 days, we’re at about 20%.

The problem is not that they haven't yet reached the target of 85%+ monthly retention rate for SaaS. Understandably, early-stage start-ups haven’t reached product-market fit yet and are still trying to listen to customers and optimize the experience so that they can get there in the future.

The problem is that this plot only tells part of the story! Key information is missing, which we should ask for and discuss in follow-up conversations to better evaluate the potential for a product-market fit of a SaaS product. Here’s what to look for:

The last 30 days – the typical focus of a retention plot for most start-ups – could have just been a fluke! Maybe the team spent some serious money on ads to get there, which is unsustainable. And we have no idea who these customers are, what got them in the door, what keeps them around and why, or how long they plan to stay.

To better understand the start-up’s customers and evaluate the potential market opportunity of the product, in your next meeting with the founder, spend 5-10 minutes digging into the questions below to help you get there.

Evaluating the Retention Definitions: Who + What

Few pitches (and pitch deck) define their thoughts about active and/or retained users, even in the Appendix. Here are questions to ask to understand the founders’ thought process:

How does the company think about and define retention? What action (or set of actions) characterizes a customer's activity on the platform?
- - Is 1 log-in per month enough?
  - Is there an action the customer needs to take, such as a comment/like that month? Or do they need to make a purchase?
  - Or is there something else that month that the customer needs to do?
Does the 30-day window make sense for the industry and the product offering?
- - Or is weekly retention more informative, for instance, when the product teaches a new skill?
  - Or does quarterly retention make more sense, for instance, with retail/fashion-related product purchases?
  - Or should retention be calculated annually if the product focuses on adherence to annual physical exams, for example?
  - Or is there another retention window that makes more sense for the industry and the product offering?

These discussions will help you evaluate the founder-product fit, as this should shed light on the depth of the team’s industry and customer knowledge.

Evaluating the Retention Story: Why + How

Has the MVP been around for more than 30 days? If so, can the founder share the cohort chart with you for all cohorts over the last year (or since inception, if the MVP has not been around for a while)?

It should look something like the plot below [ref], with the percent of customers (or number) on the y-axis and days (or weeks or another time period) on the x-axis.

Ideally, the founders are already examining this plot (or something similar) to gauge how well the product resonates with customers over time and can explain away any “weirdness/irregularities” with product/tech updates.

Here’s what to look for and why that chart matters:

How quickly were the founders able to share this plot with you?
- - This can help you see if customer retention is top-of-mind for them – and if this is something that they are/aren’t tracking in real-time.
Where are the early cohorts now, after so much time has passed? Are any of them retained past the 30-day mark? Past the 6-month mark? Past the 2-year mark (if the product has been around that long)?
- - This can help you see how deeply rooted the product is in solving its customers' pain points over time.
Is retention decreasing down to zero? Or is it leveling off at a point above that over time?
- - This will help you evaluate if the product solves a real pain for a subset of the customer base, which can become the target population to retain and expand in product-market fit efforts.
Note how long it takes for retention to level off. Discuss initiatives the founders are taking to improve retention and the customer experience, reducing friction and making the product more sticky.
- - What is the “aha” moment for these customers about the product?
What Marketing/Sales/other efforts were going on during this time period to get customers in the door and engaged?
- - How much did this cost?

These conversations should help you better understand the size of the top-of-funnel required to reach the loyal customer base for whom the product solves a real pain point.

Evaluating the Company’s Growth Potential: Scale

Now that we have a better idea of the start-up's best customers and how to activate them, we need to evaluate how realistic the company's growth plans are. Here are some evaluation points:

Given what we know about the target customer segment, are the financial projects tied to its expansion realistic?
- - What needs to be true – or needs to happen – to have the start-up reach the target projections? How realistic is this?
How is the target customer base aligned with the monetization/pricing strategy and network effects to expansion and revenue?
How does the retention/monetization/pricing strategy affect the milestones the company needs to hit this round and prepare for the next rounds’ milestones and/or potential exits?
- - Is there a target retention metric (or similar) that the start-up is trying to hit?

The answers to these questions will help you see the extent to which this can become a venture-scalable business.

Summary

Even inadvertently, charts can lie with incomplete information! We shared with you what questions you may want to consider asking as part of an initial or confirmatory diligence to understand how the company thinks about uncovering, engaging, and monetizing its core customers, which will help you evaluate the venture-scale potential of the business.

Good luck!