Product analytics Flashcards
Common goals for product analytics
- Build the right products and features
- Reduce friction
- Increase revenue
- Drive more innovation
Types of product metrics
- Business outcomes
- Product usage
- Product quality
Business outcomes
How the product influences key business and financial outcomes.
Examples:
1. Net revenue retention
2. Churn
Product usage
How users behave inside the product, which features the users use the most
and here they are getting stuck
Examples:
1. Feature adoption
2. Stickiness
How the product is doing what it supposes to do
- Response time
- Number of bugs per feature
Lagging indicators
Metrics that you want to move for the health of the business, but may be harder to see results in a short timeframe
Leading indicators
Metrics that are measurable in a shorter time frame, and have a high probability of affecting your lagging indicators
OKRs
Objectives and Key results
‘What are we focused on right now?’
Consists of an objective and three to five supporting key results that map to it.
KPIs
Key performance indicators
‘Are we ok?’
A longer-term view of performance, and typically a set of numbers that you can consistently watch over time
One Metric That Matters
‘Does what we are working on contribute to this one metric?’
Focus on a single metric that matters the most for the success of the business at different stages
North Star Metric
‘Does what we are working on contribute to our primary metric?’
A single metric that is an indicator of achieving user value over a period of time
Check metrics (Balance metrics)
‘Will focusing too much on this one metric have drawbacks?’
Metrics you track to ensure you aren’t over-indexing on any one metric — for example, your primary North Star — and causing unintended negative consequences
Product analytics strategy includes
- The questions you are asking
- The goals you’re working towards
- The metrics you use
Major goal of product analytics
Use quantitative data to identify people from whom you then want to collect qualitative data
Breadth
A measure of how many total users you have, as well as how many users you have per account
Depth
How much of the product your customers are using — specifically how many key features or areas of the product they’re utilizing
Frequency
Measures how often users — or users from a specific account — log in to your product in a given timeframe
Product Analytics
Hierarchy of Needs
- Collect data
- Refine data into metrics
- Build metrics into reports
- Take action
- Data Actualization
Aha moments
When users recognize the distinct value of your product and become committed to a long haul
Popular aha moments
- Facebook: when a user connects with 7 friends in first 10 days
- Slack: When teams send 2000 messages
- Dropbox: when a user saves 1 file into 1 folder at a 1 device
How to take action on feature adoption data
Tactic 1: Increase the visibility of the feature
Tactic 2: Improve adoption so more users are aware of the feature
Tactic 3: Make improvements to the feature
Friction
Anything that gets in the way of a user’s ability to achieve their objective or job to be done
Examples of frictions
- Interface copy that doesn’t make sense
- Pages where the next action is not obvious
Signs of friction
- Users dropping out of workflows
- Low usage of key features
Users dropping out of workflows
- Use funnels to see how customers move through your product
- Use session replay to review a visitor’s journey in your product and see where and how they’re dropping off
- Support tickets topics
Low usage of key features
Solicit feedback from users to understand why usage is lower or what’s causing them to experience friction
Benefits of sunsets
- More streamlined user experience
- Increased efficiency
Types of business cases
- Investing to usability (metric — stickiness)
- Investing in innovation
- Where to focus adoption efforts
Key questions for a business case
- Who is the target audience
- What pain is this addressing
- What is the desired outcome
Outcomes to target
- Revenue: how will this idea help increase revenue by affecting customer acquisition or retention?
- Cost: how does this recommendation reduce cost by adding new efficiencies?
- Risk: how does the idea mitigates business, economics, or market risks?
AARRR metrics pyramid
Acquisition, Activation, Retention, Revenue, Referral
+ Engagement
Acquisition Metrics short
1 Bounce Rate
1.2 Conversion Rate
1.3 Landing Page Conversion Rate
1.4 Cost of Customer Acquisition (CAC)
1.5 Channel Effectiveness
1.6 Traffic Source Distribution
Acquisition metrics explained
1 Bounce Rate
The percentage of visitors who leave your website after viewing just one page. A high bounce rate may indicate issues with the landing page (e.g., messaging) or targeting.
1.2 Conversion Rate
The percentage of users who take a desired action, like signing up for a newsletter.
1.3 Landing Page Conversion Rate
The percentage of visitors who take a desired action on a specific landing page, like signing up or starting a trial, on a specific landing page.
1.4 Cost of Customer Acquisition (CAC)
The cost of acquiring a new customer through marketing and sales efforts.
1.5 Channel Effectiveness
The success of each acquisition channel in driving traffic, sign-ups, or purchases.
1.6 Traffic Source Distribution
The breakdown of incoming user traffic by different sources, such as organic search, referrals, or paid ads.
Activation metrics (short)
2.1 Time to Value (TTV)
2.2 Onboarding Completion Rate
2.3 User Activation Rate
2.4 Trial-to-Paid Conversion Rate
2.5 First-time User Conversion Rate
2.6 Product Qualified Accounts (PQA)
2.7 Product Qualified Leads (PQL)
Activation metrics (detailed)
2.1 Time to Value (TTV)
The time it takes for a user to experience the core benefits of your product after starting to use it. A shorter TTV leads to higher user satisfaction, engagement, and retention. In product-led growth, optimizing TTV is crucial to ensure users quickly understand the value your product delivers.
2.2 Onboarding Completion Rate
The percentage of users who complete the onboarding process successfully.
2.3 User Activation Rate
The percentage of users who successfully complete a certain milestone in your onboarding process.
2.4 Trial-to-Paid Conversion Rate
The percentage of trial users who convert into paying customers.
2.5 First-time User Conversion Rate
The percentage of first-time users who complete a desired action, such as creating an account or purchasing. This metric helps assess the effectiveness of the onboarding process.
2.6 Product Qualified Accounts (PQA)
“In product-led sales, the product determines Product Qualified Accounts (PQA) to indicate when an account is prepared for sales engagement and potential conversion.” – Elena Verna, source
2.7 Product Qualified Leads (PQL)
“PQLs, or Product Qualified Leads, are the people within the existing self-serve user base with buying power.” – Elena Verna, source
Engagement metrics (short)
Engagement Metrics can be considered part of the Retention and, depending on the context, Activation (e.g., Session Length). I presented them as a separate category to emphasize the distinct metrics focusing on user interaction with the product.
3.1 Daily Active Users (DAU)
3.2 Monthly Active Users (MAU)
3.3 Stickiness
3.4 User Satisfaction (CSAT)
3.5 Session Length
3.6 Session Frequency
3.7 Feature Usage
3.8 Customer Effort Score (CES)
3.9 Task Success Rate
3.10 User Feedback Score
Engagement metrics (detailed)
Engagement Metrics can be considered part of the Retention and, depending on the context, Activation (e.g., Session Length). I presented them as a separate category to emphasize the distinct metrics focusing on user interaction with the product.
3.1 Daily Active Users (DAU)
The number of unique users who engage with the product daily.
3.2 Monthly Active Users (MAU)
The number of unique users who engage with the product monthly.
3.3 Stickiness
The ratio of daily active users (DAU) to monthly active users (MAU), which indicates how often users engage with the product.
3.4 User Satisfaction (CSAT)
A measure of how satisfied users are with the product, often determined through surveys or in-app feedback (e.g., Pendo, Gainsight).
3.5 Session Length
The duration of a user’s interaction with the product during a single session.
3.6 Session Frequency
The average number of sessions per user within a specific time frame.
3.7 Feature Usage
The frequency and depth of usage for specific product features.
3.8 Customer Effort Score (CES)
Measures the ease with which customers can interact with your product or service. It is often determined by asking users to rate the effort required to accomplish a task or resolve an issue on a scale from very low to very high effort.
A lower CES indicates a more user-friendly product, which can lead to higher user satisfaction and loyalty.
3.9 Task Success Rate
The percentage of users who successfully complete a specific task or set of tasks within your product. This metric helps assess the usability and effectiveness of your product’s features.
3.10 User Feedback Score
A quantitative measure of user satisfaction gathered through surveys, ratings, or reviews.
There isn’t a single standardized method or rating scale. This could be a numeric scale (e.g., 1 to 5 or 1 to 10), a star rating, or a qualitative scale (e.g., poor, average, excellent).
Retention metrics (short)
4.1 Churn Rate
4.2 User Retention Rate
4.3 User Renewal Rate
4.4 Customer Lifetime
4.5 Customer Health Score
4.6 Product Adoption Rate
Retention metrics (detailed)
4.1 Churn Rate
The percentage of users who stop using the product within a specific period, e.g., monthly.
4.2 User Retention Rate
The percentage of users who continue using the product after a specific period. Often monthly.
4.3 User Renewal Rate
The percentage of users who renew their subscription or continue using the product after their initial contract period.
4.4 Customer Lifetime
The average time it takes for a user to stop using the product.
4.5 Customer Health Score
A composite metric that combines multiple indicators, such as usage, satisfaction, and support interactions, to provide an overall assessment of the customer’s relationship with the product.
4.6 Product Adoption Rate
The percentage of users who adopt new features or functionality within a certain time frame after release.
Revenue metrics (short)
5.1 Average Revenue Per Account (ARPA)
The average revenue generated per account (customer) within a specific time frame. For example, monthly.
5.2 Customer Lifetime Value (CLV/LTV)
The total revenue a user generates during their entire relationship with the product.
5.3 Customer Profitability
The difference between the lifetime value of a customer (LTV) and the cost of acquiring them (CAC).
5.4 Monthly Recurring Revenue (MRR)
The predictable revenue generated by a subscription-based product every month.
5.5 Expansion Revenue
Additional revenue generated from existing customers through upsells, cross-sells, or add-on purchases.
5.6 Net Revenue Churn
The revenue lost due to customer cancellations, downgrades, or non-renewals within a specific period, typically a month/year.
5.7 Net Revenue Retention
The cumulative sum of retained, contracted, and expanded revenue over a specific period, typically a month or year.
5.8 Average Contract Value (ACV)
The average revenue generated from each customer contract, which can help assess the effectiveness of pricing and packaging strategies.
Revenue metrics (detailed)
5.1 Average Revenue Per Account (ARPA)
The average revenue generated per account (customer) within a specific time frame. For example, monthly.
5.2 Customer Lifetime Value (CLV/LTV)
The total revenue a user generates during their entire relationship with the product.
5.3 Customer Profitability
The difference between the lifetime value of a customer (LTV) and the cost of acquiring them (CAC).
5.4 Monthly Recurring Revenue (MRR)
The predictable revenue generated by a subscription-based product every month.
5.5 Expansion Revenue
Additional revenue generated from existing customers through upsells, cross-sells, or add-on purchases.
5.6 Net Revenue Churn
The revenue lost due to customer cancellations, downgrades, or non-renewals within a specific period, typically a month/year.
5.7 Net Revenue Retention
The cumulative sum of retained, contracted, and expanded revenue over a specific period, typically a month or year.
5.8 Average Contract Value (ACV)
The average revenue generated from each customer contract, which can help assess the effectiveness of pricing and packaging strategies.
Referral Metrics (short)
6.1 Virality Coefficient
The number of new users acquired through referrals by existing users. Often expressed as a ratio (<1, 1, >1).
6.2 Customer Referral Rate
The percentage of customers who refer others to the product.
6.3 Referral Conversion Rate
The percentage of referrals that convert into active users.
6.4 Net Promoter Score (NPS)
A measure of customer satisfaction and loyalty based on how likely users are to recommend the product to others.
Warning: NPS measures customer attitude and sentiment, not the actual behavior.
Referral metrics (detailed)
6.1 Virality Coefficient
The number of new users acquired through referrals by existing users. Often expressed as a ratio (<1, 1, >1).
6.2 Customer Referral Rate
The percentage of customers who refer others to the product.
6.3 Referral Conversion Rate
The percentage of referrals that convert into active users.
6.4 Net Promoter Score (NPS)
A measure of customer satisfaction and loyalty based on how likely users are to recommend the product to others.
Warning: NPS measures customer attitude and sentiment, not the actual behavior.
Activation metrics that matter
- Time to Value (TTV)
The time it takes for a user to experience the core benefits of your product after starting to use it.
- User Activation Rate
The percentage of users who successfully complete a certain milestone in your onboarding process.
Engagement metrics that matters
- DAU / MAU / WAU
- Stickiness (DAU/MAU)
- User satisfaction (surveys /in app feedback)
Retention metrics that matter
- User retention rate
The percentage of users who continue using the product after a specific period. Often monthly. - Product adoption rate
The percentage of users who adopt new features or functionality within a certain time frame after release.
Referral metrics that matter
- Virality Coefficient
The number of new users acquired through referrals by existing users. Often expressed as a ratio (<1, 1, >1). - Net Promoter Score (NPS)
A measure of customer satisfaction and loyalty based on how likely users are to recommend the product to others.
Warning: NPS measures customer attitude and sentiment, not the actual behavior.
