Product metrics Flashcards
- What would be good metrics of success for an advertising-driven consumer product? (Buzzfeed, YouTube, Google Search, etc.) A service-driven consumer product? (Uber, Flickr, Venmo, etc.)
advertising-driven: Pageviews and daily actives, CTR, CPC (cost per click)
click-ads
display-ads
service-driven: number of purchases, conversion rate
- What would be good metrics of success for a productiv- ity tool? (Evernote, Asana, Google Docs, etc.) A MOOC? (edX, Coursera, Udacity, etc.)
productivity tool: same as premium subscriptions
MOOC: same as premium subscriptions, completion rate
- What would be good metrics of success for an e-commerce product? (Etsy, Groupon, Birchbox, etc.) A subscrip- tion product? (Net ix, Birchbox, Hulu, etc.) Premium subscriptions? (OKCupid, LinkedIn, Spotify, etc.)
e-commerce: number of purchases, conversion rate, Hourly, daily, weekly, monthly, quarterly, and annual sales, Cost of goods sold, Inventory levels, Site traffic, Unique visitors versus returning visitors, Customer service phone call count, Average resolution time
subscription
churn, CoCA, ARPU, MRR, LTV
premium subscriptions:
- What would be good metrics of success for a consumer product that relies heavily on engagement and interac- tion? (Snapchat, Pinterest, Facebook, etc.) A messaging product? (GroupMe, Hangouts, Snapchat, etc.)
heavily on engagement and interaction: uses AU ratios, email summary by type, and push notification summary by type, resurrection ratio
messaging product:
- What would be good metrics of success for a product that o ered in-app purchases? (Zynga, Angry Birds, other gaming apps)
Average Revenue Per Paid User
Average Revenue Per User
- A certain metric is violating your expectations by going down or up more than you expect. How would you try to identify the cause of the change?
breakdown the KPI’s into what consists them and find where the change is
then further breakdown that basic KPI by channel, user cluster, etc. and relate them with any campaigns, changes in user behaviors in that segment
- You’re a restaurant and are approached by Groupon to run a deal. What data would you ask from them in order to determine whether or not to do the deal?
for similar restaurants (they should define similarity), average increase in revenue gain per coupon, average increase in customers per coupon
- You are tasked with improving the e ciency of a subway system. Where would you start?
define efficiency
- Say you are working on Facebook News Feed. What would be some metrics that you think are important? How would you make the news each person gets more relevant?
rate for each action, duration users stay, CTR for sponsor feed posts
ref. News Feed Optimization
Affinity score: how close the content creator and the users are
Weight: weight for the edge type (comment, like, tag, etc.). Emphasis on features the company wants to promote
Time decay: the older the less important
- How would you measure the impact that sponsored stories on Facebook News Feed have on user engagement? How would you determine the optimum balance between sponsored stories and organic content on a user’s News Feed?
AB test on different balance ratio and see
- You are on the data science team at Uber and you are asked to start thinking about surge pricing. What would be the objectives of such a product and how would you start looking into this?
there is a gradual step-function type scaling mechanism until that imbalance of requests-to-drivers is alleviated and then vice versa as too many drivers come online enticed by the surge pricing structure.
I would bet the algorithm is custom tailored and calibrated to each location as price elasticities almost certainly vary across different cities depending on a huge multitude of variables: income, distance/sprawl, traffic patterns, car ownership, etc. With the massive troves of user data that Uber probably has collected, they most likely have tweaked the algos for each city to adjust for these varying sensitivities to surge pricing. Throw in some machine learning and incredibly rich data and you’ve got yourself an incredible, constantly-evolving algorithm.
- Say that you are Net ix. How would you determine what original series you should invest in and create?
Netflix uses data to estimate the potential market size for an original series before giving it the go-ahead.
- What kind of services would nd churn (metric that tracks how many customers leave the service) helpful? How would you calculate churn?
subscription based services
