Chapter 7: Measure Flashcards
What is a start-up’s 3 basic jobs?
- Build an MVP to measure where it is now
- Experiment to improve metrics
- Deciding whether to continue in the same direction or pivot in in a new direction.
What is a vanity metric?
A vanity Metric is a metric that looks good on paper. It’s something that doesn’t show real growth but rather looks good.
Example:
Emails sent
Employees
Revenue
Investments/money raised
and so on…
What is a good metric?
Something that helps you to see your company’s real progress.
This is different from company to company but it’s very important to measure progress.
What is Cohort analysis?
Cohort analysis is measuring users in groups and measure their behavior.
IMVU Cohort analysis: https://www.google.com/search?q=imvu+cohort+analysis&client=firefox-b-d&sxsrf=ALiCzsYMdCukaMkjQC4nv-09DrsMtP8SuA:1670500805004&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjl7NC4_On7AhUBX_EDHdzTDjAQ_AUoAXoECAIQAw&biw=1536&bih=739&dpr=1.25#imgrc=F4skBkjrLwZVYM
One cohort is a group of new users who are integrating with your product. This makes it possible to see progress and changes in user interactions.
What does it mean to establish the baseline?
It means that you need to figure out what is most critical in your company. This thing should be so critical that if this thing now works, then nothing will work for us.
Then you should test this thing and you know that if this thing doesn’t work there is no point in even trying the other things.
What determines if it’s time to pivot or persevere?
If you are not getting closer to your business’s ideal, and you are stuck, it’s time to pivot.
This is why you use Cohort analysis to see if you are stuck, or making progress.
What to do instead of discussing what features to include in a product?
Test the features and see wich one the customers respond the best to.
Maybe it is both!
How did IMVU use small customer segments
They each day invited a little group into their new product. That group got measured, but because it was a small group, it was very cheap to get them onboard and measure them.
They were now sure to have data to measure in the next Cohort analysis.
What is split-testing?
Split testing, or A/B testing, is a way to split your customers up into two groups and show each group a product. After some time you can compare the two customer group behaviors and see which one to go with.
You can also show a new feature to 50% of the customer while the other 50% is still using the old version. This makes it easy to switch back to the old if the new features should have a negative impact on the group(s).
Explain Kanban Diagram
https://www.google.com/search?q=kanban+diagram+lean+startup&client=firefox-b-d&sxsrf=ALiCzsaXlBCoiRuTEJQqnPZ-d65IGP7W7g:1670502578700&source=lnms&tbm=isch&sa=X&ved=2ahUKEwjWzLKGg-r7AhXrQ_EDHY5NBwUQ_AUoAXoECAIQAw&biw=1536&bih=739&dpr=1.25#imgrc=p5nc6fqaPDG82M
What are: Actionable Metrics?
You need to could explain what have caused the metrics to move. It needs to show if you have marked progress and where the progress is coming from.
If you can’t explain what has caused the metrics to move it could be because you are using a Vanity metric.
What are: Accessible Metrics
It is to make metrics and experiments simple so people understand them easily. You have failed to do this when people come to ask you for help op decode your metrics.
It is also a good idea when you publish the metrics in the company so people can see the progress. Make sure to say what has caused the metrics, but remember not to criticize someone publicly if something has influenced the metrics negatively.
What is: Auditable Metrics
Auditable Metrics are Metrics you can confirm a true. You need to show in your report how the metrics are measured and compiled.
This is because you need to could explain to people how you have gathered these metrics because when the metrics are negative, people tend to criticize them and blame you for using bad metrics.
Therefore you need to be able to proof the metrics are faultless.
