Rashi
My Product JournalAbout
Back to Journal

⚡ Key Learnings - Metrics & Experiments

Here are the 5 key learnings I have had while working with data as a Product Manager, be it setting the right metrics or conducting an experiment, these points can serve as a quick guiding points when thinking about without feeling overwhelmed by it.

  1. Start with the decision you need to make and then go find the data, “What decisions could we make if we had all the information we need?”
    1. Take time to better understand the decision you are trying to make
    2. Then find any and all data needed to make it
  2. For every project, think of “What do you expect to happen?” to set the right targets
    1. Success metric - Tells you “what success looks like” (blue sky ceiling)
    2. Survival metric - Tells you “what failure looks like” (real world floor)
  3. Run your experiments with the goal of providing value to your users. Don’t prove value, Create it
    1. Basically ship a small thing which we think will be valuable to our users and see if it is in fact valuable to our users
    2. The way you communicate around experiment is more important than the experiment itself because nobody likes having anything proven to them ever
  4. Before doing A/B or planning experiments, think about:
    1. How many users are actually interacting with this thing?
    2. How meaningful are those interactions?
  5. How to take actions of the data:
    1. Know what metrics you’re looking at and how they connect to overall team & company goals
    2. Have clear and specific targets for these metrics
    3. Know what is going on with these metrics right now
    4. Identify underlying issues that are causing these metrics to do what they are doing
    5. Determine which underlying issues can be effectively addressed by you and your team
    6. Have a prioritised action plan for addressing these issues