Build Measure Learn (Tue Oct 3, lecture 9)

Homework due for today

  1. Read: Read Chapter 7 of “Lean Startup”. This chapter begins the justification and explanation of the importance of measurement. Here are some warmup questions:
    • The chapter talks a lot about “the engine of growth”. Explain the concept in one paragraph so that someone who is not familiar with Lean Startup could understand it.
    • What is Lazy Registration? Describe the reason why Grockit implemented it originally, how they tested its effectiveness and what they concluded about Lazy Registration at Grockit.
    • Thinking about your pilot project, what would an example be of a “vanity” metric and what of an “actionable” metric? In what way is that vanity metric?
    • Checkin: What 1 or 2 important things from the reading did you not understand? And if you understood it all, then what 1 or 2 important things were most insightful/useful/valuable to you. Please make sure I can tell which is which!
    • Deliverable: Write a short document with your responses. pdf. latte.
  2. Term Project Brainstorm: Everyone meet with 1 or at most 2 other students. Feel free to post to the Latte class forum if you’re looking for a partner. Spend 2 hours being very creative about thinking up new products. Here are tips about choosing a product: Choosing a project to work on. Each student’s “deliverable” is to write up one new product on the Product Idea List.
  3. Term Project Feedbacks: Review the Product Ideas List. Read or scan all the other ideas. Each student’s “deliverable” is to add a comment to two different projects proposed by others, with new ideas, questions, and +1’s as you feel apprpriate.

Homework Discussion

  • Each team tell the class some of your out of the building experiences


  • You can’t manage what you don’t measure. What gets measured gets managed
  • Discussion: What is a metric? Examples?
  • Vanity vs. Actionable metrics
    • The fact that I have more users today than I did last week doesn’t mean that much
    • You need to know what you are trying to optimize for
    • Then you need to figure out what changes improve those variables
    • Only then can you tell if it’s a good thing that you have more users today!
    • Discussion: What are some examples of metrics? How could we tel if they are vanity or not?
  • Useful Metrics are
    • Actionable:
      • You can use them to figure out what to do, or what you did, to cause the change.
      • Need to know what to do to get more of that result
    • Transparent: Publish metrics to the team
    • Accessible: Make the definition of what’s being measured clear. No made up words or jargon.
    • Auditable: Make it easy to see the underlying raw data
  • But, beware of ‘vanity metrics!’
    • More users today than yesterday? Not enough.
    • How do I change it. What changes will affect what metrics?
    • You can make changes, but will they matter to customers?
  • Discussion: How do you decide what metrics matter?

Term Projects

  • Discussion: About possible term projects
    • Go through all the proposals for a quick critique.
    • Review lists of products for part two of the course.
    • See which ones look good. Start thinking about which one you want to be part of!

Next Class