Build Measure Learn (Tue Oct 3, lecture 9)

Homework due for today

Legend: : Participation | : Early | : PDF | : Portfolio | : Zipped

  1. Read: Read Chapter 7 of “Lean Startup”. This chapter begins the justification and explanation of the importance of measurement. Here are some warmup questions:
    • The chapter talks a lot about “the engine of growth”. Explain the concept in one paragraph so that someone who is not familiar with Lean Startup could understand it.
    • What is Lazy Registration? Describe the reason why Grockit implemented it originally, how they tested its effectiveness and what they concluded about Lazy Registration at Grockit.
    • Thinking about your pilot project, what would an example be of a “vanity” metric and what of an “actionable” metric? In what way is that vanity metric?
    • Checkin: What 1 or 2 important things from the reading did you not understand? And if you understood it all, then what 1 or 2 important things were most insightful/useful/valuable to you. Please make sure I can tell which is which!
    • Deliverable: Write a short document with your responses, as a pdf
  2. Team Alignment Meet for at least 2 hours, face to face with your team and come up with your team’s product vision with the following elements. Notice that for this exercise it is necessary to discuss each point in the team meeeting and articulate your answer in a fairly precise way that you all can accept. I realize that they are so far not tested; you will be able to improve and amend them in the coming days and weeks. The important thing at this point is that all team members engage and understand and agree with the vision as of now.
    • Product Vision: Your current vision of the product. Make it descriptive, not salesy. Remember the reader is not inside your head they will need to have it explained.
    • Problems that it solves: Continue your explanation by explicitly stating the list of 1 or 2 or 3 real world user problems that it solves. Make it pass the common sense tests. These should not be tiny silly problems but should be recognizable by the target users. Distinguish between problems you solve and pains you remove.
    • Who is/are the customer/s: Describe groups of users or customers in a way that keeps in mind how you might reach them. If your customers are “Higher Ed Students who are having trouble finishing their homework on time”. How would you find them to talk to them and dig deeper? Also remember that your vision might include two disjoint sets of users or customers.
    • Team Deliverable: Your 1 page product vision, as a pdf


  • Ari Carr about their project
  • Feedback from class, and my proposed actions:
    • More in class discussions, activities, brainstorming. And I deduce the correlary, which is less lecture-heavy classes. I accept this criticism, I tend to be so excited about the material that I get carried away. Also requested: more guest speakers.
    • Less Homework, readings, 10am deliverables, and so on. I hear this one every year. I try specifically to make sure that the readings and homework enhance the material of the class proper. Not every single thing has to be part of the term project.

Success with Teams

Discussion: Each table discuss three things that were challenges working with your teams


  • You can’t manage what you don’t measure. What gets measured gets managed
  • Vanity vs. Actionable metrics
    • The fact that I have more users today than I did last week doesn’t mean that much
    • You need to know what you are trying to optimize for
    • Then you need to figure out what changes improve those variables
    • Only then can you tell if it’s a good thing that you have more users today!
  • Useful Metrics are
    • Actionable:
      • You can use them to figure out what to do, or what you did, to cause the change.
      • Need to know what to do to get more of that result
    • Transparent: Publish metrics to the team
    • Accessible: Make the definition of what’s being measured clear. No made up words or jargon.
    • Auditable: Make it easy to see the underlying raw data
  • But, beware of ‘vanity metrics!’
    • More users today than yesterday? Not enough.
    • How do I change it. What changes will affect what metrics?
    • You can make changes, but will they matter to customers?
Discussion: Each table, discuss What are some examples of metrics for your product? How could we tell if they are vanity or not?

Next Class