Quant and Qual Testing (Thu Oct 19, lecture 13)

Homework due for today

Legend: :Participation :Early :PDF

  1. Outgoing email: One way to get feedback will be to email a survey to groups outside Brandeis. The purpose of this exercise is to practice pitching your product and asking a random person to help you with your startup. You are asking whether the person would be willing to spend 20 minutes talking to you about a product. Your email needs to convince the person to be interested in your product, and get them to say yes. Of course that means the email needs to be clear, persuasive, businesslike and relatively brief. Conceptually you might be sending the email to a LinkedIn Groups, Quora postings, other special interest mailing lists. Each team member separately, identify a specific group that could have useful information on some of your hypotheses. Compose an email that you would send to them expecting an answer. Deliverable: Email text, explanation of what hypotheses it is testing, and where you would send it for maximum effect, and why.
  2. Teams Go Out of Building: Go out of the building! Each member of each product team should take some hypotheses off the list to test outside the building. Make sure you record details of this round of out of the building work. What hypotheses were being tested, how did the tests go, who did them, and what results so far. You will need them for the final report. Come to class prepared to talk about it.
  3. Pirate Metrics Read and listen to the seminal “Pirate Metrics” presentation by Dave McClure. Video of Pirate Metrics, and the corresponding slide deck: Startup Metrics for Pirates.

Homework review

  • Review Homework. Quick update from each team
    • Your email question
    • Your out of the building experiences

Measuring effect of a pivot

  • Identify the key ‘stages’ that users experience on site
  • Example:
    • create account
    • log into account
    • add content
    • invite friend
  • Discussion: What would stages be for our product?

  • These are not arbitrary!!
    • they figure directly into your eventual business model
    • depending on your goals and your stage they will be different
      • How many people responded to your email
      • answered your survey
      • answered a question
      • asked a question
      • etc.
  • This becomes a classic “funnel” (sales, marketing etc.)
  • Discussion: why do you think we call it a funnel?

  • Metrics… Metrics… Metrics
    • You also need a way to measure how many people make it to each step
    • Without metrics your pivots are a shot in the dark
  • Once you have a baseline
    • Do the pivot
    • Quickly determine if you improved where you expected to improved

Hypotheses and testing

  • Discussion: Hypotheses for your term products product
    • Where are you keeping them? Give me the URL so we can all look at them
    • What are some characteristics of a well stated hypothesis?
    • Let’s discuss “technology” hypotheses and their role at this stage
    • How exactly could we test it qualitatively?
    • How exactly could we test it quantitatively?
    • Repeat for some more hypotheses

Next Class