Continuous Discovery Habits – Teresa Torres

Author: Teresa Torres Type: #book

Reference: Torrest (2020). Continuous Discovery Habits

Discovery vs. delivery

Product discovery – set of activities to decide what to build Product delivery – set of activities to build and deliver it

Many teams do discovery only ad-hoc, quarterly or monthly. It's better to do it weekly so that there's continuous input from customers that teams can use in their daily product decisions.

Co-creating with customers early on > Validating your assumptions at the end of discovery

"Create value for customers in a way that creates value for the business"

Part 1: Common framework for discovery (aka Theory intro to Opportunity solution trees)

• Outcomes > Outputs
• Teams should focus on outcomes. Businesses should task teams with outcomes, not outputs.
• It's important not to drive business value at the cost of customer value. Balance is needed here.
• Product trios need to discover customer needs, pains and desires, that addressed, will drive the outcome the team seeks.
• Torres calls customer needs, pains and desires collectively "opportunities".
  • She specifically doesn't call those "problem to solve" to be inclusive of products and solutions that address desires, not necessarily fix problems.
  • So she says "opportunity space" which covers problem space and desire space.
• Discovering a solution to drive an outcome is an ill-structured problem. "How we frame an ill-structured problem impacts how we might solve it."
• Discovery has a structure that teams should map out: 1) define an outcome, 2) discover opportunities, 3) discovery solutions to each opportunity, 4) test assumptions (e.g. solution X will address opportunity Y)
  • She proposes "opportunity solution tree" as a framework to visualise and map the discovery. In her words it's "a simple way of visually representing the paths you might take to reach a desired outcome"
  • The outcome anchor is key because it filters which opportunities we should consider: we might discovery many, but should only consider those that have the potential to drive the desired outcome.
• The OST helps team adopt a continuous mindset by breaking down big opportunities into smaller ones. Teams can then more easily and continuously address those smaller opportunities. By doing so, they address the bigger one over time.
• "The best designers evolve the problem space and the solution space together." The two spaces shouldn't be separate, certainly not by role (e.g. PM owning problem, DES owning solution). All roles in a product trio should be learning together, frame the problem and participate in finding the solution. It just might be that the designer facilitates it and then develops the solution more deeply.
• After testing a solution, we need to reflect. If it didn't work, how does it change our understanding of the opportunity?
• OSTs should evolve continuously as teams progress with discovering opportunities, finding solutions and testing them. Best teams also work bottom-up to reflect how the tests and more interviews change the higher parts of the tree.

Part 2: Continuous Discovery Habits

11 habits for continuous discovery:

• Managing by outcomes
  • There are different kinds of outcomes: business vs product vs traction metrics
    • Business outcomes tend to be lagging indicators: e.g. growing revenue or customer retention
    • Product outcomes can be leading indicators for the business outcomes: it measures how well the product moves the business outcomes.
    • Product teams (trios) will generally do better to pursue product outcomes. It's more in their control.
    • Assigning product outcomes to product teams gives them more specific responsibility and ownership. With business outcomes, it would be easier to depend on other business functions.
    • Setting and assigning outcomes should be a conversation between product leaders and product trios (note how it's not just the PM!)
      • It can lead to the team being more engaged and perform better when it participates in the outcome setting. (source: Groen B. et al. (2017). "High Job Performance Through Co-Developing Performance Measures With Employees")
    • When given a completely new outcome, first setting a learning goal (e.g. discover opportunities for X), then performance goal (e.g. increase X) – can lead to better performance of the team.
    • Anti-patterns to avoid:
      • Choosing an output as an outcome (e.g. "Launch iOS app" vs. "Increase mobile engagement"). Big no no.
      • Jumping from outcome to outcome, quarter to quarter. Teams need longer time (multiple quarters) to be able to deliver on an outcome. Jumping between outcomes is a sign of fire-fighting culture.
      • Pursuing too many outcomes at ones. Teams need focus.
      • Focusing on one outcome at the cost of everything else.

Discovering opportunities

• Map what you know (experience map)
  • Set a scope. Your outcome will inform the scope. Try to frame the outcome as a question (e.g. outcome is "increase sign-ups" -> "What's preventing potential customers from signing up?")
    • Optimization outcomes might need narrower scopes, open-ended ones broader scope.
  • Map what each member of product trio knows individually first, only then come together.
  • Make it visual, not verbal -> makes for a better shared understanding
    • Verbal description can lead to illusion of understanding. Visual descriptions are much less prone to it.
    • Don't mistake this for a concept map of how your product works. Draw the actual experience.
  • Share individual maps in the trio. Ask questions, clarify. No one's perspective is the right one here.
  • Co-create a shared experience map.
  • Take the map for what it is: set of assumptions, not the truth.
• Talk to customers to uncover opportunities (capture with interview snapshot)
• Document opportunities with OST
• Use the tree structure to prioritize the opportunity space

Customer interviews

• The purpose of talking to customers is not to ask them what you should build (or what they want). The purpose is to discover opportunities (needs, pains, desires).

  • In Christensen's lingo, it would be the job-to-be-done – the progress that customers want to make.

• We must ask about past behaviors, otherwise we may not get an accurate answer. When people don't know, they often default to generalized or rationalized stories.

• When you ask people directly ("how do you do X?"), you can't know if they tell you about their ideal behavior or their actual behavior.

• Building products based on asking customers what they want is "like opening a salad bar across a McDonald's because customers said they wanted to eat healthier"

• Direct questions lead to idealized, generalized answers. And not to a good understanding of the reality.

• Direct questions tend to come from research questions. People often take a research question and just ask it in interviews. Doesn't work.

  • You want to create separate "interview questions" (i.e. what you ask) that will lead to answering "research questions" (i.e. what you want to learn)

• Synthesize notes after each interview.

  • Torres offers "interview snapshot" framework

• Interview continuously

• Product trios should interview together

Opportunity solution tree

Opportunity solution tree (Framework)

Prioritization

• Prioritize opportunities, not solutions and features.

• Focus on one opportunity at a time -> deliver value iteratively often over time, not seldom and all at once

• When prioritizing opportunities, consider:

  • opportunity size (how many customers and how often it affects?)
  • market factors ("is this opp. table-stakes or a differentiator?")
  • company factors
  • customer factors

• Don't turn this into a science, embrace the messiness of this process.

  • Ill-structured problems can't be turned into a well-structured by using a math formula on them.
  • There won't be a clear winner. There isn't one right answer what to prioritize.

• Distinguish Level 1 decisions (hard to reverse) and Level 2 decisions (easy to reverse)

  • On Level 1, be slow and cautious with making decisions ("one-way door")
  • On Level 2, more fast, don't wait for perfect data. ("two-way door")
    • Torres' quote about acting first, is better suited for Level 2:
      • "We'll learn more by making the decision and then seeing the consequences of having made that decision than we will from trying to think our way to the perfect decision."

• Discovery is mostly about two-way door decisions. Exploring an opportunity and soltuions is reversible if you learn it's not working.

• Research (by Lottie Bullens et al., University of Amswerdam) found that when people view decisions as reversible, they think about them more critically even after the decision is made. They notice the negatives + consider positives of alternatives. They're open to being wrong. When people view decisions as irreversible, they tend to focus on its positives and on the negatives of the alternatives – we try to confirm to ourselves we made the right decision.

Generating solution ideas

1. Review your target opportunity
2. Generate ideas alone
   1. Bring a diverse group. The whole team. Maybe some key stakeholders.
3. Share your ideas
4. Repeat 2. and 3.
   1. Don't limit yourself to just one ideation session.
5. Evaluate ideas (which would best address the opportunity?)
   1. Dot vote to find the best ones

• Torrest argues for testing assumptions, not ideas.
  • It's almost identical to Cagan's Four product risks
• Types of assumptions:
  • desirability ('value' in Cagan's lingo): will people want this? will they find value in it?
  • viability: is this viable for our business? can we support this solution? will it help our business?
  • feasibility: can we build this? how complex is it?
  • usability: can people use it?
  • ethical: is there any harm in building this?

Three ways to uncover assumptions: 1) Story mapping

• To make ideas clearer, Torres suggest to story map – draw out steps how end-users will get the value out of a proposed solution idea.

• Then for each step, see what assumptions (i.e. risks) there are. There will probably be some of each type. 2) Pre-mortem

• Another way to uncover assumptions is to do a pre-mortem. Especially useful for bigger initiatives.

• Key factor: phrase the "failure question" as if it was certain. "It's 6 months from now. The initiative was a complete failure. Why?" 3) Walk through the opportunity tree

• Answer "why" a certain solution would solve the target opportunity. Be very specific. Go up the ladder of abstraction. Ask multiple whys.

• It's important to identify the riskiest assumptions and test those. That narrows how many assumptions you'll test. You don't want to test all of them – you'll probably be able to say about most if they're true based on existing data from the discovery so far.

• Trips for writing assumptions:

  • Phrase assumptions as if they were true, not in a negative way.
  • Be specific. Avoid "Customers will know how to use this" or "We'll be able to build this."

• How to prioritize assumptions? Assumption mapping (by David J. Bland)

  • Map each assumption in 2x2 matrix along 2 axes:
    • Evidence: strong vs. weak
    • how sure are were this is true? the less sure (less evidence) the more we need to test it)
    • Importance: more vs less
  • Review already placed assumptions relative to each other
  • Top right quadrant contains assumptions you should likely test (not necessarily all of them, though)