My rating: 78/100
See Book Notes for other books I have read. If you like my notes, go buy it!
Dan walks us through the six steps of product development, mainly from a software perspective though principles can be gleaned for use in other industries. The software bent wasn’t quite clear when I first picked it up. I design mostly hardware, and the scrum methodology has difficulties with mechanical systems as described by David Ullman in his book Scrum for Hardware Design. That aside, I found the User Testing section of this book especially useful, and the resources list is probably great for software developers. At least there’s a lot there I’ve never heard of before.
One key item missing was the Project Definition phase, which precedes the Product Definition phase and defines the framework, personnel, resources, and tasks that will be needed during the project. Perhaps this was not on Dan’s mind when he wrote this book since it sounded like his projects were all software and he didn’t start on projects from the inception point. That’s fine, but maybe he had some insight into that administrative phase that we missed out on? I don’t know.
One very useful concept was the differentiation of problem space and solution space. “What” is problem space and “How” is solution space. Far too often technical individuals dive right into the solutions space, they say, “How are we going to solve this problem?” When they should first be looking at “What” the problem is at a level higher than the details. They should be asking, “What problem is the user experiencing, what are the nuances of that experience, and what benefits would they gain from having it solved?” Your team should be spending a lot of time talking about user benefits, and not as much time talking about product features.
A few other key tidbits:
- You should not invest in trying to grow your business until after you have achieved product-market fit.
- Test your product-market fit hypotheses before you build a prototype.
- Spend more time discussing user benefits than product features.
- Use metrics to quantify feature importance and satisfaction. Use these to determine what to work on next.
- After conducting this survey with many products, Ellis found empirically that products for which 40 percent or more of users reply “very disappointed” tend to have product-market fit.
Below are my underlined notes from the book.
Introduction: Why Products Fail and How Lean Changes the Game
The main reason products fail is because they don’t meet customer needs in a way that is better than other alternatives.
The lean product process consists of six steps:
- Determine your target customers.
- Identify underserved customer needs.
- Define your value proposition.
- Specify your minimum viable product (MVP) feature set.
- Create your MVP prototype.
- Test your MVP with customers.
Part 1: Core Concepts
Chapter 1: Achieving Product Market Fit with the Lean Product Process
Marc Andreessen coined the term product-market fit, “Product-market fit means being in a good market with a product that can satisfy that market.”
You can describe the size of a market by the total number of customers in the market or the total revenue generated by those customers. For either of those two measures, you can refer to the current size or the potential future size of the market.
You probably don’t want to enter a market where customers are extremely happy with how the existing solutions meet their needs.
To achieve product-market fit, your product should meet underserved needs better than the competition.
While the first “prototype” you test could be your live product, you can gain faster learning with fewer resources by testing your hypotheses before you build your prototype. Not all six steps are required for every product or feature. Certain steps are required only when you’re building a completely new product.
Chapter 2: Problem versus Solution Space
“What” is problem space and “How” is solution space.
Steve Jobs: “You’ve got to start with the customer experience and work backwards to the technology. You can’t start with the technology and try to figure out where you’re going to try to sell it … As we have tried to come up with a strategy and vision for Apple, it started with: What incredible benefits can we give to the customer? … Not starting with: Let’s sit down with the engineers and figure out what awesome technology we have and then how we’re going to market that. And I think that’s the right path to take.”
The internal testing tactic where you use your own product is called “dogfooding.”
Customers are much better at giving you feedback in the solution space.
Part 2: The Lean Product Process
Chapter 3: Determine Your Target Customer (Step 1)
Demographics are quantifiable statistics of a group of people, such as age, gender, marital status, income, and education level.
If you are targeting businesses, you’ll use firmographics instead; these are to organizations what demographics are to people, and include traits such as company size and industry.
Psychographics are statistics that classify a group of people according to psychological variables such as attitudes, opinions, values, and interests. You might describe your target customers as moms who enjoy using social media and like sharing pictures of their babies with friends and family.
You might define your target market as moms who currently share an average of three or more baby pictures per week on social media. If you were working on a stock trading app for active investors, you might define your target market as investors who place 10 or more stock trades per week.
Divide the market into customer segments that each have distinct needs.
In some cases, especially for business-to-business products, the customer who will use your product (the user) is not the same person who makes the purchase decision (the buyer).
Technology Adoption Life Cycle
- Innovators are technology enthusiasts who pride themselves on being familiar with the latest and greatest innovation. They enjoy fiddling with new products and exploring their intricacies. They are more willing to use an unpolished product that may have some shortcomings or tradeoffs, and are fine with the fact that many of these products will ultimately fail.
- Early Adoptors are visionaries who want to exploit new innovations to gain an advantage over the status quo. Unlike innovators, their interest in being first is not driven by an intrinsic love of technology but rather the opportunity to gain an edge.
- The Early Majority are pragmatists that have no interest in technology for its own sake. These individuals adopt new products only after a proven track record of delivering value. Because they are more risk averse than the first two segments, they feel more comfortable having strong references from trusted sources and tend to buy from the leading company in the product category.
- The Late Majority are risk-averse conservatives who are doubtful that innovations will deliver value and only adopt them when pressured to do so, for example, for financial reasons, due to competitive threats, or for fear of being reliant on an older, dying technology that will no longer be supported.
- Laggards are skeptics who are very wary of innovation. They hate change and have a bias for criticizing new technologies even after they have become mainstream. aka Luddites
Persona – A precise definition of our user and what he wishes to accomplish.
Personas are not real people but rather hypothetical archetypes of actual users.
Good personas convey the relevant demographic, psychographic, behavioral, and needs-based attributes of your target customer. Personas should fit on a single page and provide a snapshot of the customer archetype that’s quick to digest, and usually include the following information:
- Representative Photograph
- Quote that conveys what they most care about
- Job Title
- Relevant motivations and attitudes
- Related tasks and behaviors
- Frustrations/pain points with current solution
- Level of expertise/knowledge (in the relevant domain, e.g., level of computer savvy)
- Product usage context/environment (e.g. laptop in a loud, busy office or tablet on the couch at home)
- Technology adoption life cycle segment
- Any other salient attributes
Chapter 4: Identify Underserved Customer Needs (Step 2)
One of the easiest ways to tell that a product team is starting with the solution space is that instead of articulating customer benefits, they list product features. Benefit begins with a verb: help, check, reduce, maximize. Finally, many of the benefits speak to increasing something that’s desired (tax deductions) or decreasing something that is not desired (audit risk, time required to accomplish a task).
Customer Discovery Interviews
Share each of your customer benefit hypotheses with the customer during the interviews. Ask a set of questions about each benefit statement, such as:
- What does this statement mean to you? (to check their understanding)
- How might this help you?
- If a product delivered this benefit, how valuable would that be to you?
Possible responses: no value, low value, medium value, high value, very high value.
- For a response of high or very high value: Why would this be of value to you?
- For a response of low or no value: Why wouldn’t this be of value to you?
Keep asking them, “Why is that important to you?” until it doesn’t lead to any new answers.
Measuring Importance and Satisfaction
There are two types of rating scales: unipolar and bipolar. It’s usually best to measure satisfaction using bipolar scale. Importance is just a matter of degree and therefore better measured with a unipolar scale.
7 point scales are best for bipolar
“doing quan on qual” – quantitative analysis on qualitative data.
Two frameworks for evaluating feature importance: Gap Analysis and Jobs to Be Done
Gap = Importance – Satisfaction
The bigger the gap, the more underserved the need.
Jobs to be Done
Opportunity Score = Importance + Maximum(Importance – Satisfaction,0)
That is, if Importance – Satisfaction ever goes negative, just make it zero.
It is possible and essential for product teams to create a detailed and precise definition of their problem space.
When you first start off trying to solve a problem, the first solutions you come up with are very complex, and most people stop there. But if you keep going, and live with the problem and peel more layers of the onion off, you can oftentimes arrive at some very elegant and simple solutions. Most people just don’t put in the time or energy to get there.
Feature Opportunity = Importance – Current Value Delivered
Delighters provide unexpected benefits that exceed customer expectations, resulting in very high customer satisfaction. The absence of a delighter doesn’t cause any dissatisfaction because customers aren’t expecting it.
Chapter 5: Define Your Value Proposition (Step 3)
At this point, you have identified several important customer needs that you could potentially address. Now you need to decide which ones your product will address.
Here’s what Steve Jobs had to say about saying “no”:
People think focus means saying yes to the thing you’ve got to focus on. But that’s not what it means at all. It means saying no to the hundred other good ideas that there are. You have to pick carefully. I’m actually as proud of the things we haven’t done as the things I have done. Innovation is saying no to 1,000 things.
Competitors doesn’t just mean direct competitors: in the unlikely case that you don’t have any direct competitors, there should still be alternative solutions to your product that customers are currently using to meet their needs (for example, pen and paper is an alternative to TurboTax).
Once you have established the benefits and competitors, you want to go through each row and score each of the competitors and your own product.
The entries for must-haves should be “Yes.” For performance benefits, a scale of high, medium, low usually works well.
A clear value proposition focuses your resources on what’s most important, and increases your chances of success.
Wayne Gretzky said, “I skate to where the puck is going to be, not where it’s been.”
Chapter 6: Specify Your Minimum Viable Product Feature Set (Step 4)
My note: this is essentially writing the product spec in the mechanical design process.
For each benefit in your product value proposition, you want to brainstorm as a team to come up with as many feature ideas as you can for how your product could deliver that benefit.
At this point, brainstorming rules should apply. You should be practicing divergent thinking, which means trying to generate as many ideas as possible without any judgement or evaluation. When you are done brainstorming, organize all the ideas by the benefits that they deliver.
Once you have organized your list of feature chunks by benefit and prioritized them, it’s time to start making some tough decisions. You must decide on the minimum set of functionality that will resonate with your target customers. Your MVP candidate needs to have all the must-haves you’ve defined.
Delighters are part of your differentiation, too. You should include your top delighter in your MVP candidate. That may not be necessary if you have a very large advantage on a performance benefit. The goal is to make sure that your MVP candidate includes something that customers find superior to other products and, ideally, unique.
I don’t recommend that you plan more than one or two minor versions ahead at the outset – since a lot of things are apt to change when you show your MVP candidate to customers for the first time.
At this point in the Lean Product Process, you have done a fair bit of work. You have:
- Formed hypotheses about your target customers
- Formed hypotheses about their underserved needs
- Articulated the value proposition you plan to pursue so that your product is better and different
- Identified the top feature ideas you believe will address those needs and broken them down into smaller chunks
- Prioritized those feature chunks based on ROI
- Selected a set of those feature chunks for your MVP candidate, which you hypothesize customers will find valuable
Chapter 7: Create Your MVP Prototype (Step 5)
Once you have specified the feature set for your MVP candidate, you’ll want to test it with customers.
The goal is to build a prototype that lets you test your hypotheses.
Many people misinterpret the term MVP by placing too much emphasis on the word minimum. They use this as an excuse to build a partial MVP that has too little functionality to be considered viable by a customer.
One good way to test your overall messaging is the five-second test. The idea is to show customers your home page or landing page for just five seconds and then ask them to tell you what they remember and what they liked. Because customers make snap judgements about products all the time, this can be a good way to see how well your messaging conveys what your product does and why someone would want to use it.
One of the most popular tests is the landing page or smoke test. The landing page describes the product you plan to build and asks customers to express some level of interest, which is usually a “sign up” button or a link to a “plans and pricing” page. It’s also called a smoke test because there is no real product for customers to use yet.
Wizard of Oz and Concierge MVPs
Concierge MVP Example: Airbnb
Quantitative Product MVP Tests
Chapter 8: Apply the Principles of Great UX Design
A product with a great UX feels easy to use. It’s effortless to find what you’re looking for and to figure out what to do next. You don’t even notice the user interface and are able to focus on accomplishing the task at hand. The product may even be fun to use and convey emotional benefits such as confidence in your abilities or peace of mind.
Olsen’s Law of Usability: The more user effort required to take action, the lower the percentage of users who will take that action (and vice versa).
Ask users, “How easy or difficult is the product to use?” and allow the ratings on a seven-point bipolar scale.
- Very difficult to use
- Difficult to use
- Somewhat difficult to use
- Neither easy nor difficult to use
- Somewhat easy to use
- Easy to use
- Very easy to use
Color is an important aspect of a product’s visual design. Warm colors such as red, orange, and yellow are typically more energizing and passionate, whereas the more subdued cool colors such as green, blue, and purple are more calming and reserved. Many applications and websites use a blue scheme because it conveys trustworthiness and calm. Green is associated with nature, growth, and money. Purple suggests luxury and creativity. Red is associated with aggression, passion, power, and danger. Orange is energetic and vibrant. Yellow conveys happiness and sunshine. Brown is associated with warmth and the earth. Black can suggest sophistication, elegance, and mystery. White is associated with purity, cleanliness, and simplicity.
Typography. Traditional design advice has been that serif fonts work better for print materials, which have very high resolution (dots per inch), whereas sans serif fonts work better for the web, which has lower resolution.
Limit the number of different fonts you use in your product. A common approach is to select two fonts: one for body text and one for large text, such as headings.
According to Gestalt principle of proximity, the brain perceives objects that are closer together as more related than objects that are farther apart. Objects that are similar or related should look similar by having the same shape, size, or color. Avoid making unrelated objects look alike.
The brain assumes larger objects are more important and smaller objects are less important. It also assumes that elements with high contrast are more important.
Principles of Composition
- Unity: Does the page or screen feel like a unified whole or a bunch of disparate elements?
- Contrast: Is there enough variation in color, size, arrangement, and so forth to create visual interest?
- Balance: Have you equally distributed the visual weight (position, size, color, etc.) of elements in your design?
- Use of Space: How cluttered or sparse does your design feel? Ensuring your design has enough white space – the space you don’t use on the page or screen – is important to avoid designs that feel crowded to the user.
Chapter 9: Test Your MVP with Customers (Step 6)
When you are so close to your product, it is difficult – often impossible – to perceive it as a new customer does.
I recommend conducting user tests with one customer at a time.
I’ve found that testing in waves of five to eight customers at a time [per product revision] strikes a good balance. After each wave, you will be revising your product or design artifact based on what you learned and then testing it with a new wave of customers until you’ve validated that you’ve achieved product market fit.
Of the three qualitative testing methods – in-person, moderated remote, and unmoderated remote – I would recommend in-person if possible.
The third type of testing is unmoderated remote testing, which you accomplish using a service such as UserTesting or Validately.
Make sure that the customers with whom you are testing are in your target market.
One approach is to try to recruit local participants by posting online to Craigslist, TaskRabbit, and similar websites. A best practice is to include in your posting a link to an online survey hosted at SurveyMonkey, Google Forms, or another survey site with your screener questions.
Some companies use Amazon’s Mechanical Turk (MTurk) as an affordable recruiting source for remote testing. Many remote testing services, such as UserTesting, have a panel of customers available for testing.
How much should you compensate testers for their time? The typical range is $75 to $125 per hour, but it depends on your target customer and how much their time is worth.
Gift cards are convenient for both parties since they are easy to buy and easy to use. A general purpose gift card such as Visa or Mastercard has more appeal than a specialty gift card.
I was first introduced to user testing at Intuit, a pioneer in the field. After the launch of each new version of Quicken, product managers would conduct “follow me homes.” They would wait in the store aisle where Quicken was being sold. When they saw a customer who was going to purchase Quicken, they would ask if they could follow the person home, where they would observe the customer install and use the product. the ability to watch customer use our product in their real world setting gave us lots of valuable insights.
Ramen user testing, a technique that eliminates everything but the essential parts of user testing.
Because it can be challenging to try to moderate and take notes at the same time, I recommend having a dedicated note-taker. [This could easily be supplemented/replaced by a voice record on your phone]
A maximum of three people in a conference room with the customer is enough.
I’ve never seen anyone actually go back and watch video recordings [of user tests]. Skip the recordings and focus on watching the live sessions.
Prepare a test script that lists what you plan to show and ask the user.
User tests typically run about an hour plus or minus 15 minutes, maybe longer if the user is excited about your product and giving your lots of feedback. I recommend spending the first 10 to 15 minutes of the session warming the user up and conducting discovery about his or her needs and current solution. Then I like to spend about 40 to 45 minutes getting feedback from the user on the product or design artifacts. I close with 5 to 10 minutes of wrap-up, where I answer any questions from the user and ask any closing questions that I have.
Explicitly tell user up front that you want their honest feedback, even if it’s negative. Their critical feedback will help make the product better – which is the whole reason for conducting the user test.
How to ask good questions. Start by asking them about their current behavior and feelings about the key benefit you plan to provide. [aka discovery]
After discovery, you transition to the product feedback portion of the user test.
If a user takes an action on a prototype but doesn’t verbalize that they did or why they did, a good moderator might say, “I see you just clicked that button. Could you tell me why?”
Ask open versus closed questions. Long pauses are going to happen; users need time to process what you are showing them and formulate their thoughts. While such periods of silence would feel awkward in a normal conversation, they’re totally fine during a user test.
If users have difficulty understanding or using your product, it’s important not to help them, as painful as that may feel. Act as though you were a fly on the wall.
Wrapping up the test. “On a scale of 0 to 10, with 10 being best, how valuable did you find the product?,” or “Based on what you saw today, how likely would you be to use the product?”
This is also when you should give users any compensation for their time and thank them.
I also like to include two yes-no questions: “Would you be willing to participate in future research?” and “Would you like to be notified when this product is available?”
Remember, the users in your new wave never saw the earlier version of your product. So no new user is going to tell you, “Nice job fixing ____.” Instead, you measure such progress by silence – the absence of hearing complaints you heard in previous waves.
Usability versus product market fit. It’s crucial as you conduct your user tests to differentiate between feedback on usability versus product-market fit. Feedback on usability has to do with how easy it is for customers to understand and use your product, whereas feedback on product-market fit has to do with how valuable they find your product.
You need to explicitly assess product-market fit by asking how much they value your product.
It’s very important that you ensure that the customers with whom you are talking are in your target market.
Chapter 10: Iterate and Pivot to Improve Product-Market Fit
You should consider pivoting if you just don’t seem to be achieving gains in product-market fit after several rounds of trying to iterate. If, despite your best efforts, your target customers are only lukewarm on your MVP, you should consider a pivot.
Chapter 11: An End-to-End Lean Product Case Study
User Testing Script
While the recruiting was taking place, I created the script I planned to use in moderating the user testing. Each session was 90 minutes long. Here is a high-level outline of my user testing script showing time allocation:
- Introductions and warm-up (5 min)
- General discovery questions (15 min)
- Direct marketing mail
- The data about you that companies have
- Comparing yourself to other financially
- Concept-specific questions (45 min)
- Discovery questions related to concept’s main theme (10 min)
- Feedback on mockups (35 min)
- Review: What did you like/dislike about what you saw? (5 min)
- Brainstorm: What would make the product more useful/valuable? (10 min)
- Feedback on possible product names
- Thanks and goodbye
Part 3: Building and Optimizing Your Product
Chapter 12: Building Your Product Using Agile Development
This was just a rehashing of Scrum methodology so I skipped taking notes.
Chapter 13: Measure Your Key Metrics
Qualitative ~ Oprah
Quantitative ~ Spock
You should not invest in trying to grow your business until after you have achieved product-market fit.
Ask the users of your product the question, “How would you feel if you could no longer use [product X]?” The four responses are:
- Very Disappointed
- Somewhat disappointed
- Not disappointed (it isn’t really that useful)
- N/A – I no longer use the product
After conducting this survey with many products, Ellis found empirically that products for which 40 percent or more of users reply “very disappointed” tend to have product-market fit.
Chapter 14: Use Analytics to Optimize Your Product and Business
Chapter 15: Conclusion
Here’s a list of the tools I mention in the book, plus others that I’ve found useful. I also list valuable books, people, and blogs that I recommend checking out. They are all great sources of information related to the topics I’ve covered in this playbook. For an updated list of resources, visit http://leanproductplaybook.com
- Balsamiq: http://balsamiq.com
- Axure: http://axure.com
- UXPin: www.uxpin.com
- Sketch: http://bohemiancoding.com/sketch
- InVision: http://invisionapp.com
- Flinto: https://www.flinto.com
- Marvel: https://marvelapp.com
- POP: https://popapp.in
- Dapp: http://dapp.kerofrog.com.au
- OmniGraffle: https://www.omnigroup.com/omnigraffle
- Bootstrap: http://getbootstrap.com
- UserTesting: http://usertesting.com
- Validately: https://validately.com
- Ask Your Target Market: http://aytm.com
- Qualaroo: https://qualaroo.com
- SurveyMonkey: https://surveymonkey.com
- Join.me: https://join.me
- Screenleap: http://screenleap.com
- Jira Agile
- Pivotal Tracker
- Rally: https://rallydev.com
- VersionOne: http://versionone.com
- SwiftKanban: http://swiftkanban.com
- LeanKit: http://leankit.com
Analytics and A/B Testing
- Google Analytics
- KISS metrics
- Visual Website Optimizer http://vwo.com
- What Customers Want by Anthony Ulwick
- UX for Lean Startups by Laura Klein
- The Lean Startup by Eric Ries
- Running Lean by Ash Maurya
- Crossing the Chasm and Inside the Tornado by Geoffrey Moore
- Inspired by Marty Cagan
- The Inmates Are Running the Asylum by Alan Cooper
- Don’t Make Me Think and Rocket Surgery Made Easy by Steve Krug
- The Non-Designer’s Design Book by Robin Williams
- The Elements of User Experience by Jesse James Garrett
- Measuring the User Experience by Tom Tullis and Bill Albert
- Designing for Emotion by Aaron Walter
- Smart Choices by John Hammond
- Pretotype It by Alberto Savoia
- Information Visualization by Colin Ware