Empathy, Market Research, and Product Design

Product Development: Real-World Examples

Over the years, I've had the good fortune to work with many different software development teams, all populated with bright, talented people. While equally smart, these teams sometimes took very different approaches to designing and developing products.

• Flying blind. One group of developers had really no idea how customers used their complex software product. The developers understood the basic idea of the product, and some of them had a thorough understanding of the technical minutiae underlying certain screens and tools, but over the years, as features accumulated and screens became ever more cluttered, the developers themselves lost track of what was important, what was useful, and how the customer, trying to solve a problem quickly, would use the various features of the product in a particular sequence. They were grateful whenever a field engineer spent an afternoon showing them how customers would actually go about using the product to solve a specific problem. Even with this understanding, though, it was difficult to undo the complex product architecture that had evolved, or accreted, rather, over time. The product remained a cluttered collection of screens that often bewildered new users.
  
• Academic certainty. Another group of developers based their product design on ideas from their academic research and from a very early Beta test of the product that occurred two years earlier. They were certain that the product's interface couldn't be improved, until a seasoned UI designer was brought in as a consultant, questioned their assumptions, and showed them how their pretty good interface could be made even better.
  
• Proud of complexity. Another group of developers was very proud of how complex and sophisticated their software was. Company slide presentations would boast, "Over 500,000 lines of code!" The software was so sophisticated (or perhaps so complex and finicky) that even whip-smart sales and marketing engineers had trouble getting it to work without assistance from the engineering team. Because the software was so complex, it had to be sold with lots of consulting. When the price of consulting equaled the cost of the software, and when deal sizes began reaching six figures, customers balked. Deals fell through. Complexity meant difficulty, and that didn't serve the business goals of customers.
  
• Experience-focused. Another group of developers had the privilege of working from a thoroughly designed and discussed stack of PowerPoint slides that walked users and developers through all the common tasks the application needed to support. How did they come up with slide deck? The company's founders interviewed CIOs at major companies, looking for an important problem to solve. Once they identified the problem, they began designing the interface and software architecture. Coding didn't begin until the interface had been thoroughly designed. By then, the team knew what was needed on the back-end to support the features customers would be accessing through the UI. Because the UI was "finished" relatively early in the process, documentation and training materials could be developed in parallel with the software itself. Customers knew what they were getting, and management knew that customers wanted the product being built.
  

Of all these approaches, I liked the last one—the experience-focused one—the best. The company did follow a Waterfall model of development, which doesn't offer the flexibility of more modern agile development methods, but by focusing on the end user's experience, it ensured that every software release delivered a complete set of features for performing important tasks.

This latter team certainly avoided the problems of the first team, who wasn't sure about the workflows the customer would use, and who allowed the software interface to become cluttered and confusing. Ultimately, this clutter limited the appeal of the product. Technical people didn't mind it, but non-technical people did. Even though the product had real potential as a reporting tool for management, its busy, nuts-and-bolts-first interface restricted its user base to technical end users. That limitation proved to have economic consequences.

Empathy

One of the biggest challenges all kinds of businesses face is really understanding what their customers need and want. We all approach this problem with our own particular skills and work histories which illuminate some aspects of customer needs and obscure others. And, off course, customer wants and needs are not static. They change, as technology and markets change, and as customers themselves develop new capabilities, becoming more Web-savvy, for example.

The challenge for vendors is to find a way to systematically see through this clutter and find, not merely what customers will tolerate or find useful, but what they will find empowering, indispensable, and worth paying for with hard-earned cash.

It's all too tempting for company founders to sit around a conference room, discuss their past experiences, throw up some feature/benefit charts, and decide what to build right then and there. Mostly likely the arrow they design will hit somewhere on the target, rather than flying off into the trees, but it probably note strike the bull's-eye itself. Instead, there will be enough interest and heading nodding from prospects, reviewers, and journalists, that the company will proceed with its plans, which are only 65%-75% right.

A better and more cost-effective approach is to step out of that conference room and live with customers for a while. Here's a table offering a hierarchy of investigative approaches for better understanding customers.

Research Method	Advantages	Disadvantages
Online Survey (e.g., Survey Monkey)	Fast, cheap, and easy	A poor format for questioning underlying assumptions or reading visual cues from live responses. Provides at best limited insight into physical, cultural, and political factors that may ease or retard adoption of your offering. Also, requires a good list. If you're a social media expert, and you're surveying the people who follow you on Twitter, you're really learning about yourself, rather than the vast pool of potential customers who are not like you.
Phone Interviews	The more free-form format allows you to dig deeper, explore tangents.	More time-consuming, and you'll likely have to pay the person you're interviewing for his or her time.
On-site interviews and presentations of prototypes at your office	A good chance to see how people really react to what you're building. Richer, more accurate data than you can get from a survey.	Time-consuming and more expensive. Again, you'll likely have to pay people for their time.
Interviews at the customer's site (office or home)	The best way to research a product or service to understand the customer's world. Being there is the best way to gain that understanding. You may also discover other factors and business opportunities that you would have overlooked if you had kept your distance and communicated only through surveys and forums.	Expensive and time-consuming.

What's ironic (and somewhat frustrating) is that founders and investors will often dismiss face-to-face interviews and prototype-testing as too expensive and not worthwhile. Loic Le Meur, for example, for whom I have a lot of respect, once compiled a list of suggestions for start-ups that included:

7. Don’t spend time on market research. Launch test versions as early as possible. Keep improving the product in the open.

I think this approach can work with public applications like Twitter applications and social media networks like LinkedIn. I don't think it will work with more complex applications, such as business workflow or, say, healthcare payment acceleration applications, where the number of variables to get right is high, and the development costs are likely to be significant. And obviously, if your solution involves on-site hardware, giving away free test versions is expensive, even if customers agree to install them, which is doubtful.

Another limitation of this approach is that more or less consigns you to a freemium business model, because it requires lots of users testing a product to refine it. This strikes me as the tail wagging the dog. Companies should select their business model (how they'll make money) independently of how they'll conduct research. I would advise against automatically adopting a freemium model merely to avoid the expense and hard work of upfront research. Yes, by all means, always listen to customers and improve your products and services accordingly (as the freemium model encourages). But adopting the freemium model—or any other revenue and distribution model—should be considered a strategic and game-changing chess move, made at the proper time and the proper circumstances, not simply a default behavior (like P-K4 [a conventional chess move, for you non-chess players]) of young companies.

A third problem with this approach, as I mentioned in an earlier blog post, is that refining over time takes time and money. It might be more cost-effective to sit down with prospects up-front, really narrow down what the product or service should be, then refine it over time, having started from a position much closer to the bull's-eye.

A fourth problem is that some customers are simply not going to try your offering, even if it's free and it looks cool. Good luck getting someone in upper management or on a busy factory floor to even look at what your small, unheard-of company is offering. More likely, you're going to have to approach them, perhaps in the company of a third-party research or design firm, and there's going to have to be a financial incentive. You're going to need to buy their attention, but chances are, the expense will pay off for you handsomely. You'll reduce development costs and bring your offering to market faster, perhaps even with some endorsements from the customers you've interviewed.

The goal, here, is empathy. Not just a list of features, but a real understanding of the environment in which your product or service will be used, an understanding of the pressures your customers are facing, the cultural factors driving or inhibiting adoption of what you're offering, and the best language and visual interface with which to present your solution. No matter how talented a founding team is, they're simply not going to derive all that from a few long days locked in a conference room. Getting in front of customers repeatedly, gauging their reactions to prototypes, and really listening are what's required.

Outcomes

So how did the companies I mentioned back at the beginning end up faring?

• Flying blind. This company's core technology that the business continues, though opportunities have been missed, and a major engineering project (developed by a senior programmer who was unfamiliar with the market) ended up getting scraped. Recently, though, under new management the company upgraded its old flagship product and aced a major product review. So, somehow, they're soldiering on.
  
• Academic certainty. Product delays, in large part caused by outsourcing to the wrong company, resulted in lost sales, dwindling revenue, and dwindling staff. Eventually the company was sold. The product had to be rewritten (mostly from scratch, I believe) to catch up with current technology trends.
  
• Proud of complexity. The impressively complex product never turned out to be a big seller. Fortunately, the company had acquired other related products, and the company's engineering team developed these into the best products in its industry. However, the company itself never became profitable. It had gone public during the bubble. It stock price hovered in the $1-3 range for many years. Eventually the company was acquired.
  
• Experience-focused. Within a few years, this start-up, while still private, was acquired by a major Silicon Valley technology company. A happy ending.