Photo by Michał Grosicki on Unsplash

Over the past year I’ve become increasingly aware of a fundamental divide in the prevailing wisdom on how good products are built. I’m not talking about waterfall vs. post-waterfall methods. I’m also not talking about the differences between specific methods like Agile and Lean. I’m talking about different philosophies on the best way to build products in an already post-waterfall world.

These differences have been apparent for a while, but they came into stark focus for me over the past week, as I finished reading two books in quick succession: Getting Real by Basecamp and Inspired (2nd ed.) by Marty Cagan. Both books are interesting and worth reading by themselves, but even more so when you read them right after another.

Basecamp and Marty agree on the biggest challenges in building good products, but diverge quite often on how they believe teams should deal with those challenges. Let me provide a couple of examples.

The same, but different

Both books are adamant that value (defined as whether someone will use/buy your product) needs to be validated as early and as cheaply as possible, and that the old ways of doing things are expensive and wasteful. Basecamp says you do this by “racing to running software” and making sure that it’s cheap to make changes:

It’s ok to do less, skip details, and take shortcuts in your process if it’ll lead to running software faster. Once you’re there, you’ll be rewarded with a significantly more accurate perspective on how to proceed. Stories, wireframes, even HTML mockups, are just approximations. Running software is real. […]

With real, running software everyone gets closer to true understanding and agreement. You avoid heated arguments over sketches and paragraphs that wind up turning out not to matter anyway. You realize that parts you thought were trivial are actually quite crucial.

And earlier:

Change is your best friend. The more expensive it is to make a change, the less likely you’ll make it. And if your competitors can change faster than you, you’re at a huge disadvantage. If change gets too expensive, you’re dead.

On the other hand, Marty believes in building prototypes really fast, and testing those with real customers before you commit to code:

One of the most common traps in product is to believe that we can anticipate our customer’s actual response to our products. We might be basing that on actual customer research or on our own experiences, but in any case, we know today that we must validate our actual ideas on real users and customers. We need to do this before we spend the time and expense to build an actual product, not after.

And later, on prototypes:

Product discovery [coming up with a validated product backlog] involves running a series of quick experiments, and to do these experiments quickly and inexpensively, we use prototypes rather than products.

On the topic of “functional specs”, both books agree that writing long specs filled with “requirements” is a terrible way to build software. I don’t think any of us disagrees with that. But again, they diverge on the best alternative. From Basecamp:

So what should you do in place of a spec? Go with a briefer alternative that moves you toward something real.

Write a one page story about what the app needs to do. Use plain language and make it quick. If it takes more than a page to explain it, it’s too complex, The process shouldn’t take more than a day.

Then begin building the interface — the interface will be the alternative to the functional spec. Draw some quick and simple paper sketches. Then start coding it into HTML. Unlike paragraphs of text that are open to alternate interpretations, interface designs are common ground that everyone can agree on.

Marty favors a technique called the Opportunity Assessment for the vast majority of projects:

The idea is to answer four key questions about the discovery work you are about to undertake:

1. What business objective is this work intended to address? (Objective)
2. How will we know if we’ve succeeded? (Key results)
3. What problem will this solve for our customers? (Customer problem)
4. What type of customer are we focused on? (Target market)

[…] You need to ensure that every member of your product team knows and understands the answers to these four questions before you jump into your product discovery work.

Many ways to skin a cat

To summarize this another way: most people in this post-waterfall world agree that the biggest reason why software projects fail is that various risks are assessed too late, which ends up being too costly for the business to survive. Most even agree on the core principles to follow to fix this: tackle risk as early and as cheaply as possible. Where we are seeing the divide is in how this should be done.

One perspective is what we can call the prototyping movement (I’m deliberately staying away from naming specific methodologies). The goal is to align around business objectives and build functional prototypes to meet those objectives as quickly as possible, and test those with real users before engineering gets involved.

The other perspective, the real software movement, says that even that takes too long, and that nothing can replace the feedback you get from working software in production — as long as you’re able to make changes very quickly.

So which is it?

In our search for easy answers and silver bullets, the obvious next question here is, “ok, so who’s right?” But I think good product managers eschew such easy answers. Good product managers are always learning about different perspectives, but they have to learn through an added dimension—the lens of their own product and culture.

So for me, the real takeaway from these books hasn’t been the prescribed solutions — although those are certainly helpful as idea starters. The real takeaway has been all the roadblocks to good product development that I noted down as I was reading. The different kinds of risk we need to validate. The constraints we tend to miss when we brainstorm and plan. The heavy processes that do nothing more than slow teams down and make them unhappy. How to address those challenges in a particular culture is where the true art of modern product management lies, and what makes the job itself so difficult to pin down, define, and get good at.

I’ve learned a great deal about product, culture, and teams this year. But no lesson has been more valuable than this: the challenges to building good products are universal, but the solutions are not, and the biggest value I can add to the team is to work with them to figure out the best way for us, in our context to address these challenges in a way that ensures the team is happy and productive, and our customers love our products.

I’m guess if I have to make a New Year’s Resolution, getting better at this would be it.

As a product manager I know and understand the importance of making customers part of the product development process through research and interviews. Especially those of us who come from a design background have this philosophy deeply engrained. We know that “I am not the user” and we have the t-shirts to prove it! So it is with some surprise that I recently realized that sometimes — when the circumstances are conducive to it — it’s ok to trust our instincts and create products and features without talking to customers directly about it first.

See, the thing is, talking to customers isn’t something we do, it’s something we are. And if it’s something we are — if we really are immersed in our customers’ needs and behaviors and emotions — we should feel comfortable to trust our own instincts a little bit more.

With this kind of immersion comes an ability to channel our customers in a way that drastically reduces the additional benefit we might get from interviewing them about a specific issue or feature. When we not only have the knowledge of the domains we work in, but also a good understanding of how our customers navigate those domains, we end up with a powerful foundation to base our decisions on.

Does this mean we don’t need research? Of course not! But it means that maybe we don’t need to go out and interview users every time we make a product change or introduce a new feature. It means maybe we do usability testing on major changes to the site, but not when we fix something that we’ve lived and breathed with our customers for months or years.

Those are weird sentences to write. I am a big proponent of User-Centered Design, and obviously research is a central component of that. But what I’m advocating for isn’t less research. I’m saying that it’s possible to reduce the amount of structured research you do, if you have a culture of customer immersion in everything you do.

Customer immersion isn’t an easy culture to create, but it is very much worth it. As a start, everyone in the organization should be encouraged and empowered to talk to customers — whether that is through phone calls, support cases, conferences, or any other way you might be able to reach them. And since not everyone will be able to spend an equal amount of time with customers, it also means you have to listen to those who do spend a lot of time with them — and trust that they are acting as good conduits for customers’ needs.

Making the right choices about when to do structured research and when to trust your (informed) instincts will save you time and money — and make customers happy too. That’s not a bad combination of benefits.

Questionable use of stock photography aside, Colin Lernell’s Is the Product Requirements Document Dead? A Debate. brings up some interesting points. I’ve long argued that if you do it right, a good “product spec” is essential to successful product management and development. We’ve come a long way since the original concept of a 45-page PRD that no one reads (not even the person who wrote it). The format has evolved as we’ve grown accustomed to leaner development processes.

One of Colin’s suggested alternatives to the dreaded PRD is what he calls an MVPRD. I don’t like the term (the overuse of the MVP concept makes it all but useless these days). But the approach is one I agree with:

Write your first MVPRD in a short, limited amount of time (just enough to communicate to your team and start work) to avoid bloat. As you move toward or through development, meet with your team frequently to assess and iterate on what should go into the document and what should be taken out. Remember to keep it lean and that your intent is to figure out what you need in the document and if you need it at all.

Our product specs at Postmark follow a similar journey. At the beginning of a project I start a new spec document from a template in our wiki. I fill out as much of the basics as possible, which in our case consists of the following:

• Basic metadata like who the team is, links to technical specs, etc.
• The job story.
• A stripped down version of Marty Cagan’s Product Opportunity Assessment framework.
• Known dependencies and risks (such as customer support and back-office systems).

We use that information for our team kick-off call, and then we start filling out the rest of the document as we go. We also go back and make changes as we get further down the road and make decisions. We have three main sections for this part of the document:

• The solution overview provides a broad description of how we intend to solve the problem we identified in the Product Opportunity Assessment
• Sketches, Wireframes, and Prototypes links to the the initial design assets we create, in whatever fidelity is right for the size and scope of the project
• Final design has the final assets that are needed for implementation. In our case this is most often a completed front-end.

The important thing to remember about this fluid version of a product spec is that it is not a document you write from front to back at the beginning of a project and then never look at again. It’s also not something that’s every really done. It’s a document that starts small and that you keep working on as you go.

The benefits of doing it this way is that it not only helps you document all the little decisions you make along the way as you build a product, it also builds up organizational knowledge to remind you of why the team made the decisions they made. Because you will forget otherwise. So if you’re developing without a blueprint and feeling some pain and chaos, give this approach a shot. It might just provide the relief you need.

There is essentially two ways to improve at your job. You either get better at the things you should be doing, or you learn to stop doing the things you shouldn’t be doing. We tend to spend a lot of time and effort on the first aspect—and for good reason. It’s absolutely essential to keep learning. But lately I’ve been consumed with that second part. Day in and day out, at the most inconvenient times, the same question jumped into my head:

What are the biggest mistakes I’ve been making as a product manager, and how do I stop doing those things?

I took some strange detours trying to answer that question. And in the end, the answer I came up with specifically for product management had its origin in an unlikely place: some graduate work I did almost two decades ago. So let’s take a quick detour into information science theory before we return to the matter at hand…

One of the first things you learn about knowledge management is what’s called the DIKW pyramid. It provides a model for thinking about the transformation of raw data into something more useful.

• Data refers to facts and observations. They are the spreadsheets and SQL query outputs that come across our desks on an almost daily basis.
• Information is inferred from data, and distinguishes itself from data in that it is actually useful for decisions or action. Company dashboards with business metrics like MAUs are information.
• Knowledge refers to a framework for decision-making based on information, i.e. “we believe that when this happens, we should do something about it”.
• Wisdom is illusive—both in definition and in… well, attaining it. I’ve always liked the definition that says wisdom is knowledge with judgment. Wisdom goes beyond having a framework for what to do, and describes having the judgment required to make the right decisions based on the information and knowledge available.

The model isn’t perfect, but it serves a valuable purpose. There are two aspects in particular that I’ve always found useful to remember.

First, to extract value from data it needs to be transformed into something more valuable, and you don’t get to skip phases. If, for example, you try to go straight from data to knowledge without first organizing the data as information, your knowledge isn’t going to be all that accurate.

Second, confusing one slice of the pyramid for another can be really dangerous. If, for example, you’ve been able to extract some knowledge from information, but you think you’re looking at wisdom, you’re going to make some bad decisions, because you haven’t taken the time to add the necessary judgment to the information in front of you.

With that as background, let’s get back to product management. If I think about the times when I’ve made my biggest mistakes as a product manager, it can all be traced back to one of two causes:

• I misidentified data, information, knowledge, or wisdom. For example, if someone on the team who is steeped in our product and its users comes forward with a wise suggestion about where to take the product next, and I jump in with a process to take what I think is some information they provided and turn it into knowledge, I’m operating at the wrong level. Turning information into knowledge is essential, of course (remember, you can’t skip steps). But trying to pull wisdom back to an earlier phase in the transformation process is disruptive and demoralizing. We shouldn’t do that.
• I communicated the wrong slice of the DIKW pyramid to the person or team I was speaking to. For example, let’s say a designer and I go through an extensive usability testing process on a prototype of a new feature. We collect data, we group that data into information, and then we discuss it to extract the knowledge and wisdom we need to make the appropriate changes to the product. The difficult part is knowing what to communicate to who. For some on the team, the end product (the “wisdom”) is good enough. Others, particularly those with good data transformation skills, might prefer to see all the information so that they can give feedback on the knowledge and wisdom we landed on. We need to know the difference of what’s needed by who, and share appropriately.

Knowing the mistakes you tend to make is only half the battle, though. So all of this led me to a statement I’m printing out and putting up above my desk, to help me avoid making those mistakes:

An effective product manager shepherds data from a variety of sources through the transformation of becoming information and knowledge and wisdom. They do so without getting bogged down in unnecessary process, and they only share the relevant parts that each person or team needs to make progress.

I’m sharing this here with the hope that it will resonate with some of you who may have been grappling with the same questions.

There’s one more important point I want to make. From my personal life I know the dangers of wallowing in introspection for too long, so I don’t intend to stay down here in mistakes-land. Wrestling with these questions was a helpful exercise, but it’s not a place I want to get stuck in. So I’m going to head out to that other place for now. You know, the one where we get to learn new things and improve our skills. Maybe we’ll see each other on the road.