Dhanji Prasanna wrote a great article about his experiences on the Google Wave team, and the difficulties of working in large development teams. He brings a particularly interesting perspective to UI engineering:

To say we should have been better prepared or organized is to miss the point - large teams starting on a new project are inherently dysfunctional. One common consequence of all this chaos is that experienced engineers seclude themselves to their area of expertise. At a company like Google, this generally means infrastructure or backend architecture. A major externality of this is that fresh grads, and junior engineers are shunted to the UI layer. I have seen this happen time and again in a number of organizations, and it is a critical, unrecognized problem.

UI is hard.

You need the same mix of experienced talent working in the UI as you do with traditional “serious” stuff. This is where Apple is simply ahead of everyone else - taking design seriously is not about having a dictator fuss over seams and pixels. It’s about giving it the same consideration that you give any other critical part of the system.

I’ve experienced this first-hand, and I’ve also seen what happens when backend developers are forced to do UI work (which can happen for a variety of reasons). I’ve heard developers say that they don’t like to do UI work because “it’s not real programming”. They prefer to focus on the real stuff, not this fluffy CSS/JavaScript thing.

Whether or not their perception is accurate is only one part of the discussion. What I want to point out is this: If you make backend developers do front-end work that they’re not passionate about (or worse, work they find embarrassing to do), they’re not going to be motivated to expand their knowledge and do a good job. That’s unfair to everyone and disastrous for the product.

It’s essential to have dedicated UI engineers in an organization so that everyone can focus on the technologies that they’re obsessed with.

Jason Mesut made quite a few waves this week with his presentation Truth and Dare - Out of the echochamber into the fire. It’s definitely worth your time so I recommend you click through and read it before you continue here. Ther’s a lot to like and a lot to think about.

Jason explicitly asks for feedback and counter-arguments, so I do want to address one slide in particular, shown below:

Now, I might be putting the puzzle together wrongly here, but since these slides came out right after I published a two-part series on Smashing Magazine called “Breaking Down Silos”, I’m going to assume h’s talking about those articles. If my assumption is wrong this is going to be really awkward, but oh well.

So, let’s look at why Jason is calling this concept “naive”. I wasn’t at the presentation, so all I have to go on is his bullet points.

Organisations are complex

Of course organizations are complex, and if anyone tries to argue otherwise they’ve never worked in one. But I never said that this is simple:

There are no shortcuts to breaking down silos. You can’t fix the environment if the organization doesn’t understand the problem. You can’t improve the development process if the right environment doesn’t exist to enable healthy guidelines. You have to climb the pyramid brick by brick to the ultimate goal: better software through true collaboration.

I don’t propose “7 steps to a happier you” in the article. I propose a process of understanding the problems and unique needs of the organization, followed by a tailored solution that takes those unique needs into consideration.

People are better in small groups

I absolutely agree, and that’s why prioritization at an organizational level needs to take this into account and empower small teams to do the work without interference. Her’s what I said in the article:

[Once strategic priorities are set], projects would move to small dedicated teams, which would have complete ownership of the design and implementation. The product council sets the priorities, not the details of implementation”Š””“Šthose are up to the teams themselves.

I go on to talk about the importance of autonomy and the meaning people find in their work when they work in these small groups.

Change takes too long

I don’t understand the argument here, so maybe this is one of those “voice-over required” points. But if the argument is that change takes too long so we shouldn’t even try, I don’t buy it. Her’s how I end the article, again acknowledging how difficult it is:

Building collaborative environments is not easy, because change management is not easy. But the positive outcomes of doing this far outweigh the pain of making it happen. You’ll end up with happy, creative teams that feel a sense of ownership over what they’re building and a sense of pride in its quality.

I’d also like to point out that I wasn’t being academic in these articles. Everything I wrote is based on principles we’ve tried and applied in real life in the organizations where I’ve worked. There’s always room for improvement and growth, but this wasn’t a theoretical exercise.

I know this doesn’t matter that much in the bigger scheme of things, and I admit that the only reason I’m even writing about it is a slight irritation with the word “naive”. But if Jason is indeed referring to my article (again, this is going to be really awkward if he’s not) I at least wanted to clarify my viewpoint.

So there that is.

A couple of weeks ago our 2-year old daughter threw my wife’s phone in the swimming pool. The resulting journey through the Vodacom customer service labyrinth to replace the phone was frustrating, but it also gave me a new level of understanding and empathy for the immense challenges of providing customer service to hundreds of thousands of people.

This is an article about social media, customer support channels, and the principles every company should establish in their culture to serve their customers better. And (spoiler alert!) I do manage to get a new phone for my wife.

”Umm, So, Our Daughter Threw My Phone In The Pool”

What’s most surprising about getting a call about my wife’s phone suddenly finding itself at the bottom of our pool is how completely nonplussed I was about the whole thing. When you become a parent the kinds of things that upset you change significantly. I think I’ve discovered a pattern: if there is no blood involved, there’s really no reason to get upset. So after establishing that there was no blood involved, I proceeded to the next step - trying to replace the phone.

My wife had an LG Generic (or whatever it was called) on one of Vodacom’s cheapest plans, and the thing has been driving her nuts. She’s had her eye on my iPhone for a long time, so I decided to try to upgrade her. The problem is that I’m not eligible for an upgrade until the end of December. And that’s where this journey starts.

My first step was to walk into a Vodacom store to ask for assistance. This is pretty much the extent of the conversation that took place with the support representative:

Me: “Hi. My daughter threw my wife’s phone in the pool, so I’d like to get her an iPhone please.” Rep: “Your contract isn’t due for an upgrade until the end of December.” Me: “I understand that. I’m saying that my wife’s phone is now wet and doesn’t work any more, so I would like to give you more money by going onto a more expensive plan.” Rep: “It’s against policy to do an early upgrade. That’s why you should insure your phone.”

Imagine that conversation with a “Sucks to be you!” look on the representative’s face, and you’d have a really good idea of how it went down.

Having failed with the first point of contact, I took to Twitter:

The response was very quick, asking me to DM my number so that someone could call me. I sent my number, and a representative called me the next morning. I thought this was getting somewhere, and I was already starting to write this post in my head. My headline (“Social media works!”) needed some work, but it was going to be great.

But not so fast”¦ I told my story to the representative, who looked up the account and told me the same story: “Sorry, it’s against policy.” (At least this time someone was sorry about it). I threw out what would become my standard line throughout this process: “You realize I’m trying to give you more money, right?” But no luck. The conversation ended when the rep told me, “I will ask the upgrades department if there is anything we can do.” Translation: “You’ll never hear from us, ever again.”

After not hearing from the “upgrades department” I sent another DM, and got a call from another rep. Same story. Against policy. “You realize I’m trying to give you more money, right?” Sorry, against policy. I then took it to the next level and told the rep that I will be taking my business to MTN, convinced that this statement would trigger some script alarm somewhere and get me a free ticket to a ride up the “escalation path”. Not so much.

Rep: “Oh. Well that’s not good.” Me: “No, it’s not. Anything you want to do about that?” Rep: “Well, this is our policy. Can’t be changed.” Me: “You don’t want to tell someone that I’m about to take my business elsewhere?” Rep: “I’ll make a note in the system.”

At that point I gave up and decided to wait until I am eligible for an upgrade. That decision lasted about 3 days. I decided to give it one last shot, and tweeted Vodacom’s CEO:

And this is where the story gets boring, in a good way. Pieter Uys tweeted me back (in my first language, which means he looked at my profile and thought before responding). 4 hours later I got a call to say we can do the upgrade. End of story. No questions, no statements about policy. I can do the upgrade = happy customer + more money for Vodacom.

Your Call Is Not That Important To Us

Before moving on to the main point of this article I want to tell another quick story. I’ve been banking with ABSA all my life. I’ve also been unhappy with ABSA all my life, but that’s a story for another day. I recently mentioned ABSA on Twitter and linked to this post. The post got retweeted a few times, and then I got this:

That really interested me. Here is a bank (FNB) that monitors what people are saying about their competitors, and joins the conversation in relevant ways. Notice that he wasn’t pushy, he was merely getting in on the joke. I tweeted back:

I said this would be a short story, so I’ll just say this. One week later someone from FNB was sitting with me, filling out forms to transfer all my accounts from ABSA to FNB. They took care of the whole thing, I didn’t have to fill out a single form. All because of a tweet. And I’m pretty sure ABSA doesn’t even know (or care) that they lost another customer.

It’s All About The People

All customer support revolves around people, processes, and tools.

CRM and community tools like Salesforce, Get Satisfaction, and Twitter give support reps the means to communicate with customers. Processes set guidelines for what those interactions should be like. But all of that is useless unless the people doing the support understand and live out the culture of the organization. The ease of establishing that culture also depends a great deal on the support channel used.

Synchronous, 1:1 support like in-store interactions and phone support is expensive and extremely difficult to manage. Unless you’re Zappos and call yourself “a support company that happens to sell shoes”, most companies don’t have a deeply ingrained support culture. So it’s very hard to filter the right processes and culture through to the 1:1 support channels, since they are generally pretty far removed from “management”. They are therefore very rarely empowered to make decisions that might not follow policy, but would be the best thing for the customer (and the company).

I would argue that my early upgrade situation is a good example of this. That representative in the store should have been empowered to ignore policy and upgrade me on the spot. It’s not her fault that she’s not allowed to do that, it’s just the way it is.

On the other hand, asynchronous, 1:many support like live chat, online forums, and social media platforms are much cheaper, and I would argue also easier to manage from a support culture perspective. You’re able to set appropriate guidelines (more on that later), and in general the people who manage those channels have a much more direct path to different resolution scenarios (and therefore more decision-making power).

All this to say that I am not upset any more about my bad experiences in the store and initially on the phone. Because I recognize how incredibly difficult it is to nurture a true culture of customer-centric support. And to find that balance between empowering everyone in the company to break policy when they feel it’s needed, while still having enough process in place so you don’t give away control of your short-term and long-term business strategy.

I don’t have an immediate solution for this, but I want to write about it because I believe it’s a very real problem that a lot of companies are struggling with. Especially now that social media support channels are getting so much adoption.

Lessons In Customer Service

Even though I don’t have the perfect answer, I do want to spend a little time discussing some recommendations I have for better customer service, based on my recent experiences with FNB and Vodacom.

1. Understand what engagement really means

There is no substitute for authenticity. When Pick n Pay asks what I’m going to be doing today, it doesn’t feel like real engagement. Why would I want to tell a supermarket that? When Vodacom sends me the scripted answer “I heard about the problem you experienced”, that tells me they didn’t really take the time to think about the response when sending it (“Well, of course you heard about it, I sent you a tweet!”).

When FNB joins a conversation in a natural way, or when the CEO of Vodacom responds to me by name - that’s engagement. It’s such a simple rule: read, think, respond like a human.

2. Web governance is essential

Web governance “defines decision-making processes for the web, and sets policies and standards for web content, design, and technology””in a way that respects subject-matter expertise” (from Web Governance: Become An Agent of Change). Defining user-centered standards for every touch point with an organization is enormously important to those who want to succeed, and it’s not getting enough attention at all.

One part of web governance that needs more attention in particular is content strategy, which “plans for the creation, publication, and governance of useful, usable content” (see The Discipline of Content Strategy for more). Among other things it defines the tone and language and underlying principles for talking to customers. Every company should do this before they open their Twitter account or create a Facebook page. (Btw, if you’re in South Africa and need help with stuff like this, talk to Kerry-Anne)

How you talk to your customers makes a huge difference to their experience, and if you don’t define a strategy for it, your community will define you and you’ll have no control over it. That’s not a good place to be.

3. Empower support representatives

I want to come back to this. As mentioned earlier, I recognize how difficult it is to walk the line between empowerment and total loss of control. But I think there are ways to test this out as a strategy without giving the whole house away.

Start with one specific department, call center, or representative. Allow them to make some decisions based on what they feel is right for the customer and the company, and see what happens. If they break some rules/policy, ask them why they did it, and follow up with the customer to see how they felt about the exchange.

This kind of empowerment isn’t a binary switch for the whole organization. Start small, test, and see if it might be possible to build a culture that encourages doing The Right Thing.

All’s Well That Ends Well

My story had a happy ending. But I know there are an enormous amount of customer support stories that don’t end that way. The rise of cheaper, more efficient channels for customer support can make experiences better not just for customers who engage in those channels, but for everyone. We can take the lessons from the asynchronous channels and apply them to the 1:1 interactions.

Be authentic, get in on the joke, and break some rules every once in a while. Because they did that, FNB has a new customer and Vodacom didn’t lose one. I think that makes it worth it.

In this post I argue that we need to communicate the differences between the science and art of Visual Design better to help change the common perception by stakeholders and clients that user experience is purely subjective.

One of the most difficult aspects of visual design is finding the right science:art ratio to accomplish user goals. I’ve always subscribed to what Tim van Damme calls the mathematics of design. You start with the science:

If art is about talking and expressing yourself, interface design is about listening and disappearing into the background. You listen to the content and its context, and take it from there, one step at a time. Don’t worry about the looks, just start with the variables. 1 + 1 + 1 + ”¦ Baby steps, over and over again until what you have on your screen feels right.

And then you mix in art where appropriate:

But sometimes, even 1 + 1 is too much to handle, and you need to clear your head. This is where art comes into play, in the broadest meaning of the word: Paintings, illustrations, architecture, human beings, even nature is art. They won’t help you decide whether you should draw a 1 or 1.5 pixel highlight, but allow you to take a step back and just decide on what’s more suitable or pick one and move on.

Of course, this is not a serial process. Great designers are able to design within that delicate balance between science and art, and find the right ratio as they’re doing it. And even though it’s not easy, I do feel that most designers inherently get this - that visual design is science and art combined in different levels based on the needs of the user and the application.

What’s even harder is explaining this to stakeholders and clients in a convincing way. Over the past week I’ve seen so many comments about how “UX is subjective” and “standards always change” that it got me thinking about a possible solution to this problem. I haven’t figured it out, but I’d like to write down some initial thoughts for discussion.

The problem with Visual Design

I think as a UX community we’ve done a good job of splitting out the different elements of UX Design. Stakeholders and clients are slowly starting to understand the difference between Information Architecture, Content Strategy, Interaction Design, etc. And most people also now understand that those functions are not just gut feel or whatever is the trend of the day. We’ve done a decent job of showing the evidence behind the decisions we make - thanks in large part to the results of user experience research methods like ethnography and usability studies.

But Visual Design is the odd one out in this equation. It walks the line between science and art so tightly that most stakeholders and clients only see the art part. So they look at a design, make a gut call, and think that it’s all just whatever style the designer fancied on that particular day. Sure, some of it is our own fault, and many designs don’t have enough science at all. As Zeldman pointed out:

When Style is a fetish, sites confuse visitors, hurting users and the companies that paid for the sites. When designers don’t start by asking who will use the site, and what they will use it for, we get meaningless eye candy that gives beauty a bad name ”” at least, in some circles. Not enough designers are working in that vast middle ground between eye candy and hardcore usability where most of the web must be built.

We have to find a better way forward.

Breaking down the elements of Visual Design

So how do we fix this? One way is to provide a much clearer distinction between the different aspects of visual design. I’m not saying we should split the job title into two functions, I’m saying we should be more explicit about the goals and outcomes of visual design. And it needs to be simple, so it can’t be too detailed. I’m not 100% sure how to do that yet, but here is one suggestion:

• Hierarchy Design could refer to decisions made during the design process that sets the appropriate visual hierarchy based on the scientific principles of visual perception (such as contrast, grouping, balance, symmetry, etc.). See Designing for the Mind as an example.
• Aesthetic Design could refer to decisions made during the design process to help the design fit the brand promise and elicit an appropriate emotional response (such as choice of color, typography, button styling, etc.). See In Defense of Eye Candy for more.

Now, as I already mentioned, there is a lot of overlap between these activities, and you can’t have one kind of visual design without the other. But there has to be a way for us to talk to our stakeholders and clients about the visual layer of design that is not based on style preference but on “hardcore usability” as Zeldman puts it.

As we continue to grow and define the different elements of user experience I believe that Visual Design has the most baggage to overcome simply because of the history of web design and its initial focus on what’s pretty vs. what works. What works is not subjective, and we need to communicate that effectively to our stakeholders and clients. It’s not their fault for not “getting it”, it’s our fault for not explaining it properly. Let’s change that.

Over on the 37signals blog they just reposted an old article entitled Product roadmaps are dangerous. Jason Fried says the following:

Instead of the roadmap, just look out a few weeks at a time. Work on the next most important thing. What’s the point of a long list when you can’t work on everything at once anyway? Finish what’s important now and then figure out what’s important next. One step at a time.

It’s hard to disagree with a person (and a company) you have great admiration for, as I do for Jason and 37signals. But I do think it’s important to set the record straight on product roadmaps - particularly when it comes to large organizations. The post highlights two main concerns with product roadmaps:

• Product roadmaps assume you know what’s going to happen 6 - 18 months from now
• Product roadmaps set expectations, so you can’t change them (and if you do change them it becomes a worthless exercise)

So let’s look at each of these points in turn.

Product roadmaps assume you know the future

Jason writes:

When you let a product roadmap guide you you let the past drive the future. You’re saying “6 months ago I knew what 18 months from now would look like.” You’re saying “I’m not going to pay attention to now, I’m going to pay attention to then.” You’re saying “I should be working at the Psychic Friends Network.”

This is not what a product roadmap is, or what it’s supposed to do. The purpose of a product roadmap is to set forth a long-term vision for the business, and break that up into smaller, meaningful pieces of work, based on what you know now. It’s fallacy to believe that this is an unchangeable list of dates about where the business is headed. A product roadmap that doesn’t react to day-to-day changes in the market and within the company is a pretty dumb document.

At my organization we are very clear that the product roadmap is a flexible guideline that can (and must) change frequently as needed. But it gives the teams (and the management team) something to work towards. It’s a common vision, a sense of direction that’s more than just fluffy language - it’s concrete evidence that we’re headed somewhere good, and we know how to get there.

We can change direction as many times as we want. This doesn’t make it a useless exercise, it means that instead of starting fresh on a new “roadmap” every few weeks, you build on your past successes, don’t make the same mistakes twice, and keep making measurable progress since you can see where you came from.

Product roadmaps set the wrong expectations

Elsewhere in the post:

The other problem with roadmaps is the expectations game. People expect you to deliver what you say you will in 4, 5, 6 months. And what if you have a better idea? What if ther’s a shift in the market that you need to address? What if what you thought wasn’t what actually happened? Any change in the roadmap nullifies the roadmap. Then the map isn’t a map at all.

If you have this problem it doesn’t mean that product roadmaps are wrong, it means that you’re doing it wrong. As long as everyone in the organization buys into the fluid nature of the roadmap, you won’t have this problem. In our organization we do this mainly through the mechanism of what we call the Product Council (I was partial to Intergalactic Product Force, but for some reason that didn’t fly so well). Here’s how it works.

The Product Council is made up of the heads (VPs) of every department in the organization: Engineering, Marketing, Support, Category, etc. This body has a weekly meeting where we discuss the current product roadmap and priorities. We ask ourselves, Are we still working on the most important things? If something more important comes up, we prioritize it higher in the roadmap, and something else shifts down. If we’re happy with the direction, we do nothing. If a new opportunity arises we ask ourselves, Is this more important than what we’re working on right now? Or is this something we should work on next? If so, what moves down in the priority list?

From here, I communicate with my Product Team about any changes, and we discuss this to make sure no one missed anything. But then - and this is important - the Product Manager has complete autonomy and ownership over the implementation of the roadmap. The Product Council sets the priorities (with input from all parts of the organization), but the Product Managers work with their development teams (and others) to set the timeline, the implementation details, the design, everything.

There’s more to it, but in the interest of brevity I’ll stop there. This process has three main advantages:

• It gives the management team complete transparency into what the Product team is working on, and it allows for anyone to make the case for a change in priorities. Why this kind of transparency is so essential is a subject for a different blog post, but in short, it takes away a vast majority of the politics you see in many organizations, and it frees up the teams to do what they do best - execute.
• It prevents scope creep. Nothing can go on the roadmap without something else moving out or down. As anyone who ever worked at a large organization knows, this is an absolutely critical part of a successful development cycle.
• It gives the Product Manager and their teams what they need to be successful: direction and autonomy. As Jocelyn Glei says: “Give your team members what they need to thrive, and then get out of the way.”

Why product roadmaps are safe (and essential)

At a practical level I went through the exercise of figuring out how we could execute in my organization without a roadmap. And I just can’t see it. Changes to current pages/flows affect changes we’ll make down the line - I have to think about that.

If you’re serious about frequent incremental change as opposed to large redesign projects (as we are), you can’t live without a roadmap because you’ll have no idea how far you’ve gone, what you still need to do, and what’s more important than something else. And perhaps most dangerous of all, everyone in the organization will come to you and want all their projects done right now, and you’ll have no systemic method for dealing with that in a way that’s best for the business.

Andy Wagner summed up my feelings on this issue quite succinctly in a comment on the 37signals post:

[Product roadmaps are] an opportunity to dream about what the future might look like so that as you make your day-to-day responses to the customer, you can do so consistent with building the future state. It emphatically should not be anything to be slave to, it should be dynamic and notional, not static and specific.

Jason says, “The further you get from now, the less you know. And the less you know, the worse your decisions will be.” We agree on that. My argument is that without a roadmap you only see now. And if you only see now without seeing yesterday and tomorrow, you don’t see a whole lot. And “the less you know, the worse your decisions will be”.

About a week ago I had a discussion with a colleague in our development team about the new product development process we rolled out a few months ago. One of the words he used to describe the new process is fair.

It was a passing comment and I didn’t really think much of it at the time, but since then I keep going back to that conversation, and the importance of fairness in the product management profession.

Loads of articles have already been written about the characteristics of a good Product Manager (see the end of this post for links to some of them). They’re all fantastic, and I go back to them constantly when I’m in the hiring process or looking for personal development areas. But I’ve never been able to find the one characteristic that I think a PM simply cannot do without. I now think that fairness is that characteristic. Here’s why.

The role of fairness in effective product management

Let’s look at one definition of the word fair:

fair. adjective. free from favoritism or self-interest or bias or deception.

Free from favoritism

One of the fastest ways to become ineffective in a PM organisation is to start playing favorites with a particular internal group, product line, or user base. As soon as people sense that you are not looking at all ideas and input equally and fairly, a lack of trust inevitably follows. And without trust, you’ll have to work a lot harder (and longer) to bring people along for the ride on your roadmap.

Free from self-interest

If you start doing things purely with reasons like “because I want to” or “because I’m being measured by this metric,” that same lack of trust happens very quickly. You cannot be effective by nursing your own pet projects and ignoring all the other needs around you.

Free from bias

This often happens when PM’s get news they don’t want to hear — especially from the user research team. If something doesn’t test well, don’t make up reasons why you are right and the users are wrong. Do the right thing and realign the design.

One of the hardest skills for a PM to learn is to take their own emotions and feelings out of the equation when it comes to decision making. Yes, a lot of gut feel goes into a product vision, but that cannot be based on personal preferences or preconceived ideas.

Free from deception

This one seems obvious, but you see it often, especially when it comes to metrics and assessment. Don’t ignore/distort negative data or make a problem someone else’s fault. The PM’s job is to own a product — and this means owning its successes and its failures. You’ll gain trust and respect if you not only claim the successes, but also acknowledge the failures and commit to do it better next time.

The elements of fairness in product development

The PM role is often referred to as “the great diplomat,” and with good reason. It is our responsibility to balance a variety of needs from inside and outside the business, and somehow turn that into a roadmap that delivers business value as well as meets user needs. A focus on fairness accomplishes that goal:

• Fairness to users. Approach users with respect, openness and transparency. Understand their needs, and explain to them when you’re doing something that makes it more difficult for them to meet those needs. (Don’t be like Twitter and hide behind reasons like “consistent user experience.”)
• Fairness to the business. Do everything you can to understand the needs of marketing, merchandising, customer support, etc. Pull them into the planning process, be clear about how projects are prioritised, and help them adjust to that process so that they can define their project goals in the right way to get on the roadmap.
• Fairness to technology. Don’t force the development team to make the technology do things it’s not capable of doing. Understand the technical debt in the organisation, and work actively to make those improvements part of regular development cycles.

A lot of this comes naturally in good product managers, but we need to be conscious of it every day. Fairness is a table stakes characteristic for an effective product manager. If you do it right, the real work of building great product can begin. But if you don’t, you’re dead in the water, working with a team that has no reason to trust that you’re doing the right thing.

More characteristics of good product managers:

• Recruiting Product Managers
• The desirable characteristics of a Product Owner
• Owning an Agile Product

In this post I make the argument that companies should hire designers that have a strong foundation in psychology and design theory. I further make the case that when Product Managers and other UX practitioners give feedback on designs, they should move away from their personal visual preferences and instead focus on feedback that relates to the user and business goals they are trying to meet.

I recently had a discussion with an Interaction Designer friend of mine about the perils of Design by Committee.  His take on it was, in essence, as follows (paraphrased a little bit):

The problem is that everyone thinks they’re a designer. Not everyone can code, so they don’t go to developers telling them their markup needs to be more semantic or more valid. But everyone has a gut feel about design - they like certain colors or certain styles, and some people just really hate yellow. So since everyone has an emotional response to a design, and “it’s just like art,” they think they know enough about design to turn those personal preferences into feedback.

It is a real shame that we have come to this point where design just means “making things pretty” for some. Even among people who do understand it’s about more than that, there is sometimes this misconception that there is so much personal taste involved that “your guess is as good as mine.” And to add insult to injury, they way that feedback is given to the designer is often more destructive than helpful. Look, I’m not against design feedback at all - it’s an essential part of the process. But if you’re going to give feedback, do it right. More on that a little later.

The truth is that design is science. Yes, of course it’s a creative discipline, and every designer has a style they like, and strong ideas on aesthetics.  But it is important to understand that aesthetics is the last mile of design. Before that (if you do it right) comes hours and hours of analysis and thinking, resulting in solutions that are built on very real psychological principles, easily proven using the user experience validation stack.

The major problem with focusing too much on aesthetics (and our personal views of it) in design is this: it makes the site harder to use.  Jeffrey Zeldman’s excellent 1999 piece Style versus design places some of this blame on designers themselves, but it sums the problem up perfectly:

When Style is a fetish, sites confuse visitors, hurting users and the companies that paid for the sites. When designers don’t start by asking who will use the site, and what they will use it for, we get meaningless eye candy that gives beauty a bad name ”” at least, in some circles.  Not enough designers are working in that vast middle ground between eye candy and hardcore usability where most of the web must be built.

So let’s be clear. Design is not about what feels good to you. Design is about meeting business goals. It’s about solving user needs so that your business can make more money. In Mike Monteiro’s recent post on Giving Better Design Feedback he builds on this idea and gets to the crux of it:

First rule of design feedback: what you’re looking at is not art. It’s not even close. It’s a business tool in the making and should be looked at objectively like any other business tool you work with. The right question is not, “Do I like it?” but “Does this meet our goals?” If it’s blue, don’t ask yourself whether you like blue. Ask yourself if blue is going to help you sell sprockets. Better yet: ask your design team. You just wrote your first feedback question.

This madness has to stop. I’m not arguing that we should create ugly web sites. In fact, as long as the focus is first and foremost on meeting user needs and business goals, beautiful sites can make the experience simply wonderful. But I am arguing that those of us in the Product Management and User Experience field need to fight for three things:

• A design culture that is routed in theory, analysis, and critical thinking.
• A company culture that hires designers with strong roots in theory and analysis
• A managerial culture that trusts their designers to do their jobs.

So how do we as Product Managers and UX practitioners help affect this change?  We need to gain a much better understanding of what goes on under the hood of design. We need to do this so that we can give better design feedback, but also for the sake of the business in general, so that we are able to make a strong argument for what’s right when it comes to design culture. And lucky for us, there are plenty of great resources online to help us learn how to objectively look at design, and understand the science behind it. Here is my initial suggested reading list:

• The psychologist’s view of UX design is an incredibly detailed look at user behavior on websites, with additional outgoing links to more great content. I go back to this post often.
• Designing for the Mind explains how the brain interprets visual information, and how that translates to web design that works (and is also aesthetically pleasing).
• Gestalt Principles Applied in Design goes further down the path of using established psychological principles to create better designs.

But just how important is it that we solve this problem? In a scary, almost prophetic way, Zeldman ends his post on style vs. design with the following words of caution:

Most of all, I worry about web users. Because, after ten-plus years of commercial web development, they still have a tough time finding what they’re looking for, and they still wonder why it’s so damned unpleasant to read text on the web ”” which is what most of them do when they’re online.

I fear that now, more than 10 years after he wrote those words, we’re still in the same boat. Let’s change that. The next time you need to give feedback on a design, don’t go with your gut. Go with science, and keep your user/business goals in mind. Ask your designers goal-oriented questions, and if they can defend the decisions they made, trust them. If they can’t, you just gave constructive feedback. As you should.

As an enthusiastic (but mediocre) runner, I’ve been thinking a lot about the similarities between marathons and product management.  With all this focus on sprints, we sometimes forget that as one sprint runs into the next one, and the next one, and the next one, at some point it doesn’t feel like a sprint at all any more.  And before you know it you’re tired, overworked, and excessively grumpy — basically teetering on the edge of sanity.  There has to be a better way, right?

So when it comes to long runs, I think you can plot the relationship between distance covered and your level of joy as follows:

In general, you start off feeling great.  But as the first mile progresses, you start getting weird thoughts.  “What have I done?” “This is the stupidest decision I’ve ever made!” “I’m going to die!!!” And by the time you finish the first mile, you’re ready to collapse in a crying heap of shame.  But then something strange happens: you start enjoying yourself.  You hit your stride, you look around, you enjoy the scenery and the people… And before you know it you think that you can carry on like this for forever.

But of course, you can’t. At some point, things start going South again, and you feel like you’re never going to be able to finish. A downward spiral of self-doubt ends with a dose of endless self-pity in the dark depths of hell. And you’re ready to give up again. But then you see it: “1 mile to go.” And suddenly everything feels ok again. You pick up speed and race across the finish line like the champion that you know you are. And you say, “That was great! When can we do it again!”

This is a post about what I call First Mile Problems and Last Mile Problems in day-to-day Product Management.  If you replace “Distance” with “Project timeline”, and “Joy” with “Sanity level” in the graph above, I think you’ll have a good overview of what it means to be a PM. And the problems you experience at the beginning of a project are often distinctly different from the problems you experience later on. So I’m hoping that we can help each other here, and hopefully end up with a graph that’s a little more… stable.

First Mile Problems

1. Aligning the team

One of the toughest challenges in Product Management is getting everyone on the project team working in the same direction towards the same vision. Sure, you all like each other and work well together, but how do you take all those differing opinions, stop the brother/sister-like bickering, and get to work? This isn’t something I’ve completely figured out, but I do think some of the answer lies in what Business Week calls I-shaped people.

We’ve all hear about T-shaped people — those who have a deep understanding of one specific topic, and a basic knowledge of a broad range of different topics. But I-shaped people are different:

These have their feet firmly planted in the mud of the practical world, and yet stretch far enough to stick their head in the clouds when they need to. Furthermore, they simultaneously span all of the space in between.

[They are able to] rise above the specifics of a particular problem to think about them in a more abstract, and in some ways, more general way.

I-shaped people are able to build consensus around the team’s direction because they’re able to not only see  where the product is going, but they’re also stuck in the weeds enough to have credibility to make the day-to-day decisions. I look at it this way:

When you set the vision for the product, keep your head in the clouds:

• Communicate the vision clearly to the team, don’t expect they’re just going to implement whatever you tell them to do. Bring everyone along for the ride — it will make the vision better, and keep everyone invested in what you’re doing.
• Show how you plan to get there, so that the team knows it’s more than a pipe dream. Use broad strokes to communicate ordered priorities that will get the team to the vision. Which brings me to my next point…
• Use pictures. It doesn’t matter what kind. Sketches, wireframes, hi-fi mocks, whatever it takes to get the message across and allow everyone to have the same vision in their heads. Not showing at least some framework is a recipe for running into trouble in the last mile.
• Above all, remain flexible. The vision is a plan, and nothing more. Things will change. New ideas will come up. The interface will change. That’s more than ok — it’s a good thing. But you have to start walking in the right direction, otherwise you’ll never get anywhere.

When you get going on the project, keep your feet on the ground:

• Be in the details. Understand every pixel on the page. Know about every single Jira ticket. Show that you care about the journey, not just the destination. Be there for your team to remove every and any roadblock that might come up and prevent them from making progress.
• Communicate clear expectations, then trust your team. If you did your job right, you hired a team of amazing people. So don’t second-guess them. Ask questions, but allow the designer to defend his/her decisions, and trust them as long as the thought process makes sense.
• Most importantly, don’t hover. This is a hard one for me, but don’t stand over the designer’s shoulder. I have a rule — if a designer has Photoshop/Fireworks open, I stay away. They’ll come to me when they’re done. As the excellent article World Wide Mush points out: “If you want to foster creativity and excellence, you have to introduce some boundaries. Teams need some privacy from one another to develop unique approaches to any kind of competition. Making everything open all the time creates what I call a global mush.” Designers need time to design on their own before they share their thoughts with the rest of the team.
• And finally, learn what you don’t know. This one might be a little controversial, but I tend to agree with Six Revisions in their post Should Web Designers Know HTML and CSS: “Web designers should know HTML/CSS ”” even if it’s just limited to the fundamentals ”” for the sake of being able to create web designs and web interfaces that work on the medium.” If you don’t know how to code, learn. You don’t have to be great at it, you just need to understand it. If you came from a technical background and don’t know much about UX design principles, LEARN it. It will garner respect, make you more effective, and above all, result in better products.

I want to end this section with a final word on trust, which is a topic that’s very close to my heart. The site “99%” recently posted a fantastic article called What Motivates Us To Do Great Work. It’s a must-read for all managers of creative teams. The crux is this:

For creative thinkers, [there are] three key motivators: autonomy (self-directed work), mastery (getting better at stuff), and purpose (serving a greater vision). All three are intrinsic motivators.

As creative thinkers, we want to make progress, and we want to move big ideas forward. So, it’s no surprise that the best motivator is being empowered to take action.

When it comes to recommendations for creative leaders, [authors of a recent study] don’t mince words: “Scrupulously avoid impeding progress by changing goals autocratically, being indecisive, or holding up resources.” In short, give your team members what they need to thrive, and then get out of the way.

Like I said: trust, don’t hover…

2. Stakeholder buy-in

Once you have team alignment on the project and the vision, it’s time to turn your attention outward and focus on stakeholder buy-in. This part of the process often makes me feel like I’m on the wrong end of a very angry mob. It’s a stage when Product Managers need to use all the communication skills they can muster. Because when it comes to design, everyone has an opinion.

So this is where we need to talk about the phenomenon of Design by Committee.

If you haven’t seen the Oatmeal comic How A Web Design Goes Straight To Hell, you need to go read that first, and then come back here — it’s a perfect summary of the problem. The ultimate post on this problem has also already been written, so I’m not going to spend too much time on it — just go read every word in Why Design by Committee Must Die in Smashing Magazine. I do want to highlight a couple of areas in that article, and add some of my own comments.

One of the main problems we have in web design today is that everyone thinks they’re a designer. With coding it’s different — not everyone can code. But design is different. Like art, everyone has an opinion on design. You like it or you don’t. And because you have this immediate visceral reaction to a design, it’s tempting to confuse that with knowing what makes a design good. But that’s simply not true.

As posts like Designing for the Mind and  Gestalt Principles Applied to Design have shown, what makes a design “good” has very little to do with taste, and everything to do with the proven psychology of visual perception. “Pretty” is a small part of design compared to applying the principles of solid user experience design to an interface. So please, let’s leave design to the people who are trained in this stuff. Have I mentioned the importance of trust?

The Smashing Magazine article ends with this advice, which I agree with:

The sensible answer is to listen, absorb, discuss, be able to defend any design decision with clarity and reason, know when to pick your battles and know when to let go.

On a practical level, this is how I apply that piece of advice:

• Respond to every piece of feedback. This is tiring, but essential. Regardless of how helpful it is, if someone took the time to give you feedback on a design, you need to respond to it.
• Note what feedback is being incorporated. Be open to good feedback, don’t let pride get in the way of a design improvement, and let the person know what feedback is being incorporated.
• Explain why feedback is not being taken. If a particular piece of feedback is not being taken, don’t just ignore it. Let the person know that you thought about it, and explain the reason(s) for not incorporating that feedback. It’s hard to get upset at you if you explain clearly why you’re taking the direction you’re going with. And if you’re not sure how to defend the decision…
• Use the user experience validation stack to defend decisions. Read the post Winning a User Experience Debate for more detail, but in summary, try to first defend a decision based on user evidence — actual user testing on the product. If that’s not available, go to Google and find user research that backs up the decision. In the absence of that, go back to design theory to explain your direction. I mentioned some sample posts above that should be helpful, and also check out The Psychologist’s View of UX Design.

When it comes down to it, everything once again comes back to trust, and a clear definition of roles and responsibilities. Senior management needs to trust their hiring judgment, trust their Product Managers, and give them the authority to make decisions for the projects they are responsible for. And, of course, hold them accountable if those decisions end up being wrong. It’s all we want: give us the tools we need to get the best possible product out the door, and hold us accountable if we’re not successful.

Last Mile Problems

Ok, so let’s assume you have a team that’s moving along, working away on a project, and all the stakeholders are bought in, so you’re on a roll. But it is almost inevitable that things will get rocky again towards the end of the project. Here are some Last Mile Problems and thoughts on how to deal with them.

1. People issues

I probably don’t even have to describe this one. We all know what happens. People get tired and cranky. They start distrusting each other. A small mistake creeps in and the whole team jumps in to point it out. Whatever the catalyst, the outcome is all too familiar — a team that fights more than they work. So what do you do?

I’m not a fan of self-help books at all. Let me start by saying that. But a previous manager recommended Crucial Conversations — Tools for Talking when Stakes are High to me, and it’s been one of the most helpful books I’ve ever read. You have to read it cover to cover, but in summary, it describes how, when people are put in situations where they feel threatened for one reason or another, they react in one of two ways:

• Silence. When things get rough, some people simply withdraw, either by going quiet, glossing over the issues, or masking what’s really going on by changing the subject and talking about other things.
• Violence. Others resort to attacks (both directly or indirectly through passive aggressive behavior), putting labels on people (“he’s an idiot who writes bad code…”), or trying to control the situation without considering the opinion of others.

Both of these responses are destructive and not conducive to good teamwork.  So the rest of the book focuses on how to deal with those behaviors.  How do you create a safe environment where everyone feels they can voice their opinion without feeling disrespected or unvalued? I’ve found the following process really helpful in situations of team conflict:

• Commit to seek a mutual purpose. Recognize that there is a problem, and get the team to commit to fixing it and not just sweep things under the rug.
• Recognize the purpose behind the strategy. If different people on the team want different things, ask them to explain why they want this done a certain way. Even better if this can be phrased from a user’s perspective. For example, saying “I want this button to be red because I believe it will convert better” shows that the purpose behind the strategy is to increase conversion.
• Invent a mutual purpose. After everyone voiced the purpose behind their strategy, discuss a purpose everyone can agree on, before looking at specific tactics/strategies to get there. For example, if one team is concerned about conversion and the other team about user confusion, can you bring those together, and invent a purpose that encompasses both ideas? Can the purpose be to reduce drop-off while staying within the site’s visual style guide?
• Brainstorm new strategies. Once you know what the purpose is, the team can brainstorm solutions that will fulfill that purpose. Continuing our example, move the discussion away from button color, and find ways to move elements around to reduce user confusion and drop-off, while committing to testing different button colors and adjusting the style guide based on whatever the data says.

I used an imperfect example to explain those points, but the principles work — give it a try!

2. Delivery issues

Stress-levels rise as delivery dates approach, especially if it looks like those dates aren’t going to be met. The first thing we need to clear up is that humans are horrible at estimation (here’s a very interesting post trying to solve that problem). I think the solution to this kind of stress is, as controversial (yet recently very popular) as it is, to do away with detailed estimates.  Tom Fishburne sums it up really well in his post Waterfall Planning:

It’s important to know the business inside and out and have a clear trajectory where w’re headed.  But there is a point where planning becomes overplanning.  All that time spent planning is time spent not doing.

Gantt charts are particularly problematic in this regard, as summed up by another great post called Project Managers, Not Calendar Monkeys:

I’ve been working on a way to balance the benefits of Gantt charts (ability to input dependencies and adjust an entire schedule with the push of a button) with the best way to communicate project schedules to clients. Most people can’t read a Gantt chart, and no client should have to. Gantt charts are more useful for planning next steps than providing an at-a-glance project status anyway.

So what’s the solution? I don’t think we really know yet. I am all for planning and knowing where you’re going, as I pointed out earlier in this post. But I think holding a team to specific deadlines isn’t the right way to increase velocity, and results in more stress than it’s worth. So what’s the alternative? I think you build (and sell to executives) a team driven by different types of goals:

• Build a team driven by priorities, not timelines. Know what are the most important things to work on, work on those first, and move on to the next thing. Have goals (that can include timelines), but realize that things can change. Trust your team that they will work towards those goals, and then knock off the priorities one by one. Of course there will be deadlines driven by external pressures, but keep those at a high level so that the team has a clear goal to focus on. This will ensure better quality (because you’re not cutting corners and rushing to get things done) and higher velocity (because everyone is focused on doing the right things in the right order).
• Build a team driven by action, not meetings. In progress meetings, I prefer to represent my team so that they can go on with their work uninterrupted. If someone wants to attend the meeting and speak for themselves, that’s fantastic, but I don’t think it should be a requirement that everyone on the team be present for strategy and update meetings. Note that this is different from working design meetings, where actual solutions are being produced. A meeting where you walk out with a deliverable (a sketch, a wireframe, a product requirement) is a good meeting. All others should be handled by the Product Manager/Owner so that the team can go about their business uninterrupted.

Speaking of which, allow me to bring in one more voice of reason on meetings, Mike Monteiro. In his excellent post The Chokehold of Calendars he writes:

Meetings may be toxic, but calendars are the superfund sites that allow that toxicity to thrive. All calendars suck. And they all suck in the same way. Calendars are a record of interruptions. And quite often they’re a battlefield over who owns whose time.

Make sure the focus is on working, not telling other people what you’re working on.

So there that is

As I read through this post again, I realize there might be some resistance to some of these ideas. Not that anything I said here is particularly new, but because I know there are several counter-arguments that can be made to these points. That’s exactly how it should be.

I’d love to get some input on this, particularly ideas and tips on how you’ve been able to deal with some of these problems. This is all a work in progress, as I assume it will be until the end of time.

This post is based on a recent talk I gave at the South African Scrum User Group, entitled Roadmap to serenity - How to stay sane as a Product Owner.

This is the third post in a series I recently started on software development and the role of the Product Manager.  If you haven’t already done so, it might be a good idea to read Part 1 (Overview) and Part 2 (How to ensure that product requirements are informed by user needs) before your read on.  This post continues the discussion on Product Requirements and the different sources that should feed into requirements.

In Part 2 of this series I discussed the role of user needs in product requirements, and in this article I’d like to talk about the role of business needs and technology needs, and making sure that the right balance is struck when incorporating these (often loud, often conflicting) voices in the organization into what gets built.  So, let’s dive in…

Business needs

When I was at eBay, we often heard the mantra from our executive team, “If you fix the user experience, you fix the business.”  Lovely words, but when it comes time to decide what to build, “Fix the business” usually comes first.  This is, of course, not a bad thing, but unfortunately the best user experience often means taking revenue-generating features out of the product.  Would we have banner ads if UX really was king?  Don’t think so…

Still, you have to make money.  That is, after all, the point of the business.  The trick is to understand the difference between good revenue streams and bad revenue streams, and opt for the good ones as much as possible.  A good case study on this is eBay’s interesting approach to photos in product listings on the site.  eBay started charging users to add photos to their  listings pretty much from the very beginning.  This was back in 1995, and in those days storage wasn’t dirt cheap, so it was a natural thing to do.

As the years went by, and more and more photo sharing services popped up that allows users to upload and stores pictures for free, this approach became increasingly frustrating for users.  The other side of the story is that it’s actually in eBay’s best interest for users to upload photos of their items — items with photos convert way better than those without photos.

Still, it took many months to convince the executive team to make it free for users to upload photos of their items.  This is an example of a bad revenue stream — it brings in money, but to the detriment of users and the overall success of the business.  When it comes to adding revenue streams to your product, the important question should always be: are you doing this so people will buy it, or are you doing this so people will want to use it and be willing to pay for it****?

In a recent interview on Microsoft and tablets, Steve Ballmer said the following:

And so we are working with [our] partners, not just to deliver something, but to deliver products that people really want to go buy.

And in that lies the core of what’s wrong with Microsoft — the difference between making products users want to buy vs. making products they want to use.  When you make products people want to use, charging for the value it brings (i.e., looking for good revenue streams), becomes so much easier.  Approaching it from the more negative side, I guess you could also say it like this:

Technology needs

One of the dangers of product roadmaps and the PM’s role is that back-end maintenance and optimization can start to take a back seat.  This is a huge mistake, best explained through the metaphor of technical debt. In Steve McConnel’s great post on this topic, he defines technical debt as follows:

The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job.

The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future. “If we don’t get this release done on time, there won’t be a next release” is a common refrain””and often a compelling one.

He goes on to explain why this can become a problem:

If the debt grows large enough, eventually the company will spend more on servicing its debt than it invests in increasing the value of its other assets. A common example is a legacy code base in which so much work goes into keeping a production system running (i.e., “servicing the debt”) that there is little time left over to add new capabilities to the system. With financial debt, analysts talk about the “debt ratio,” which is equal to total debt divided by total assets. Higher debt ratios are seen as more risky, which seems true for technical debt, too.

Technical debt isn’t always wrong — quick hacks to get a product out the door is often the right choice.  But as with most debt, it’s important to start paying it off in small chunks as soon as it’s incurred, before you get into too much trouble.  If you’re interested in this topic, also read Andrew Chen’s great post called Product Design debt vs. Technical Debt.

Striking the right balance

Now that we’ve discussed user needs, business needs, and technology needs, the obvious question is: how do you decide what to build now vs. later vs. not at all?

For that, the right answer is unfortunately, in my experience, the traditional cop-out answer: it depends. It depends mainly on the following factors (in no particular order):

• The level of user engagement and involvement.  If users are screaming for a particular feature, or if there are rumblings around “why haven’t you done anything for us recently?”, it could be a good time to up the level of customer needs you meet.
• The stage of the product in its lifecycle. If the product is just at the beginning, customer needs will most likely come first.  As the product matures, technology and business needs become more important and should start taking precedence.
• The financial state of the business.  If there are ways to add good revenue streams, those opportunities should always be taken.

Depending on where the business is on each of these 3 factors, the different inputs might be weighted differently. If the product is going through a growth spurt with lots of buzz, more attention could be placed on user needs. If the product is mature and making good money, technical needs might get more weight.

Exactly how this is balanced in each version/release of the product has no clear answer, and it’s where the art of product management comes in. But one thing is for sure — none of these needs can be ignored for any extended period of time. Take too long to pay down technical debt, and your platform will become bloated and unable to scale. Focus on making money too much, and users will fall out of love with your product.

Successful products have clear product management leaders who are able to take all the different requirements inputs, place it into context with other external and internal pressures, limits, and opportunities, and design a product vision and a (flexible) product roadmap that ultimately increases product/market fit (which I mentioned briefly in Part 2 of this series).

But what do product requirements look like, and what is the Product Manager’s role in that process?  That will be the topic of the next post…

I recently started a series on software development and the role of the Product Manager.  If you haven’t already done so, it might be a good idea to read Part 1 (Overview) before continuing.  In this post I’d like to write about the first step in the development process, namely Product Requirements, and the various sources of input that go into deciding what to build and how to improve your product.

As I started writing I realized the topic is just too big for one post, so I’m going to split it up into a few different posts:

• Part 2 (this post) will be about user needs as an input to product requirements.
• Part 3 will be about business needs and technology needs as inputs to product requirements.
• Part 4 will be about  the PM’s role in the Product Requirements phase.

Even though the focus here is not on what kind of product/service your company should develop and sell, I do want to briefly mention product/market fit, because it is probably the most important aspect to figure out to be a successful business.  No one talks about this better than Marc Andreesen, so I wanted to quote from one of his (now deleted) blog posts:

The quality of a startup’s product can be defined as how impressive the product is to one customer or user who actually uses it: How easy is the product to use? How feature rich is it? How fast is it? How extensible is it? How polished is it? How many (or rather, how few) bugs does it have?

The size of a startup’s market is the the number, and growth rate, of those customers or users for that product.

The only thing that matters is getting to product/market fit.  Product/market fit means being in a good market with a product that can satisfy that market.

With product/market fit figured out (no easy task), and a workable product to start with, it’s time to get serious about building and improving the product — and that’s the stage where this post starts.  At the heart of a good product roadmap stands a Product Manager that is able to strike a balance between user needs, business needs, and technology needs.  So let’s look at each of those in detail, starting with user needs.

User needs

Identifying user needs is at the core of a user-centered design process, and it involves gathering feedback from the users of your product through a variety of methods to uncover unmet needs and opportunities for improvements.  This is called user experience research (UER), and there are plenty of resources available on the topic, so I’m just going to provide an overview here.

First, it’s important to mention the fundamental difference between market research and user experience research:

• Market research seeks to understand the needs of the market in general, and is generally more focused on areas like market fit, brand perception, advertising and pricing research, and ways to uncover perceptions of the company and its products.
• In contrast, user experience research focuses on users’ interaction with the product.  It relies heavily on observational techniques, since users are notoriously bad at describing their experiences or predicting their behavior.

There are many ways to classify different UER methodologies, but my preference is for a classification that lines up the different methods with the outcomes required by different phases of the product development process.  In that approach, there are three classes:

1. Strategic research

Strategic research is done with the goal of coming up with new product ideas and business opportunities, or preliminary design explorations during the product discovery phase to  help with the brainstorming of design solutions.  Here are some of the methods that fall into this class:

• Ethnography.  A technique long used in Anthropology that has only recently found its way into the toolkit for research on interactive products.  It involves going to users’ homes or offices, and observing how they use your products in their natural environment.  It allows the researcher to see users in action where they feel most comfortable.  Ethnography is all about observing and listening.  It is generally not task-based like usability studies, but follows a loose interview script with the goal of uncovering needs and insights that users are unable to articulate.  I have an extreme positive bias towards ethnographic research, especially in contrast to focus groups (of which I’m not a fan at all, but that’s a subject for a different post).  I have seen first-hand how ethnography sparks innovation when it shows how users make up their own workarounds for the limitations of software, which in turn reveals opportunities for product improvements.  When it comes to exploratory methods to help with product strategy and roadmaps, there simple is nothing better than a good ethnography study.
• Participatory design.  Another favorite, this technique brings users together to solve design problems in a way that would make sense to them.  The purpose is not to take users’ designs and implement them, but to find out which elements and activities are most important to them.  My preference is to do this in diads, where 2 users work together on a design.  This forces both participants to be active, and you learn as much from their conversation with each other as with the designs they put together.  The usual process is to provide users with a blank page or basic framework, cut-outs of various elements that could go on a page, and watch and listen as they make trade-offs and design decisions on what should go on the page based on how they would use the product.  This technique especially helps guide interaction design because it gives a glimpse into users’ process as they go through the site.
• Concept testing.  This is a good way to gather feedback on an approach before wireframes or mockups are created.  Storyboards/comics are great artifacts to use for this kind of testing, since it takes design out of the process, and gathers feedback from users on the process you intend to design.  Although mostly done in-person with 6-8 users, there are also great tools for large-scale concept testing, like Invoke.  Below is an example of a storyboard one of our researchers at eBay used during early concept testing for one of our products:

Further reading on strategic research:

• Ethnography in Industry: Objectives?
• How to set up and run participatory design sessions, analyze data and present results effectively

2. Design research

Design research includes most of the methods that are associated with traditional user experience research.  Its role is to improve and refine existing designs in various levels of fidelity.  Some of the methods include:

• Usability testing.  This is, of course, the most well-known UER method, and what most people default to for any kind of design feedback.  Task-based usability testing in a lab is a fantastic tool, but it has become a little bit too much of a “when all you have is a hammer…” method.  Usability testing should be used to uncover usability problems with a proposed interaction design.  It should not be used for feedback on visuals, finding out which design users prefer, or uncovering new product ideas.  There are other techniques that are much better suited to that task.  Usability testing involves 1-on-1 sessions with users where the researcher observes them as they perform assigned tasks.  This kind of direct observation is a great way to understand what users would actually do on the site (as opposed to what they say they would do), as well as to uncover the reasons why they do what they do.
• RITE testing.  Rapid Iterative Testing & Evaluation (RITE) is a very specific form of usability testing, but I wanted to call it out because it is my preferred way of testing.  It involves a day or two of focused usability testing, followed by a design cycle where the feedback is immediately injected into the design before the next round of testing.  Doing several of these cycles quickly means your outcome isn’t a bloated Powerpoint deck with a bunch of recommendations; your outcome is a better design that incorporated user feedback in real time.  As the debate continues on how UX can be more involved in Agile development, this technique should become increasingly important since it fits in perfectly with the Agile mindset.
• Desirability studies.  Invented by researchers at Microsoft (yes, really - see this Word article where they outline the approach), this has become another favorite technique for me if I want feedback on the visual aspects of a site (not so much interaction), and specifically which visual approach users like more when there is more than one alternative.  It involves a survey to a large number of users where they are asked to rate one of the proposed design alternatives using a semantic differential scale.  The survey is done as a between-subjects experiment, meaning each user sees only one of the proposed designs, so that they are not influenced but the other design alternatives.  The analysis then clearly shows differences in the visual desirability of the proposed design alternatives.

Further reading on design research:

• Complete beginner’s guide to design research
• Usability testing overview
• Do’s and Don’ts of usability testing

3. Assessment research

Often neglected, the role of assessment research is to measure the impact design decisions have made, and to evaluate success and continued areas that need improvement.  This requires larger sample sizes to ensure the ability to compare before/after metrics with statistical significance, so these methods are mainly quantitative in nature.  Methods include:

• Product surveys.  Everyone hates surveys, but it remains an effective way to assess how design changes are affecting user perceptions of the site. Different from most market research surveys you receive (and delete) in your inbox, these surveys deal specifically with user perceptions of the interaction and design.  It’s not always effective as standalone research studies since there is so much bias in surveys with their <5% response rates, but if you can run surveys over time, and control the sample so that the bias remains the same, it can be a very good tool to ascertain the success of your design changes.
• Online user experience assessments.  Another favorite, tools like Keynote allows you to gather real-time click-through data as well as subjective user feedback.  It uses a proxy or a browser download to give users tasks on a site, and ask them questions about the experience while their activity is being tracked.  This often produces a mountain of data, which can be quite overwhelming and not always effectively used if there is not enough time/resources available to analyze the data.
• Analytics.  Web analytics need no introduction, but there are several tools specifically aimed at user experience, with my favorite being Razorfish’s Advanced Optimization tool for web forms.  By placing a small piece of JS on form pages, it gives you a mountain of data on how users interact with forms, including what error messages they receive, how much time they spend in each field, the last field they were on before closing the form, scrolling data, and the list goes on.  It’s a great way to improve form conversion.  I honestly don’t know why they don’t market this thing more.

So that’s an overview of user research methods — there are many more, but I wanted to focus on the ones I’ve found especially useful in my own work.  The real power of user research starts to happen when you combine methods and triangulate results to come up with a product strategy that takes a variety of quantitative and qualitative insights into consideration.  If you’re interested to learn more about this, Using Multiple Data Sources and Insights to Aid Design is a good post on the topic.

There are so many resources available on user research — your best bet to stay on top of the latest happenings in the field is to subscribe to UX Booth and UX Matters.

In part 2b, I’ll talk about the other two inputs to Product Requirements: Business Needs and Technology Needs.  I’ll also discuss how all of this fits into creating Product Requirements, and what the Product Manager’s role is in all of it.  Stay tuned!