Bill Selman wrote a great post on the importance of research triangulation. If you have trouble making the case for qualitative research in a company smitten with “big data”, show them Why Do We Conduct Qualitative User Research? (emphasis mine):

There is no one research method that satisfies answering all of our questions. If the questions we are asking about user behavior, attitudes, and beliefs are based solely on assumptions formed in our homophilic bubble, we will not generate accurate insights about our users no matter how large the dataset. In other words, we only know what we know and can only ask questions framed about what we know. If we are measuring, we can only measure what we know to ask. Quantitative user research needs qualitative user research methods in order to know what we should be measuring and to provide examples, theories, and explanations. Likewise, qualitative research needs quantitative research to measure and validate our work as well as to uncover larger patterns we cannot see.

There are some great answers in the Quora thread How do you prevent scope creep in agile projects when usability testing? From Todd Zaki Warfel’s answer:

Usability testing doesn’t have to be a formal 3 month engagement. There are plenty of resources out there for doing rapid testing, which can easily be worked into a typical 3-6 week sprint cycle. We do it all the time. Take a week out of the sprint to plan, test, compile recommendations and get to work implementing them. Or as Jeff Gothelf commented, just pick a single day each week to test. You won’t need may participants. If you test weekly, then 3-5 will show a pattern. And anything you don’t catch this week, you’ll catch next week.

From Jeff Gothelf’s answer:

Usability testing needs to be worked in to the sprint timeline. In fact, it should take place during each and every sprint. We test every Thursday. It’s never more than 3 or 4 people. We show whatever we have ready on that day. Sometimes it’s live code and sometimes it’s a paper sketch. Everything in between is fair game. The purpose of this is to get constant feedback, mid-iteration, on the work being designed. In many cases, developers and product owners view portions of these sessions and see the issues that arise. An expectation is built within the team that, based on how things test on Thursday, there may be some tweaks to the scope.

Peter Morville’s Creating a Cultural Fit: Using ethnography with users and stakeholders is one of the most useful UX articles I’ve read in a while. He talks about the importance of understanding the cultures of both users and companies to create good products. He concludes:

In short, the right design is one that fits the company and its customers. A mismatch on either side results in fatal error. We must use ethnography with our users and stakeholders to search for a bi-cultural fit. This is tricky since culture is mostly invisible. That’s why we should start with a map.

This is a must-read.

I really like Alex Maughan’s post In a word, Human. He discusses the dangerous gap between those who make software, and those who use it:

Successful user experiences happen when producer whims get sidelined, and primary focus is placed on that little bit of common sense shared by the majority of humans using your product or service. It’s when you empathetically put yourself in the most common psychological flow of a human using a piece of technology they know almost nothing about.

This is a topic I’m very passionate about, so I agree wholeheartedly with Alex.

I need to tell you about a conversation I had with my 4-year old daughter this morning. You’re going to have to stick with me, because despite the potty talk this is going somewhere, I promise. The backstory is that lately she’s been forgetting to wipe after going to the bathroom (this is the type of stuff you’re here to read, right?). So when she got up this morning, this happened:

Me: “Did you go to the bathroom?”

Her: “Yes.”

Me: “Did you wipe?”

Her: “Oh… no. I don’t think so…”

After a few seconds of silence, she said something that’s pretty obvious, but hasn’t occurred to me before as the cause of the problem:

Her: “The toilet paper is behind me so I forget. We should put it in front of me so I can see it.”

Well, of course. Mystery solved. See, a few weeks ago I accidentally broke our toilet roll holder (yes, I’m pretty sure this is the kind of thing you come here to read…), and since then, we’ve been putting the toilet roll on the back of the toilet. And that was the cause of my daughter’s sudden “forgetfulness” in the wiping department.

This immediately made me think of product design, and in particular, the all-important principle of recognition vs. recall:

Showing users things they can recognize improves usability over needing to recall items from scratch because the extra context helps users retrieve information from memory.

It also reminded me of a great story Marco Arment once told in Right versus pragmatic. You should read the whole thing, but the gist is that at a previous job people kept dropping trash at the door before they left the bathroom (what is it with me and bathrooms today?), despite increasingly passive aggressive signs being put up by “management” to please throw trash in the bins. Here’s Marco’s illustration of the problem:

Source: Marco Arment

The solution that worked in the end? Put the trash can by the door. Seems obvious, but it still took them quite a while to figure it out. Instead of trying to change behavior — or to get people to remember to do something they’re not used to — they put a trash can where people are already throwing stuff, and that solved the problem.

Ok, enough with the bathroom examples. I recently had a discussion with a health plan who has a tool for their members to find doctors who are in their network. They mentioned that they get a lot of customer service calls from users who complain that they can’t search by a doctor’s name. This frustrates the health plan because the interface clearly lets you do that. But when you look at the interface you can see why they are having this problem. Here’s the home page:

Can you see where to go to search for a doctor by name? Not sure if you guessed it, but you have to click on the “Advanced Search” link under the Doctors icon:

Here’s the point of all this. Just because a feature is in the interface, it doesn’t mean users will find it. Just because an option is in a menu, it doesn’t mean users will know how to access it. And just because the toilet paper is somewhere in the bathroom, it doesn’t mean my daughter is going to find it.

The lesson is pretty simple, but there are enough everyday examples out there that it’s something we should continue to remind ourselves of. The next time you design an interface, say this to yourself over and over: If I want people to use toilet paper, I should put it where they can see it.

Lauren Chapman Ruiz shares an interesting viewpoint about the role of empathy in design in Inside the Empathy Trap. The issue is that we can’t just have empathy with one person, we need to have an understanding of a wide variety of goals and needs:

When we design, we pursue a broader type of empathy. As a colleague once said to me, designers need to identify with the whole user base. User-centricity is about the ability to recognize that there are a number of personas, each with different goals, desires, challenges, behaviors, and needs. We design for these personas, recognizing that each has different goals they’re trying to accomplish and with different behaviors in how they go about achieving them.

This is another good argument for why personas aren’t dead — they help us to keep our entire target audience in mind without getting overwhelmed.

I like the work Growing Agile does, so I clicked on Karen Greeves’s Agile & UX: What’s wrong with working one sprint ahead? with great interest. I’ve been struggling with this issue myself, but I’ve just never come across a better way to integrate UX and Agile, so I was very interested in their viewpoint. The post definitely got me thinking, but I have some issues with their objections. In the interest of letting you in on the thought process, I’m going to take some lines from the post and explain why I don’t agree1. Let’s start with their main concern with UX working one sprint ahead:

The main issue with this is one of focus. If the UX designers work one sprint ahead of the rest of the team, then the UX designers and the team actually never work on the same stuff at the same time. This means they are never focused on the same thing.

This is (partially) true, but isn’t it also true for individual developers on the team? They’ll be working on different stories. That’s the reason we have standup meetings: to make sure that those individual tasks work together to finish the whole.

I say it’s only partially true, because people can focus on more than one thing… So while the UX team is working on the next sprint, they can still be available to help with changes and questions on the current implementation work.

It doesn’t make sense to have the same standup meeting because you aren’t working on the same stuff.

Again, very few people are working on the same stuff. And this is actually an opportunity for the UX team to highlight any issues they have that developers can help them with.

It’s harder to help each other out because you are focused on finishing different stuff.

I don’t really get this point — we should be able to shift focus, and helping each other in different contexts often produce better results. Isn’t that the principle that code reviews are based on as well?

The people working ahead consider their deliverable some interim artefact, not working software, so there is a handover of both knowledge and responsibility.

In our context the UX team’s deliverable is working software in the pattern library, which uses the same frameworks and environment as our front-end developers, so that they can keep going seamlessly. So it’s possible to solve this problem.

People don’t see themselves as part of the same team, so you end up having a UX team and a dev team. This inhibits collaboration and communication.

I don’t agree with this point, either. We’re always going to have different skills on teams. Product Owners have different skills than the development team too. In fact, I think it can increase collaboration if developers are asked to provide input while the UX team is working on an upcoming feature.

The people working behind aren’t involved in the decision making that happened without them and as a result they don’t understand the reasons for certain design choices, this often leads to assumptions, and rework.

This doesn’t have to be the case. We have developers involved from Product Discovery all the way through prototyping, testing, and final specifications.

I’m not advocating writing code before an UX work has taken place. I’m saying the whole team should be involved in that work. Obviously the UX designer(s) take the lead here, but everyone on the team needs to see users use their product and understand the user journey map.

Ok, on this we agree!

In each sprint the whole team should set aside time to look at UX designs for the next sprint, as well as usability testing whatever has been completed.

I agree with this, and it’s how we work. But where I’m confused is that this still means that the UX team is working one sprint ahead of development — they’re working on “UX designs for the next sprint.” The only difference from how it’s usually done when UX and Agile works together, is that it formalizes the time that developers spend on being involved in the UX process.

If I understood that correctly, then it turns out we agree after all, and maybe this post wasn’t necessary…

---

1. This is an important discussion, and I think they add valuable points to it, so please don’t see this as some kind of takedown attempt. I know (and like) Karen and Sam, but I also believe in public, respectful disagreement. So that’s my disclaimer! ↩

I love this quote from Suzanne Slatcher, who worked at Pixar in the early years. From the really interesting article Building The Next Pixar:

“Business is just an idea, like a movie,” says Slatcher. “What if we did this in this place at this time, in this style of packaging, with this choice of flavors? Would it work? There’s still a back and forth between creative and the audience, and you can’t be like ‘if I build it, they will come.’ No, we’re in a democratic world where everyone has opinions. If you’re making your cartoon and your joke’s not funny, it’s just not funny, it has to go. If people don’t like a flavor, they’re right, we’re not right.”

That last sentence is a key concept in product development as well. If people don’t like our “flavor” of product, they’re right and we’re wrong. We have to fix it, not try to convince them that they don’t “get it”.

I’m a big fan of sketching, and this post by Jacob Rader does a great job of explaining why it’s such an important part of the design process:

Sensemaking is the active conversation we have with the events that we encounter; it’s our ability to take in information, process it, and derive meaning and action from it.

Rather than focusing on the external, sketching provides us a sensemaking process for our own creative flux. When we’re presented with a problem our minds go to work to create this cloud of ideas, populating it with information and attempting to form connections. This facilitated sensemaking turns that abstract concoction into a concrete reality. This works because sketching forces us to make decisions and apply structure to our ideas. By externalizing we pass those fragments through a filter of our own experience creating a foundation to build our ideas from.

When we externalize the pieces of an idea through a sketch we’re making a testable “design move” which we’re able make judgements around. This positions us to make further moves that iteratively cycles and builds an idea.

Gary Graham writes about some of the dangers of the smart city movement in Too-smart cities? Why these visions of utopia need an urgent reality check:

Ideally a future city will have inner-city areas that are sustainably created through private, for-profit initiatives, and investment based on genuine competitive advantage — not through artificial inducements, charity or government mandates.

The people living in cities far outnumber the people making decisions about what those cities should look like in the future. They are disconnected from the plans being made by companies and even governments on their behalf.

We need to start working with everyday citizens to find the right questions — and then work with them towards developing solutions to the problems they raise.

The article also mentions the city of Brasília as an example of a failed experiment because it didn’t take the needs of its citizens into consideration. That’s a topic that’s near and dear to me:

Together, we can avoid building digital Brasílias — projects that generate buzz, but don’t meet the needs of the people who live there.