A couple of articles caught my eye today because they tie in well with my Intuition vs. Science in design post from yesterday. In Design and uncertainty Ellen Beldner writes about an essential characteristic for every designer: acknowledging that your assumptions will be wrong more often than not. She also makes a great case for usability testing:
The problems come when you don’t admit, as a designer or product person, that intuitions based on your mom or yourself may or may not extend to what most other people actually do. So a designer who seems like a hotshot Howard Roark out of college may be great for that one particular project. But when you ask him or her to work on a design for a domain that they don’t “intuitively” understand (since they don’t have years of experience being within that particular community) they’ll flail if they don’t know how to turn to research and data to inform their opinions.
You should always design the product you think/believe/know is what people want — there’s a genius in that activity that no instrumentation, no data report, no analysis will ever replace. But at the same time you should be relentless in looking at the data on how people actually use what you’ve built, and you should be looking for things that show which assumptions you’ve made are wrong, because those are the clues to what can be made better.
This all comes back to that necessary balance between science (hard data) and intuition. Usability testing and contextual research help us understand unfamiliar domains enough to kick off the design process. Intuition lets us meet those users’ needs in creative ways. And analytics, combined with qualitative user research methods, help us figure out where we got it wrong and how we can do better.
If a rat is rewarded for choosing a rectangle over a square, it will learn to respond to “rectangularity” and start to favor rectangles in general. So maybe we are like the rats, and what w’re seeking while idly yet compulsively cruising Pinterest is really just the reliably unpredictable jumble of emotions that their wistful, quirky juxtapositions evoke. Maybe that is our rectangularity.
Ther’s a German word for it, of course: Sehnsucht, which translates as “addictive yearning.” This is, I think, what these sites evoke: the feeling of being addicted to longing for something; specifically being addicted to the feeling that something is missing or incomplete. The point is not the thing that is being longed for, but the feeling of longing for the thing. And that feeling is necessarily ambivalent, combining both positive and negative emotions.
The gap between the technical skills required to use the software we make, and what the majority of people are actually capable of, is widening at an alarming rate. Not only that, but we often appear to not even like the people we design and develop for. We champion empathy as a core tenet of user experience design, yet we are mighty quick to point out how bad our moms are at “computers”, and how we hate going home for the holidays because we end up spending all our time on family tech support.
I worry that technology is advancing so quickly that it’s no longer able to ground our thinking in a bit of reality. Some might see this as a good thing, but I don’t. Unless we make a conscious effort to get back into the minds of our users — and not chuckle at what we find when we peek in there — we’re going to run away with the web and leave most of the world standing around in bewilderment, wondering what just happened.
Don’t believe me that this is a thing? Ok, here are some examples. First, CNET’s Greg Sandoval describes the downfall of Netflix in his must-read article Netflix’s lost year: The inside story of the price-hike train wreck. It’s long, so it’s easy to skip over this sentence that perfectly sums up why things went so wrong for Netflix:
But even visionaries can misread their customers when they are blinded by their past success.
CEO Reed Hastings thought he had his customers figured out, but he didn’t. At all.
Among the missteps: Digg botched its re-launch in the summer of 2010, and, more importantly, he said the company was slow to respond to the criticism. ‘We were desperately trying to figure out how to get traffic back,’ he said. ‘A bunch of the community had already revolted by the time we fixed it.’
Once again, they thought they knew their customers, and once again, they didn’t.
For the last example we’ll go even further down the technical totem pole, lest we forget what goes on in the bottom half of the Internet. In 2010 ReadWriteReb wrote an article about Facebook Connect and AOL Instant Messenger called Facebook Wants To Be Your One True Login. But their SEO was so good that if you went to Google and typed in “Facebook login”, that article would be the first result. It wasn’t long before they started receiving comments like this:
That’s right — people thought that they were on Facebook, and that the “new design” had inexplicably taken away the ability to log in. Things got so bad that they had to put up this message in the middle of the article, which is still there today:
The editors of ReadWriteWeb made one more mistake, though. They assumed that people know what a browser is. Watch this:
You may be asking yourself, “How do these people survive on the Internet? How do they get anything done online?” Well, we’d better believe it — they’re here, walking among us in plain sight. One can argue that things have gotten even worse since that article and video came out. Matthew Berk recently did an analysis of 1.3 Billion URLs and found that 22% of Web pages contain Facebook URLs. Google used to equal Browser for most people. Now Facebook is becoming the browser — it is people’s viewport to the Web.
Ha ha, we scoff. Who wants to do Microsoft Office on a tablet? Office is boring. And tablets have a completely different use case to laptops. Who would want one to run full Windows?
Answer: Lots of people. People with different priorities, working different jobs, living in different countries. People we don’t quite understand.
The rabbit hole just doesn’t end, no matter how deep you go. We haven’t even talked about YouTube comments or Clients From Hell. But I’ll stop here, and just say this: the landscape we create software for is scary. It’s terribly comfortable over here on Twitter, but how can we design software and applications for people we don’t hang out with?
It seems like such a simple problem to solve, but I’m not seeing much evidence that talking to customers is a widespread thing among startups and even many established companies. We love talking about User Experience Design in the abstract — especially if it means we can argue about whether it exists or not.
But you know who are the real heroes of UX (you know, if it actually exists)? The ethnographers. The user researchers (who had to change their name from “usability engineers” because definitions blah blah blah). The real heroes are the people who spend their days understanding user needs, and fighting with all their might to get people to make things that solve real needs, not things that floor us with their beauty and radiance and lack of utility. Douwe Osinga’s description of Google Wave comes to mind:
Wave started with some fairly easy to understand ideas about online collaboration and communication. But in order to make it more general and universal, more ideas were added until the entire thing could only be explained in a 90 minute mind blowing demo that left people speechless but a little later wondering what the hell this was for.
Let’s stop that from happening to the things we make. We don’t have to leave this job to The Researchers. We can all talk to the people who use (or might use) our software. We can go to a coffee shop and get feedback on wireframes for the price of a few cappuccinos. We can sit and watch some of our family members use the web, and make notes. We can try to spend a fixed percentage of every week talking to users. If we don’t, we’ll continue to widen what Jakob Nielsen calls the Usability Divide:
Far worse than the [digital] economic divide is the fact that technology remains so complicated that many people couldn’t use a computer even if they got one for free. Many others can use computers, but don’t achieve the modern world’s full benefits because most of the available services are too difficult for them to understand.
Whereas the [digital] economic divide is closing rapidly, I see little progress on the usability divide. Usability is improving for higher-end users. For this group, websites get easier every year, generating vast profits for site owners. That’s all great news for high-end users, but the less-skilled 40% of users have seen little in the way of usability improvement. We know how to help these users — we’re simply not doing it.
We know how to help these users — we’re simply not doing it. We don’t have a choice, we have to talk to them. It’s easy to start: take your laptop with you on one of your coffee breaks, and ask some people if you can show them what you’re working on. They’ll love giving you feedback, and you’ll walk away with a better understanding of the usability divide — and some very real ideas about how to narrow it.
I’m linking to Sam Biddle’s Does Google Have Any Social Skills at All? with some caveats. I’m not a fan of headlines like that (here’s why), and there are way too many hyperbolic, trolling statements like “Nobody really uses Google+”. Still, the article makes some good points about Google’s lack of understanding of their users:
The keynote sounded one futuristic clarion call after another: Glass, the wearable computer; Google Now, a smartphone system that provides intricately tailored life information; the Nexus Q, a social media streamer; and a fancy new way to throw parties with Google+. But underneath each of these feats of technology you could see a hollow, lurching weirdness that makes you wonder: Who will use any of this stuff besides the actors in Google’s promo videos? [”¦]
In each case, Google has balanced on golden fingers a product — clearly with a lot of time, thought, and money behind it — that just doesn’t seem to jibe with the way we actually live our lives. There isn’t any lack of effort or innovation here, but rather a gaping disconnect between the way data geeks and the rest of us see the world.
I know Google does a lot of user research on their flagship products, but it doesn’t look like they do any Product Discovery or user need analysis on these new products. Maybe they’re genius designers, so they don’t need to do research. Or maybe not.
Towards the end of Android Fragmentation Visualized, an article by OpenSignalMaps that analyzes 3,997 (!) distinct Android devices across different dimensions, comes this attempt at setting the world record for Silver-Lining Hunting:
One of the joys of developing for Android is you have no idea who’ll end up using your app.
Pardon my lack of eloquence, but, LOL! First, there’s the obvious logical fallacy: no-one knows who will end up using their apps, no matter what phone/platform they develop for1. Second, not knowing who will end up using your app is a bad thing. Most apps fail because they are unable to reach product/market fit. And one of the major reasons for not reaching product/market fit is not understanding your market — the people who will end up using your app.
The variety of resolutions visualized in the OpenSignalMaps post is staggering, and trying to spin it as anything but a nightmare for developers is commendable but misguided.
User and market research can help you make educated guesses, but will only take you so far until you have a real product in the wild. ↩
South African bank Absa just rolled out their new online banking portal. There are two things about this launch that raise red flags for me. First, from ABSA rolls out new Internet Banking revision:
This launch follows a successful trial with the bank’s 36 000 employees over the past few months. The trial allowed the project team to identify and solve any defects and gauge the response from users, via over 1 300 feedback emails received from employees.
It’s shocking that we still have to talk about this, but let’s just state it again, as clearly as possible: You are not the user. You cannot test a system on employees — who know all the intricacies inside and out — and think that you’ve done appropriate user testing. There are plenty of solid arguments and evidence for this, but for now I’ll just quote Jakob Nielsen:
One of usability’s most hard-earned lessons is that “you are not the user.” If you work on a development project, you’re atypical by definition. Design to optimize the user experience for outsiders, not insiders. The antidote to bubble vapor is user testing: find out what representative users need. It’s tempting to work on what’s hot, but to make money, focus on the basics that customers value.
The development of Absa Online saw up to 140 individuals, working across three continents, putting in an astonishing 450 000 man-hours of development work. Four million lines of code were written in this, a “first-of-its-kind” technology deployment in South Africa.
Well, that is just silliness. As one of the commenters on the article points out, it’s Bill Gates of all people who said, “Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.”
The size of the project, how long it took, and how many people worked on it is completely immaterial. What matters is if the thing works well to help real users accomplish their goals. I really hope it does, because this looks like an outrageously expensive project. Hopefully it doesn’t become South Africa’s version of the Four Seasons $18m redesign.
I’m a big fan of Stephen Few and his approach to data visualization. His book Show Me The Numbers: Designing Tables and Graphs to Enlighten has been immensely helpful in my work as a user researcher, and I’ve been lucky enough to attend one of his excellent seminars. So I’ve been really interested to hear his viewpoint on the latest Infographic craze that’s taking the pageview world by storm.
But now, Stephen finally weighed in on his blog with a very sensible argument about the nature of data visualization, and where common web infographics fit into that universe. He starts his article Data Art vs. Data Visualization by making an important distinction:
There are as many definitions of data visualization as there are definers, but at the root of this term that has been around for many years is the goal that data be visualized in a way that leads to understanding. Whatever else it does, it must inform. If we accept this as fundamental to the definition of data visualization, we can judge the merits of any example above all else on how clearly, thoroughly, and accurately it enlightens.
By data art, I’m referring to visualizations of data that seek primarily to entertain or produce an aesthetic experience. It is art that is based on data. As such, we can judge its merits as we do art in general.
He goes on to give three reasons why it’s harmful when data art masquerades as data visualization.
We get approximately the same type of pleasure from talking about ourselves on social media as we do from having sex.
That just didn’t sit right with me. I’ve posted plenty of updates on Facebook and Twitter, and it definitely didn’t… ok, I’ll stop there. I don’t want this to get awkward. All I’m saying is that this doesn’t feel right. So I decided to trace the statement back to the original study that it’s talking about, to see what’s going on.
In a series of experiments, the researchers found that the act of disclosing information about oneself activates the same sensation of pleasure in the brain that we get from eating food, getting money or having sex. It’s all a matter of degrees of course, (talking about yourself isn’t quite as pleasurable as sex for most of us), but the science makes it clear that our brain considers self-disclosure to be a rewarding experience.
The LA Times links back to the original research paper, which has the decidedly less sexy title Disclosing information about the self is intrinsically rewarding (PDF link). The paper explains that the study was about whether or not people would give up money to talk about themselves (my emphasis added):
Just as monkeys are willing to forgo juice rewards to view dominant groupmates and college students are willing to give up money to view attractive members of the opposite sex, our participants were willing to forgo money to think and talk about themselves.
The word “sex” appears six times in the 6-page paper, and only once in the context that these other news stories use it. The “Discussion” section starts off as follows (my emphasis added):
Despite the frequency with which humans disclose the contents of their own thoughts, little has been known about the proximate mechanisms that motivate this behavior. Here, we suggest that humans so willingly self-disclose because doing so represents an event with intrinsic value, in the same way as with primary rewards such as food and sex. Intriguingly, findings also suggested that both parts of “self-disclosure” have reward value. Although participants were willing to forgo money merely to introspect about the self and doing so was sufficient to engage brain regions associated with the rewarding outcomes, these effects were magnified by knowledge that on’s thoughts would be communicated to another person, suggesting that individuals ï¬nd opportunities to disclose their own thoughts to others to be especially rewarding.
Note that they talk about “rewards”, not “pleasure” like in the news stories. The core research hypothesis is that sharing about ourselves has intrinsic value. To quote from a different section (my emphasis added):
Interestingly, a number of earlier researchers have put forward the hypothesis explicitly tested here — that self-disclosure will act as an intrinsic reward; however, despite calls to do so, this notion has not previously been tested empirically. As such, the current study validates a long-standing hypothesis that self-disclosure arises — at least in part — from the subjective value associated with it.
No mention of sex there, whatsoever. It’s also important to know what the phrase “intrinsic value” means in the philosophical sense of the word, because it’s essential to understanding the results of the study:
The intrinsic value of something is said to be the value that that thing has “in itself,” or “for its own sake,” or “as such,” or “in its own right.”
So let’s be very clear about what this research shows. The hypotheses tested (and confirmed) is that people like talking about themselves on social media because it has intrinsic value. In other words, we like sharing because it’s enjoyable for its own sake as a social activity. They make the point (in passing) that this is similar to other activities with intrinsic value such as food and, yes, sex. It’s not that posting on Facebook makes you feel the same way that having sex does. It’s that all these things share a common thread: the subjective, intrinsic value that they possess.
But hey, that message isn’t nearly going to rack up the same number of page views as saying that “We get approximately the same type of pleasure from talking about ourselves on social media as we do from having sex.” I read so many “a new study suggests that…” articles that I just take at face value. Today I decided to read the actual research paper, and realized what kind of distortions happen the further you get from the source of a story. I’ll certainly be a lot more cautious about these kinds of stories going forward. Welcome to the new age of journalism, I guess.
The Heavy Chef asked me some questions about design (and a little bit about this site). If you’re interested, you can read the interview here. Here’s a tiny excerpt:
I think the biggest epidemic in the design world right now is that we open our design software too early in the process. We have to spend time understanding the problem and user needs first, before we grab the mouse. There are so many products out there that look great, but don’t really solve a user need.
Instead, designers should raise their voices much earlier in the strategy discussion, and bring their design thinking skills to the essential practice of finding what Marc Andreesen calls product/market fit. Oh, and we need to use more paper to share those ideas. Sketches are fantastic low-fidelity prototyping tools, and it’s cheap to test and iterate on.
The basic premise of design research is that spending time in the contexts where people do the things that they do can inform and inspire the design process with a nuanced understanding of what drives people’s behavior ”” which can then be used as a foundation for understanding and exploring the opportunities for new products and services.
I don’t think you can overstate the value that in-person, observational research brings to product design.
