Giff Constable makes the case for validation and learning in the product development process before you launch something. From You Are Spending 3x-5x More Than You Should:
Agile/lean has helped people debunk the “big upfront design” phase, but far too many replace it with nothing. I agree that waterfall-style, big-upfront-design is a waste of time and money. I do believe that getting into the market with a designer and an engineer and learning is a critical use of time.
Mark Boulton’s How we work is a great post about their design process. I particularly like his point about personas, a method that I have defended before as well:
The tool is not the important thing here, [what’s important is] how you can use something to help people think of other people. To help an organisation to think of their customers, or designers to think of the audience they’re designing for, or the CEO to think in terms of someone’s disability rather than the P&L.
What I find generally useful about running a workshop like this is that it exposes weaknesses in an organisation. If a client pays lip-service to a customer-centric approach, it will soon become very evident in a meeting that that’s what’s going on.
I also like his view on agile in an agency environment: “We make things and then fix things as we go.”
Vijay Sundaram takes the concept of technical debt and applies it to User Experience Design:
You know you’re a subprime borrower when your team decides to “clean this up next release”. But the next release has a backlog of critical bug fixes, must-have feature requests, and of course plenty of new design shortcuts that emerge in the process. What’s more, the usage data for the last release is showing great engagement and no one has complained yet about the shortcuts you took. Fixing those can’t possibly be a priority right now. Before long, the team is grafting one design shortcut onto another, and the only way to unwind them is a wholesale reboot.
The article made me think of Henrik Kniberg’s excellent Good and Bad Technical Debt (and how TDD helps), in which he explains the concepts of a “good mess” and the “technical debt ceiling”:
A fresh mess is not a problem. It’s the old mess that bites you. […]
If you are coding up a new feature, there are lots of different ways to do it. Somewhere there is probably a very simple elegant solution, but it’s really hard to figure it out upfront. It’s easier to experiment, play around, try some different approaches to the problem and see how they work. […]
The problem is, just like in the kitchen, we often forget to clean up before moving on. And that’s how technical debt goes bad. All that “temporary” experimental code, duplicated code, lack of test coverage — all that stuff will really slow you down later when you build the next feature.
So regardless of your reason for accumulating short-term debt, make sure you actually do pay it off quickly.
I’ve been wondering how this applies to UX debt. Is it ok to take some design shortcuts, knowing that you’ll come back later and fix it (up to a “UX debt ceiling”)? My gut feeling is to say no, because design is so path-dependent. Your early choices constrain your later design choices, so if you make the wrong choice it can really hurt down the line. For example, if you skip user research in the beginning of the project, you’re highly likely to make the wrong choice about what to build, which isn’t something you can fix easily later on.
The solution is to have shorter UX (and development) cycles, which is becoming commonplace anyway with the rise of the Lean movement. I’m reminded of IDEO Design Director David Aycan’s 2010 article in HBR, Don’t Let the Minimum Win Over the Viable, in which he explains the importance of taking small steps so that it’s easier to correct course when you realise you’re on the wrong track:
Sketching or mocking up experiential prototypes and then testing them with consumers or potential partners, while also explicitly jotting down your operating and business assumptions and using them to discuss the business with industry experts, allows you both to pick a promising route to invest in the development sprint and to pivot with confidence. For example, by prototyping multiple consumer experiences and business models before investing in an initial MVP, GoGo was able to launch an airline WiFi and in-flight service experience that combined the best of multiple alternative services that might be offered in flight. One might think of this approach as testing multiple MVPs in parallel.
Creating multiple options in tandem creates more confidence in the core variables, which in turn means that pivots may be less drastic or disruptive later on. This approach can be applied beyond product features to business models and operating approaches as well.
He uses this sketch to illustrate the approach:
This makes a lot of sense in the context of UX and technical debt as well. A pragmatic approach to dealing with software debt seems to be something like this:
The trick is to be ok with the early messes, but not let them deteriorate so much that you’re unable to clean things up when you finally decide it’s time. Remember the broken windows theory: “maintaining and monitoring urban software environments in a well-ordered condition may stop further vandalism neglegence and escalation into more serious crime technical issues.”
I’m a big fan of the recent move away from user stories to job stories to design better products. Alan Klement provides a good overview in Designing Features Using Job Stories. That said, I’m worried that personas are on the verge of extinction as collateral damage of this evolution. We can’t let that happen. Alan explains his issue with personas as follows:
The biggest and most pertinent problem with Personas is this: Personas are imaginary customers defined by attributes that don’t acknowledge causality.
These attributes, generally in the form of demographics, do not bring a team closer to understanding a customer’s consumption, or non-consumption, of a product. The characteristics of a Persona (someone’s age, sex, race, and weekend habits) does not explain why they ate that Snickers bar; having 30 seconds to buy and eat something which will stave off hunger for 30 minutes does explain why.
The problem with this argument is that it refers to marketing personas, which are generally not very useful for design. Marketing personas are usually based on segmentation data, and ends up being mostly about demographics that cluster similar groups of users together.
But we shouldn’t confuse marketing personas with design personas, which are specifically created to guide the development of product features. How are they different? Well, first and foremost, design personas are based on needs, goals, and dimensions that have a direct impact on their interaction with the product. In other words, they incorporate causality, which takes care of Alan’s gripe.
For example, below is a design persona for a short-term loan company. There are a few things to note:
My point isn’t that job stories aren’t necessary. On the contrary, I think job stories are much better than user stories for product design. But job stories are a valuable augmentation to design personas, not a replacement for them. There is still a huge amount of value in personas. They have names and faces, so the whole team can picture them. As opposed to a mythical “average” user, they are solid people we can imagine using our product to achieve their goals. This is helpful because by focusing on individuals that are closer to the edges of the experience, instead of the average, we’re able to cater design for a larger portion of the user base.
In the documentary Objectified, Dan Formosa from Smart Design says, “What we need to do to design is to look at the extremes. The middle will take care of itself.” As an example, he talks about how they once designed garden shears specifically to cater for people with arthritis. They knew that if the shears worked for that “user”, it would work well for everyone. That’s the power of personas.
I understand and agree with the concern that personas can sometimes be oversimplified caricatures of users that don’t take specific situations and actions in consideration. Without proper research personas also tend to be be shallow and not very useful. But those are dangers that are easy to avoid. Remember that personas aren’t prescriptive, they’re descriptive. You can’t identify a persona and then try to predict people’s behavior off it. But with solid research and analysis you can use personas effectively to help focus development efforts on target users, and help define what features should be included in (and just as importantly, excluded from) the product.
As a side note, in addition to the job story format I also sometimes like to use what I call problem stories. These are like user stories, except that they incorporate “triggers”, which takes causality into consideration. The format I use for problem stories are:
User has problem when trigger.
For example, a Product Manager on a financial services product might have a problem story that states, “Investors are not able to submit supporting documents online when they need to make changes to client portfolios.” That becomes a statement of the problem that needs to be solved through product improvements, and a good way to develop features by focusing on user needs.
All this to say that job stories (and problem stories!) are great ways to guide product and feature development. But if we use them to replace design personas, we’ll be throwing tons of useful context and understanding out along with it.
The jobs-to-be-done framework isn’t new, but I’ve only recently started digging into it much more since it’s been gaining a lot of traction everywhere I look. For a nice primer on the topic see Eric Portelance’s recent article for Teehan+Lax called The Iceberg of Jobs-to-be-Done, in which he explains how crucial this framework is for good product design:
[Most successful products are created by] people who understand the importance of creating products that solve real customer problems, and have a set of tools and frameworks like jobs-to-be-done that they use to identify and validate the real human problems they’re trying to solve in the market.
The progress-making forces diagram has been particularly useful for me in client work, since it helps people understand how difficult it can be to change existing user behavior. I’m not a huge fan of the diagram on the JTBD site, so we made a new one:
The basic premise of the diagram is this. For someone to move from their existing behavior (a product they’re currently using) to new behavior (switching to a new product), there are two types of forces at work: progress-making forces, and progress-hindering forces.
Progress-making forces move people from their existing behavior to the new behavior, and consists of the push of the current situation (things they’re not happy with in the current product) and the pull of the new idea (things that sound appealing about the new product). Progress-hindering forces, on the other hand, hold people back from switching to new behavior. It consists of allegiance to the current behavior (things they really like about the current product) and the anxiety of the new solution (worries about learning curves and not being able to accomplish their goals with the new solution).
What this comes down to is that for someone to switch from an existing product to a new product, the progress-making forces have to be stronger than the progress-hindering forces. This might seem obvious, but applying this model to your product planning can inject an extremely healthy dose of reality. Is the product really that much better than a current solution? What does the new product have to do to overcome people’s allegiance to what they’re currently using?
In the context of product design this can be a crucial component to making a go/no-go decision on whether to go ahead with an idea, so it’s always a mental test I run with the teams when we’re working through our planning.
Getting feedback is an essential component of good design. No matter how smart we are, we are going to get too invested in our solutions, and we need the help of knowledgeable outsiders to nudge us in the right direction. The problem is that feedback sessions can get out of hand quickly, because we’re just not very good at providing (or receiving) feedback. We are prone to seeing the negative parts of someone’s ideas first, so we often jump straight into the teardown. This puts the person who is presenting their designs in defensive mode right away, which usually starts a negative spiral into unhelpful arguments and distrust.
There is, however, a better way. In an interview on criticism and judgment, French philosopher Michel Foucault once laid out the purpose of any good critique. In his view, criticism should be focused not on what doesn’t work, but on how one can build on the ideas of others to make it better:
I can’t help but dream about a kind of criticism that would try not to judge but to bring an oeuvre, a book, a sentence, an idea to life; it would fight fires, watch grass grow, listen to the wind, and catch the sea foam in the breeze and scatter it. It would multiply not judgements but signs of existence; it would summon them, drag them from their sleep. Perhaps it would invent them sometimes — all the better. Criticism that hands down sentences sends me to sleep; I’d like a criticism of scintillating leaps of the imagination. It would not be sovereign or dressed in red. It would bear the lightning of possible storms.
Keeping this purpose in mind, I particularly like the process used by Jared Spool and his team at UIE. The team uses this specifically for design critiques, but it can be applied generically to any kind of feedback session. Here’s how the process works:
Let’s not forget that as designers we are responsible for making sure feedback sessions happen, and that they happen in a respectful and useful way. Scott Berkun has a great set of ground rules about critiques that are worth remembering:
If we change our approach to provide critique, not criticism, we’ll be able to build on the best ideas of others, and iterate faster to better products. So remember: design like you’re right; listen like you’re wrong.
Amen to everything in Mona Patel’s article The Lean Agency:
While being lean is awesome, being innovative means that spending time and money on smart research and devoting ample time to thinking through the problem space can sometimes mean the difference between a good design and a great design. Our focus is not just on making something usable, but on creating value for a business and really impacting people’s lives.
And a double Amen to this:
Yes, agencies typically end engagements with deliverables. But, we don’t charge our clients just for the deliverables. We charge them for the value that we provide and the objective insights, fresh perspectives, and innovative solutions that we offer. We provide an innovative vision for an exceptional user experience in the form of an artifact, or deliverable. The presentation of our insights and recommendations in a solid deliverable can often be the tipping point for organizations seeking to change their product or experience for the better.
This is how we work at Flow as well. And especially since we started using Expanded User Journey Maps, good deliverables have become the difference between a successful and unsuccessful project.
I realise it will ruin some of my coffee street cred to say something positive about Starbucks. However, their use of ethnographic research outlined in Maria O’Connell’s Not Just Coffee: Starbucks’ Rise to Success is commendable (and clearly successful):
Starbucks interviewed hundreds of coffee drinkers, seeking what it was that they wanted from a coffee shop. The overwhelming consensus actually had nothing to do with coffee; what consumers sought was a place of relaxation, a place of belonging. They sought an atmosphere.
The round tables in a Starbucks store were strategically created in an effort to protect self-esteem for those coffee-drinkers flying solo. After all, there are no “empty” seats at a round table. Service counters are built out of natural materials like warm woods and stone, rather than plastics and metals, to create a homier atmosphere.
It’s still so frustrating to see how many companies embark on their redesigns or MVPs without doing contextual research first. You might get the usability of your product right, but without utility, it will still be useless. As Milica T. Jovanovic points out in Better safe than sorry:
Startup culture is using a bunch of clichés to tell (mostly) young people that it’s ok to invest an enormous amount of time and energy into something and then let it fail. Well, it’s not ok. It’s bollocks. There is nothing wrong in investing your time and effort into something you are passionate about, but you can make sure that the risk of failure is as small as possible.
In short, do your research first!
Related reading from the Elezea archive: Coffee, sense of place, and designing whole experiences
In How I Made $4000 Selling A Product I Didn’t Have one entrepreneur explains how his new startup deceived users into thinking the product already existed (even though it didn’t). They did this so that they could collect credit card details to validate whether or not users would actually pay for the product. You should really read the whole post, but here’s a key section:
It doesn’t feel good to deceive prospective customers (or anyone for that matter). I didn’t like this bit. Then again, is there really a big difference between this and in putting up a landing page to test a new idea? I don’t know. I think if your intention is right (i.e. your heart is in the right place), then this deception is more of a white lie.
Is this what it means to be a lean startup these days? It’s at worst fraud, and at best an extremely dark pattern. I get the need for validation before launching a product — I’m a big proponent of it. But the user-centered design and Lean UX methodologies both give us great ways to do validation in an ethical and honest way: through prototype testing with potential target customers.
Prototype testing helps us find out if a product is useful before we launch it — whether it has good utility as well as good usability. Sure, it doesn’t give us absolute validation on whether or not someone will actually pay for it, but that’s unfortunately part of the danger and excitement of creating software. Or are we really at the point where we agree with the ancient Greek tragedian Sophocles when he said, “Profit is sweet, even if it comes from deception”? I hope not.
I just don’t think deception of any kind is ok, even if “your heart is in the right place”. This isn’t user-centered, it’s persuasion. And as Cennydd Bowles put it in The perils of persuasion:
What privileges the designer [or the entrepreneur] to dictate desired behaviour? And since we’re for hire, does that mean we’re ethical relativists, bending people toward whatever agenda lines our pockets?
Profit is sweet, even if it comes from deception.
This isn’t really a post about one entrepreneur’s methods. I’m more interested in where the line is here, and I think this is crossing a very dangerous one. Where does this approach end? At what point will we, as users, constantly have to worry that every time we enter our credit card details online it might be for a product that doesn’t actually exist? Even if it isn’t fraud, that’s not the type of relationship I think we should build with our customers.
Incidentally, I recently watched Mike Monteiro’s excellent talk at Webstock called How Designers Destroyed the World. It’s embedded below — please watch it. But I’ll close with this quote from the talk that I find very relevant to this discussion:
We need to fear the consequences of our work more than we love the cleverness of our ideas.
We’re responsible for the work we put into the world. We always have a choice to be honest or deceitful. And we have to consider how those choices add up in the long term. That’s our job.
I agree with every glorious word in Erika Hall’s How the ‘Failure’ Culture of Startups Is Killing Innovation. It’s a brilliant and impassioned defense of making design research part of every product development process:
Somewhere along the way, it got to be uncool to reduce one’s risk of failure.
Part of this may be because the risk of failure is dramatically lower than it used to be. But another reason is that many people don’t actually understand what research is, and have somehow conflated concepts like “rapid prototyping,” “lean startup,” “minimal viable product,” and “[insert] other smart-sounding thing to do” with avoiding research.
It’s really hard to not just go ahead and quote the whole article, because it’s so good. I’ll leave you with just one more bit and an enthusiastic recommendation to read the whole thing:
Maybe knocking out a prototype or building a company is the fastest, cheapest way to learn. But often it’s not. Sure, a prototype can tell us if the user understands the potential solution — but if it’s solving a problem no one has, why bother building it in the first place?
Ok, I lied. One more bit:
It becomes immediately apparent, when we try to understand our fellow humans through research, that we are not rational creatures. But when it comes to making business decisions, research helps address that irrationality and increases our chances to succeed. And make no mistake: in a world where design makes or breaks success, all product design decisions are business decisions. Asking the right questions will lead to useful insights.
