Menu
Search
2 February 2015

The logistics of usability testing on mobile devices

I’m currently working on the design of a native app for Jive Software, and we really wanted to do some usability testing on a prototype before development starts. There are, of course, a multitude of reasons to do usability testing on prototypes, but we had some very specific issues we wanted to address:

  • We are a distributed team. Product Management is in Palo Alto, Product Design is in Portland, and Development will happen in our Tel Aviv office. So we knew from the beginning that a functional prototype would be essential to communicate within the team. Flat wireframes / PSDs just weren’t going to cut it.
  • We’re working on something in a fairly new market, so we need to do quite a bit of validation to make sure we get the utility of the app right.
  • And of course, our designs are never as good as we think they are, so we wanted to make sure we correct the majority of the usability issues before development starts.

I read a lot about prototyping native apps and mobile usability testing, and since everyone’s process is different, I wanted to give an overview of the process we settled on, since we’re really happy with it so far. It was very important to me that all our offices would be able to observe the usability tests, so that guided a lot of the decision-making. Here’s how we’re doing it…

Prototyping

After reviewing and trying out every prototyping tool known to man, I settled on Proto.io for this project. There are so many great options out there that it’s hard to go wrong on a choice of tool. Proto.io was the best for me because of a few key features:

  • I needed full interactivity — since we used the prototype for usability testing it needed to feel as real as possible.
  • I needed lots of flexibility in the animations/transitions supported, since we’re in new territory and need to try out lots of different things. Proto.io supports any interaction I threw at it.
  • I needed to use a mix of built-in components and my own assets, and Proto.io handles that pretty well.

Of course, I can’t show what the prototype looks like at the moment, but as soon as the app launches I’ll update the post and embed it here.

Mobile usability testing rig

I got lost in mobile usability testing guides for days — there are so many good ways to do it. At first we considered remote testing, but it just wasn’t a good option for us because we were going to do part IDI (in-depth interview) and part usability testing, so we needed a way to be in the room with users and dig deep into certain areas.

I’ve seen some crazy setups — my favorite and weirdest is probably MailChimp’s “hug your laptop” idea. It’s a brilliant hack, but I was worried our non-tech savvy users would have trouble with this, so I needed another solution.

I ended up going with Bowmast‘s Mr. Tappy kit, and I attached a Logitech HD Pro Webcam C920 to it. We played around with it in the office first to make sure it’s going to work:

With that out of the way, it was on to the next challenge — how to stream it everywhere.

Streaming to observation rooms

This part was much easier than I thought, simply because we already use Vidyo for videoconferencing in all our offices. So every time I started a usability test I would start my meeting in Vidyo, and then people from our other offices could dial into my meeting from their observation rooms. They could see the room on one screen, and the participant’s phone on another. It worked like magic:

What we ended up with is a setup where I can do usability testing in person on mobile devices, record the sessions, and have people observe these sessions from anywhere in the world. It was an incredibly productive few days, and I’m now working my way through fixing all the usability issues we picked up. Can’t wait to show this to you when it goes live!

Further reading: