I wrote an article on analytics and usability testing for the UserVoice blog, so I wanted to share it here as well. From How = What + Why (Or, Where to Start Improving Your User Experience):

The first thing to clear up is this: in order to know where to start to improve the user experience of your product, you’re going to need more than a single data source. No single method can tell you the whole story you need to make good decisions. Usability testing will show you where there are interaction problems, but it won’t show you the magnitude of those problems. Web analytics will show you lots of numbers, but it can’t tell you how to fix issues you encounter. Even worse, it’s not always clear how to interpret analytics in isolation, so you might not even be able to tell if some things really are issues in the first place.

There is hope, though. It’s in the combination of these two methods that we find the elusive answer to our question. Combining usability testing with web analytics gives us the insight we need to make the right UX decisions about our products.