Andy Beaumont wrote a great piece about his popular Tab Closed; Didn’t Read Tumblr site, which documents websites that obscure their content behind modal overlays. His point on analytics in The Value of Content is spot on:

Analytics only tell you part of the story — if that’s all you bother to find out, and you have absolute faith in those numbers, then you’re going to end up putting a modal overlay on your site. Analytics will tell you that you got more “conversions”. Analytics will show you rising graphs and bigger numbers. You will show these to your boss or your client. They will falsely conclude that people love these modal overlays.

But they don’t. Nobody likes them. Conversions are not people. If you want the whole story here you should also be sat in a room testing this modal overlay with real people. Ask them questions.

Once again, this points to how important research triangulation is to make good decisions based on insights, not just data. Real insights are found at the intersection of different research methods. Not over in the corner with just one method.