The limits of user testing: Iteration vs. Innovation

Newman and I talk a lot. And I mean a lot.

We got into a conversation on Wednesday about iteration and innovation.

At the core of the discussion is a new client, LessAccounting. When they hired me, Allan Branch, their CEO basically said to me, “show me what you’ve got”. He ordered user testing omakase. Now that’s baller.

It was at that moment that I understood why sushi chefs feel the need to rise to the occasion when an order comes in omakase. The term in Japanese literally means “I’ll leave it up to you”. But there’s more than a whiff of gauntlet throwing in it too. It means exactly what Allan said, “show me what you’ve got.”

And that’s exactly what I set out to do.

First things first. Newman and I crawled all over their website. I asked a bunch of question. We did competitor research. Nice.

And pretty quickly it seemed apparent: I wanted to do a new site from scratch.

I felt (and feel) that I can justify that opinion. But something didn’t sit well with me. User testing is by its very nature about the user. The “show me what you’ve got” isn’t meant personally. I’m not supposed to have the right answers. I’m supposed to know how to find the right answers.

Allan and I talked about it and he just kept being a baller. He said, “Big changes are fun, little changes are easy. Let’s redesign it. Do you have wireframes?”

That led to the wireframes that I was drawing on Wednesday and ultimately to the conversation with Newman.

While considering the new design a thought popped into my head that an old friend of mine used to say all the time.

“You can’t get there from here.”

And it all became clear.

It’s all about iteration and innovation

Innovation is required when iteration won’t suffice. The way Newman and I talked about it was a little odd but it illustrates the concept quite clearly.

Think of the current LessAccounting website as a giraffe.

Think of the website I want to design as a monkey.

There’s no way to iterate a giraffe into a monkey. You can’t get there from here. Even evolution doesn’t work like that.

If you want a monkey website instead of giraffe website, you have to start from scratch. Once it’s built, then you user test the site until it’s the 800 lb. gorilla that you want it to be.

And now for something sorta topical

Waves were made not too long ago about something the CEO of Zynga supposedly said to an employee.  In a San Francisco Weekly article, CEO Mark Pincus is quoted as telling an employee, “I don’t fucking want innovation. You’re not smarter than your competitor. Just copy what they do and do it until you get their numbers.”

As harsh as that sounds, run it through the marketing filter and you have to admit – there’s a logic to it. A leaked Zynga memo making the rounds where their CEO justifies their culture of iteration. Generally the press has been really hard on the guy. They use quotes like this to fog up their glasses.

Google didn’t create the first search engine. Apple didn’t create the first mp3 player or tablet. And, Facebook didn’t create the first social network. But these companies have evolved products and categories in revolutionary ways. They are all internet treasures because they all have specific and broad missions to change the world.

We don’t need to be first to market. We need to be the best in market. There are genres that we’re going to enter because we know our players are interested in them and because we want and need to be where players are. We evolve genres by making games free, social, accessible and highest quality.

My guess is, we Americans like our business leaders to be innovators. We celebrate Thomas Edison, Henry Ford, and Steve Jobs. This guy, Mark Pincus doesn’t make the top 500. But on the other hand, Zynga is single-handedly owning the casual gamer market. Everybody from EA to Nintendo chases these guys. So clearly there’s something to be said for iterating.

On the other hand, you’re probably reading this on an Apple product, so innovation is a powerful tool too.

The trick is to know when you can’t get there from here. When you can, iterate. When you can’t, innovate your way into the ballpark and then iterate until you’ve hit the bulls-eye.

4 comments on “The limits of user testing: Iteration vs. Innovation

  1. Pingback: Hands on Review of the InVision Wireframe App | A Better User Experience

  2. Pingback: Rethinking Personas for Small Business, Part 2: Be the Bieber | A Better User Experience

  3. Pingback: The Nuts and Bolts of How to Design a User Survey | A Better User Experience

  4. Pingback: 6 Reasons Your Website Sucks (And Why Replacing It Won’t Help) | A Better User Experience

Leave a Reply