5 Things We Learned From Our First User Test

Do you remember what a mystery sex used to be? (Don’t worry Mom, it still is, I swear. ) You’d talk about it endlessly with your friends just telling and halfway believing each others lies. And because you didn’t know what you were talking about, you boned up on your reading. [Boned up?  good grief – ed] All of the sudden, Cosmo – either your sister’s, your Mom’s with their “100 ways to spice up your sex life” was interesting reading.

You mean they just make lists of this stuff and print it for you to read?

At some point, the reading up becomes boring. The boundaries are well known and what used to be exciting is now just another list of stuff that other people are doing right.

That’s when you know it’s time to get down to business.


That’s basically where Ben and I have been with this user testing stuff for the past few weeks. You can hear it in the podcasts. Enough with the reading! Let’s make with the fu…n user tests!

And so we did.

On Wednesday, we showed you the video of how it all went down. If you missed that post, we’ll save you the trouble. Here ya go.

<iframe width=”560″ height=”315″ src=”http://www.youtube.com/embed/z4bRPPL5pbg” frameborder=”0″ allowfullscreen></iframe>

Could you tell how vulnerable I was in that clip? You just wait Kate, let’s do it again in a few years. I’ll be way better.

Coming out of that experience, we learned a lot. Here’s a list of the Top 5 takeaways.

5. Be prepared

I suppose that we were preparing the whole 4 months leading up to this first user test.  We had practiced on ourselves and friends, but this was the first ‘real’ test with a stranger in a strange place.   The rehearsals were key in getting ready the mechanics and hardware.  But there is no way to prepare completely. There’s no way around sucking the first time and being un-smooth.

Smoothness didn’t seem to matter, luckily.  Perhaps authenticity trumps smoothness.  If you’re genuine and honest, people may overlook a herky-jerky delivery.

4. Getting permission is the hardest part

Getting the permission to record and use the coffee shop was tough.  We had to explain who we are and what we are doing.  If you’re not used to doing that, it can seem really awkward. But you have to do it.

It’s true that you really only need to get permission one per place and that it’s not really necessary to do a user test. I mean, you can do one at home or in your office just the same. But doing it this way, it was much more of a challenge than actually conducting the test.

This is an example of activation energy. The hard part is starting and now it’s much easier to keep it going.

3. It Gets a Reaction

While we were at the coffee shop we ran into four folks who work at a hotel on the island. It was a manager and her sales team. The minute they found out that we were doing website user testing, their body language became more guarded and the manager slowly made her way towards the door. We had nothing to sell and weren’t even interested in getting them to sit down for a test but just the words were enough to cause them discomfort!

Equally interesting, after their initial reaction, they warmed up to it.  They talked about how they user tested the items that go into each hotel room. The manager told a story about having to get new alarm clocks for the rooms and that she wanted to test it before they used them. In her words, “if I can’t work it in two minutes then it’s not going in the hotel rooms”.  She found a way to apply user testing to her own field.

We’ve speculated that the strong reaction is because small business owners have their personal identity wrapped up in their business. It’s hard to willingly submit to judgment.  But, once they realize that it’s a way to improve their bottom line and that people are ALREADY judging them, they tend to get on board.

2. Good Things Come Available in Bite Size

We wanted to see if it was true that you could get value out of asking a stranger what they think of a website. Is it valuable or is it just a stunt? It turns out that we got some good data that we can give to the website owner. The moral of the story is to get out there and ask people their opinion about your website. You will learn something. Simple as that.

1. People Are Cool

Meeting people is always a highlight.  Humans are social animals.  We need to communicate and interact.  That’s a big part of what this blog is about.  The folks we met at the Daily Grind helped us dive deeper into what User Testing is all about.  We are thankful for that.  It didn’t have to go that way – we could have been chased out of there. Instead, it went really well.

It’s important to say that too. If you read most media, it’s mostly one group of people complaining about another group of people. You’d think that we are all angry people but that’s just not the case. People are cool.

Bonus: What we learned about the website in the user test

  1. Better and more headlines – People skim websites. It’s an issue of cognitive load. Successful user testing, as a rule, reduces the cognitive load a website imposes on its viewers. Better headlines would help the user quickly scan the page and drill down to the relevant content and would reduce cognitive load.   At 3:10 our user-tester, Kate, responded to a question about services with, “I probably gotta read about it first, right?”.  She was under a higher cognitive load because of the camera and the test, but that helped to reveal that one couldn’t skim the headlines for meaning. They weren’t there.
  2. Include pricing info (5:50) – As a professional we all get used to not seeing prices on a website that sell a service. “If they want to know, they’ll call” is a common thought. Be that as it may be, it’s also true that most people would prefer to at least know the general pricing structure. If you choose to not have prices on the site, you should at least address it.  “Prices range from”, “Free Consultation” or, as our user suggested “10% off” are examples.  You could also explain how your prices are structured – by the hour, by site, do you have packages of services, etc.
  3. Improve some of the photos (1:20 and 4:10) – The compass image could be replaced.  Kate mentioned the color balance being ‘off’.  We feel it draws attention without leading to the critical path of the site.  It’s not imediately clear why the compass is there.  Suggest: Either create a new photography or create a headline that explains the compass.

Implementing these suggestions would significant improve the user experience of your site.

Enjoy the holidays and be on the lookout for day-after-Christmas podcast!

Leave a Reply