Show of hands: Who is afraid of user testing surveys?

I was. To me, they’re long and tiring. They’re hard to write. It’s hard to get people to take them. It can be a tough job to figure out someone’s experience of a web site using only a few questions on a page. But, you know you need them, kinda. So how do you start? One good answer is Three Quick Questions.

Hello there, Three Quick Questions

Three Quick Questions is a new micro-survey tool that understands the challenge of surveying users about a website.  Designed to fill the gap between a full blown usability survey and a simple A/B test, the tool presents your website, asks three questions, collects the open-ended answers and measures the time to completion.

You give them

1 website, 3 questions, and $5.

They give you

answers from 10 participants within 4 hours

Our experience with Three Quick Questions

For the past few months, as you may know from our series of articles on the topic, we’ve been going through a website redesign process with LessAccounting. Just two days ago, Ben wrote an in-depth article on how to write a survey. And to be honest, it’s a little intimidating.

What if there was a way to gather some survey data quickly? What if you don’t want or need a huge, feature laden-test and all you really need is some quick feedback? Then I have good news for you. Three Quick Questions will be your Superman.

Our Three Quick Questions

1. Name at least three features of the LessAccounting app.
2. What is LessAccounting?
3. Who is the target user for this app?

All three of these questions target one of the 6 Things Your Home Page Must Have (to Keep from Sucking).

We wanted to know if users could identify: what the site does, who it’s for, and its value proposition.

The Results

Here is the results page. Let’s take a closer look.

Question 1

Question 2

Question 3

What Does It All Mean?

According to Three Quick Questions, the time taken to answer the questions is a good rule of thumb on how difficult it is for the user to answer the question. Along with the times, they also state:

On a well-designed website that easy to navigate, it should be possible to answer a simple factual question in less than 20 seconds. Between 20 and 40 seconds is OK; longer than 40 seconds is bad and risks losing visitors.

Scroll down to find detailed timings for each question. Don’t worry if a couple of the individual tests show a very long time – these can be due to slow internet connections and technical glitches. If several visitors got an incorrect answer for one of the questions, consider making that piece of information more prominent, or reviewing the way it is presented.

By that scale, the results suggest that it’s a bit hard for users to both know what LessAccounting does and to list 3 specific features. As for who they’re for – the score came back within an acceptable range. We should investigate further just how easy or hard it is for users to identify specific features of what LessAccounting does. The same is true of what they do. There seems to be a lack of message clarity. When you look at the specific answers, the majority of people got the right answer but for some, it took as long as 3 minutes!

I have to be honest. I’m willing to believe that the site can be designed better. But I also have to wonder at some of the users when it takes 3 minutes for somebody to answer a question that it actually in the main heading on the website. Further testing will reveal if this is a problem.

The other interesting thing of note is that there was a wide range of answers for who the target market is. The answer we were looking for was “small businesses”. Assuming people understood the question, the fact that we got a range of answers – even though it didn’t take very long – suggests that this too lacks clarity.

Summary of Three Quick Questions app

My assumption going into this test is that the website lacks clarity. This opinion is largely based on the fact that the bounce rate of the front page of the website is around 55%. What we’ve found is that these Three Quick Questions generally validated my assumption.

So that’s cool.

But, I can’t shake the feeling – based on how long it took users to answer the questions – that there was a problem with either (a) our questions or (b) our users.

And the truth of the matter is, everything I just tested is going to require further testing in order to get something conclusive.

So where does that leave us?

My God, it’s full of stars!

Easy to use: 5 /5 – Okay, the tool is simple to use. It’s straight forward to use and the results are easy to read and understand too.

Cost: 5/5 – It’s $5. I sound nonchalant about the results because my assumption was validated. But can you imagine if I wasn’t? That’d really set the wheels spinning. It’s totally worth $5 to get your bearings.

Value of the Results: 3/5 – On the downside, the results are always going to be hazy. This is due to the fact that we don’t know anything about the survey takers or the conditions under which they are taking the survey. It’s totally possible that the people who took the longest did so because they took time out to play their next move on Words With Friends. We just don’t know.

Conclusion

Three Quick Questions is great for anybody who is looking for quick feedback. The feedback itself will be a little hazy, but it will definitely give you insights on where to explore further. But for something that’s really just three quick questions and only costs $5 to have the potential to deliver real insight, it’s a no-brainer for designers and website owners who want some quick feedback on their site.

4-Stars