Recently, my buddy who I profiled in an earlier blog post, sent me a link to a user-test. Actually, it was a Facebook update that looked like this:

The link takes you to a start page shown below:

Because I’m interested in karma points, I became a member of fivesecondtest.com.  I clicked on Start Here and it shows:

Ah ha! The test tricked me.  I clicked on “Show Image” and was shown a screenshot in my browser window. But the screenshot was too big and I had to scroll back and forth to see it.  I wasted my 5 seconds and blew the test. And then, the screenshot went away and I was asked,”what is the purpose of the website?” and given a text input area.

The prompt was: “Take a look at the website screenshot, and look at the design/colors/text.”  The questions were:

• What is the purpose of the website?
• Do you like the color scheme?
• Is it too cluttered?
• Is there too much text on the left of the iPhone image?
• Is the navigation bar easy to understand?

Yep, I didn’t read the instructions. If I had, I would have known to concentrate and look at for 5 seconds. Duh, it’s called fivesecondtest.com! Sometimes, I’m late to the party.

My immediate reaction was, “How can you ask these questions based on viewing the screenshot for 5 seconds?”.  What questions would you ask after only giving the user 5 seconds to view the site and test?  Can you get reliable data based on this type of test? How does the social aspect and the repetitive nature of the tests affect the results?

Now, I’m interested and I decided to do another.

The next test reiterated the 5 second rule in its prompt.  Prompt: “You have 5 seconds to view the image. Afterwards, you’ll be asked a few short questions”  This test asked:

• What does this company do?
• What did you think of the layout and design?
• What stood out most to you?
• If you were looking for a website, would you continue to go further into the site to find out more?
• Does the website look professional?

Okay, I’m getting the hang of it now.

I do another one – this one the prompt is, “You are a young woman who regularly purchases beauty products (skin care/cosmetics)”.  And, the questions:

• How would you rate this page on a scale of 0 (really bad) to 10 (really great) ?
• Was it clear what the product was?
• What would you do next?
• What did you like most on the page?
• What did you like least on the page?

I saw my buddy on Facebook and we chatted about it.  He said, “You get a lot of BS answers because people really don’t have incentive to answer thoughtfully.”

He also gave me an idea of how it works using the ‘community option’ service level of fivesecondtest.com. You get 30 responses for free for signing up.  And then, for every test that you take,  your test can be given. Paid plans start at $20 (100 test responses) a month and go to $200 (1000 responses) a month.

To summarize my opinions, I feel that this type of test could be a part of a good user testing plan. I was not impressed with the questions in the three tests that I took. Following the G.I.G.O ( garbage in garbage out ) principle, bad questions will yield bad answers. I doubt that the questions / answers will yield any actionable information beyond confirming pre-existing assumptions about the sites. However, I feel that a well-designed question that takes into account the nature of the test could produce actionable results. question should be general and open-ended. They should deal with impressions and feelings, rather than specifics.

Finally,taking my opinions into account, here are my top five questions from the 15 I saw:

• Does the website look professional?
• What stood out most to you?
• What did you like most on the page?
• What did you like least on the page?
• Was it clear what the product was?