When I first sat down to write this blog post, my primary goal was to answer these questions.

1. What is relationship between the results of Feng-GUI and of fivesecondtest.com?
2. Which website is better if you had to pick ONLY ONE to use?

They are certainly not the same test – One is human driven and the other is completely automated, a machine.

My secondary goals are:

1. to learn the interface (and back end) for these two tools and,
2. sharing them with you, my dear reader(s).

There’s a third goal – which normally would be the primary goal – which is to get some feedback on a client’s website – mushroommaestros.com – but this time, I consider it to be a bonus. In the interest of full disclosure I should mention that mushroommaestros.com is my brother’s site.

What I did

I’m going to take a screenshot and submit it to FENG-GUI. I’ll use their full service – which is pay as you go, $2.5 per image. I’ll buy 10 images for $25, and I’ll have to use these within one month.

Next, I’ll submit the same screenshot to fivesecondtest.com. And, write questions that would measure the user’s impression of the site.

Roadblocks or interesting questions:

Do I submit a screenshot of the whole page or just the visual portion? The limit of both of these testing options is they don’t mimic the actual performance environment of viewing a webpage. Simply put, viewing a screenshot is not viewing a webpage. When viewing a screenshot you can scroll up and down, click away or use any interactive elements. I think I’ll use the ‘visible portion’ screenshot for both tests. I may go back and change to the full page screenshot.

Intermediately as I look at the screenshot, I’m interested to change a few elements on the page. I would make the menu text bigger and I would put a stronger call to action visible ‘above the fold’.

In FENG-GUI:

I signed up for the account by using PayPal and they send me an e-mail with the account information. I uploaded a photo and following the instructions click on the analyze button. Very simple and very quick – I started (and finished) the test in less than 30 minutes.

Reading the Feng-GUI help section, I learned that they recommend using a screen capture from “above the fold”. Problem solved.

The Feng-GUI test generates several different reports

• Attention heat map – displays the most attractive elements
• Gaze plot report – shows the scan paths and order between elements and includes the shortstops (fixations)and fast movements (saccades) of the eye.
• Opacity map report-this is another way to visualize the heat map report
• Area of interest (AOI) report-this report gives a percentage of attention based on predefined areas. I didn’t do this report because I forgot to predefined my areas.

You can download a zip file of the individual report graphics – your original image with an overlay of the four reports.

Impressions of Feng-GUI:

I’m happy with the reports. I was worried that the image of a girl would distract attention from other elements. I’m happy that the first ‘fixation’ was the text, “We sell fresh mushrooms”.  Now the questions are, weathered the text is readable and attractive.   Or, rather, does it meet the goals of the page.  The goal of this page is to bring people farther into the site and make it clear of the primary business goal of selling mushrooms.

How might this affect my tests with 5 second test.com? What are the questions that I want to ask?

In FiveSecondTest.com

I had signed up for the sevice before – So I login and click create test.  There are 3 type of tests you can do – Click test, navFlow and FiveSecond test.  A start a new fivesecond’er and type in this information:

Prompt or Task:

• You searched for “fresh mushrooms in Oakland California” and clicked on “Mushroom Maestros: fresh mushrooms, urban myco-farm supplies…” (added – Please maximize your browser window and look for 5 seconds.)

Questions:

• What does this company do?
• What is the name of this company?
• Does this website look professional? or Does this company seem professional?
• What stood out most to you?

Once you input the questions – you can have a maximum of five – you may preview the entire test process before publishing the test.

On previewing the test I decided to remind the user’s to expand their browsers. The new prompt looks like this now:
You searched for “fresh mushrooms in Oakland California” and clicked on “Mushroom Maestros: fresh mushrooms, urban myco-farm supplies…” – Please maximize your browser window and look for 5 seconds.

I set the tests to give 15 responses.

Impressions of Fivesecondtest.com

After a few days and only using the fivsecondtest users, I’ve received three responses to the survey. While it’s too early to tell anything definitive, I do have a couple of thoughts. Because the ‘game’ of the site (Do other’s test to get credits to  give your tests), the responses will be from the demographic of the site users-which may not be your target audience. And, they may not really care about your test and the real purpose is to generate credits to do more of their tests… It could be a good thing but your responses may lean towards the technical and ‘hasty’. For instance, I got a response that said “needs a stronger call to action”, which is something I don’t think the general public thinks of.

I think I will hold off on distributing the test link to my extended network, so that I can continue to get this web designer, http://usabilityhub.com – user perspective.  Perhaps I’ll wait until I get 10 responders and then link to it on my facebook and twitter accounts – Or ask a friend to do it.

Summary and conclusion

What is the relationship between 5 second test.com and Feng-GUI? It’s too early to tell.  There is too little data from the human test and I don’t think that my questions really were valid to answer that primary question.

If I had to only pick one to use, at this point, I would use Feng-GUI because it very quickly gives me the specifics and actionable information. I can tell my client, “I changed this because of that.”

I’m very happy to learn how to use both of the tools.  And, I got a few ideas on how to tweak the mushroom maestros design.  Ultimately, that’s the best result – We test to answer questions and generate more questions, generating more tests and so forth.  The iterations are what I’m after.

PS: I spent a solid hour trying to get that gallery to work.  The captions wouldn’t work and I couldn’t figure out how to exclude the Watson image using the visual interface – had to use the Gallery shortcode and then I lost the settings for the rest of it.  Regardless, it’s Friday and count the stars in the sky.