Recently, I read two posts that helped me think about the entire UX process, and specifically why many user testing instruments yield lukewarm results in the hands of the beginning.

Let me start by introducing the posts and the the authors.

Mike Hughes, user experience architect at IBM in Atlanta, wrote a post about the ‘think aloud’ protocol, which is when a test participant verbalizes their thought process as they do tasks on a website.  He points out a major problem with the technique and offers two ways he uses to avoid them.  I’ll go into this more later, but first I want to introduce you to Abby.

Abby Covert (aka Abby, the IA, in NYC) wrote two posts on ‘Heuristics’, which are defined as a learning or problem solving tool.  Simply put, a heuristic is a set of ‘rules of thumb’ for a certain topic.  For example, CRAP is a heuristic for visual design.  It helps designers remember and discuss 4 general principles for putting things on a page.  Abby, a gifted and clever UX designer, updated several existing UX heuristics in her post.

 Mike and Think Aloud

Mike’s ‘think aloud’ article highlighted something that we have noticed and struggled with. It seems to be a common problem. In user tests, when participants speak out loud, they offer thoughts about the design rather than their experience. Hughes advocates two techniques to get better results: Instructive Practice, and “Reinforcement and Extinction”.  Instructive Practice is training the participant in performing the think aloud task.  Reinforcement and Extinction is subtlety guiding the participant during the task itself.

My take away from his post is that people don’t understand the talk aloud protocol enough to spot this problem and avoid it.  It seems that they ask the wrong questions or accept poor responses. The reason for this COULD be that they don’t know what they are looking for.  It could be they aren’t asking for specific enough questions that are not relevant to their design process.  It could be they are not SMART.

Speaking of smart, meet…

 Abby the IA and Heuristics

Abby’s post is a presentation of her IA Heuristics .  She basically looked around at the existing user experience heuristics ,synthesized them and updated them to fit her experience and our time.

Here are her 10 principles (based loosely on those by Jacob Nielson, Lou Rosefeld and Peter Morville):

1. Findability
2. Accessibility
3. Clear
4. Communicative
5. Useful
6. Credibility
7. Controllable
8. Valuable
9. Learnable
10. Delightful

It’s out of the scope of this post to thoroughly review the principles, but if you are interested in the details, go to one of Abby’s speaking engagements or view the  presentation slides here –   Additionally, she explains how to use the Heuristic principles of UX (and sells a laminated poster) at this blog post : IA Heuristics Journey

My take away from these posts are that Heuristics align the whole process.  They provide structure to each of the activities, people and documents in the design process.  They are an aid to problem solving and learning.  Learning is what you are doing when you do a ‘think aloud’ protocol test.  So, now you know what to ask and know how to evaluate the responses.

[By way of closing, I’m going to interview myself about this post.  Perhaps this is how I imagine what the comments to be.  I don’t know, it seemed like the thing to do.  I’ve never seen it done before, so at least I’m innovating. – Newman]

Question and Answer Time:

Why did you share this?
Because both posts have been on my mind and I feel they offer valuable insight into improving the design process and making better websites.

Are they connected?  Why mention them together here?
I do feel they are connected. Both the ‘think aloud’ and the heuristics are, in a way, about framing a conversation in order to produce results.  When we design something – and Ben and I have spilled much ink about this – we are trying to have the right conversation and ensure productive communication between all the stakeholders of a design team.  In ‘think aloud’, you are trying to have a productive interaction with a user.  Heuristics can provide an agreed upon guide for design conversations.  This list can serve as the ‘codec’ – or the backstory – of the design deliverable. It’s purposes is to guide and record all of the design assumptions and decisions.

Where are you going with this?
I feel that heuristics can align the entire process. At the research and design phase – generally toward the beginning –  it helps you to remember all the possible facets of a design and an experience.  At the testing and evaluation phase – it’s a way to list and remember all the things to test and provides reference for conversations about the design.

And what about Mike’s ‘instructive practice’?
It can be used teach people to reference a design using the principles layout in the heuristics.  hmmmm…

Are there any dangers here?
I do think that – similar to personas – that raw design heuristics should not be put in front of the user test participant in a ‘think aloud’ session.  The tasks should be designed to highlight the principles or test our assumptions about the principles.  But, we should not ask the user something like “Is this learnable?” or “Is this clear?”

You wouldn’t ask “Is this clear?”
Well, that one is okay.  I do think we run into danger when we ask our user’s to be designers. And our designers to be (unbiased) users!  I guess the point about Mike’s ‘instructive practice’ is that users evaluate things in different ways – that’s good.  We need to find HOW they evaluate, not the results of the evaluation.  The ‘how’ is their experience. The result is their opinion.

In the interest of brevity, let’s wrap up.  Any final thoughts?
Yeah, Heuristics keeps your design ‘tight’ and ‘Instructive practice’ during think aloud keeps your testing ‘tight’.  That’s how they fit together – Alignment.  A design heuristic can manifest itself in all phases of a project, mike’s  tips can help maintain that alignment during a think aloud session.