A few days ago, Ben wrote a post that did a nice job talking about the basics of A/B testing. Today’s post was supposed to build on that by doing a survey of the available A/B testing tools. But a funny thing happened while I was working… I got riled up.

Do you ever wake up in the morning and say to yourself, “Today is a great day to do A/B testing” or “Man, some multivariate split user testing would be great right about now.”? Unless you are a serious geek or UX professional, chances are pretty good that the thought has never crossed your mind. If you’re my clients, you wake up and say, “I need more sales!”.

That’s the kind of thought that keeps you up at night.

“I wish I had more money.”

As a web developer for several small clients, everything they do is attached to that idea. In their lives, their business budget fights with their personal budget. The money that goes to pay for user testing tools is coming out of food money. So when I recommend something, it needs to make sense to my clients. They have to see the value or they’re not going to buy in.

When I visit an A/B testing tool’s website for a client, I want to be able to send it to them and for it to make some intuitive sense.

Sure, I can explain the benefits of A/B testing to him and how savvy use of it can lead to more revenue. But it’s nice to have a little help.

[highlight color=”eg. yellow, black”]You need to address both me (the tech-guy) and my client (the money guy) right up front.[/highlight]

Let’s look at a few and see how they stack up to a non-techie small businessman.

Google Optimizer

Google does a pretty decent job of explaining the benefit in the headline. But I have one problem. It says “conversions”. That’s great for those in-the-know but for most of my clients, they’d relate much better to the use of “sales/leads”. I also like the use of the three icons. The whole thing is not-sexy in that Google way, but on the other hand, the name Google carries a lot of weight. Google’s like Oprah, famous enough to get away with not having to be sexy. (Sorry Oprah.)

Optimizely

Optimizely proudly claims in its headline “A/B Testing you’ll actually use!” and I have to say, they’re right! The problem is, they’re solving a problem for me that I haven’t yet had. I’ve not had an experience where I got an A/B testing tool and then ignored the crap out of it. Maybe that’s a common thing that I don’t know about but to me, it’s not a problem.

Still, I get their point. Their system is easy to use. Great.

What about the money? Will it make my client more of that? Because that’s what they want.

I appreciate that Optimizely is talking to me. And I like the idea that they’re going to make my life easy. But I also need them to make my sales pitch on their behalf easy too.

Split Testing Pro

Right off the bat Split Testing Pro, you’re talking my language. A split second later, your automatically-playing audio file starts talking too. I could punch you in the face.

Did I mention I’m riled up?

First of all, quit that nonsense. It’s hard to take you seriously when you make me start off by trying to find the ‘stop’ button. And now that I mention it, it’s hard to take you seriously on many fronts. I love the sales pitch. But the whole thing just smells like an infomercial. It makes me think you’re cheap, or a rip off, or are somehow inauthentic.

Of the three we’ve looked at so far, you’re the one that’s talking my client’s language but you’re also the one I think they’d walk away from first. It’s too much too fast. Buy my man a drink first.

Visual Website Optimizer

I almost love you. You’re talking to me with the “easiest A/B testing tool” and you’re talking to my clients with the “20% increase in sales”. But just two things. I’d be more explicit about how A/B testing has a positive effect; perhaps with a well worded sub-title. Alternately, you could also box in the 4 success examples. The spacing is such that it’s a little odd the first time you look at it. But otherwise, well done!

I mean, I have no idea if you’re a good tool or not, but you’d definitely catch my client’s eye.

KISSmetrics

It’s a little unfair to put KISSmetrics in this group. A/B testing evidently isn’t really their main deal. They don’t even talk about it “above the fold” on their website.

Once you scroll, you get this lovely paragraph:

I appreciate you talking about a healthier bottom-line but I’m curious if you couldn’t do a little more of that right off the bat. Again, you’re an analytics program so you probably track A/B events the same way that Google Analytics does – which is to say, it’s not a priority in your software (I don’t know because I haven’t used your program yet). But I’d still like to see some reference to revenue or the bottom line or more money a little more prominent on the page.

Conclusion

For the most part, A/B testing websites are talking to people they feel are already informed about their product. I wouldn’t say that’s a safe assumption. Techie guys like me are still going to share your site with my non-techie clients. For the most part, every site listed could improve how they communicate to bean-counters and small businessmen.

Why don’t you A/B test that idea and get back to me. I’d love to know the results. 🙂