There's no replacement for a good research plan

Yes, you should test your Web properties, especially the transactional ones (e-commerce, subscriptions, digital libraries); we routinely recommend usability testing combined with other techniques such as multivariate testing or A/B testing. Anchor Wave's Anthony Rivera (@ant1832) asked us about The Ultimate Comparison of Multivariate Testing, originally tweeted by always-on-top-of-it Smashing Mag - which solution do we recommend? Good question, since startup costs range from free (Google Website Optimizer) to $33,000+ (Accenture). Where to begin?

Here's the short answer: We recommend selecting the best fit for your company. (You need to decide whether you need both multivariate and A/B testing; check out Avinash Kaushik's simple explanation of the difference. Or check out the handy glossary from Web Analytics Blog.) Essentially, multivariate testing swaps out components of a Web page, while A/B testing swaps out two (or more) versions of a Web page. Both are valuable and merit consideration.

Basic criteria to consider when choosing a solution:
- Scale: How extensive is your digital product?
- Flexibility: Can you modify your test targets?
- Self-service: Do you need or want hand-holding?
- Return on investment: If your digital product is a major source of revenue, shouldn't you invest in testing it with something more than a free solution?

There are two basic fallacies to avoid in considering multivariate testing or A/B testing:

The first fallacy is the idea that testing one page is enough. User experience consists of multiple steps (or screen views) through a Web site, Web application, or kiosk. Whether you select multivariate or A/B testing, the tool you choose must account for task sequences - the steps that users take through your digital product.

This is not to say that one page can't make a huge difference. Our usability testing shows that common, simple missteps - such as an interface change at a critical transaction point - will discourage users from completing a purchase. But the larger point is that measurement must be designed to span the entire process - especially because conventional metrics may fall off the map when the user proceeds to a different URL.

The second fallacy is the idea that ANY product replaces a considered research plan. In our practice, we rarely encounter well-planned research, where the organization routinely measures success against defined goals. (A notable exception: Playboy, where measurement is a regular activity conducted by research professionals.) As with usability testing results, well-planned research is most valuable when shared within the organization. The purpose of conducting research, after all, is to inform your business activities.

If you have not established a research plan - even at a back-of-the-napkin level, with definition of user profiles at a minimum - it's probably too soon to engage in multivariate or A/B testing. Because you really don't know what you need yet.

Are you an enterprise, a medium-sized business, or a small business? Your research initiatives should map to the size of the business. The enterprise MUST invest in professional research (and if it does not, it will eventually fail or under-perform, which amounts to the same thing). The small business can take advantage of free or almost-free tools. The medium-sized business can find solutions somewhere in the middle. Regardless of budget, a basic research plan must be in place before solutions are selected. It's a basic part of your Web strategy.

So, how were your e-sales this year? E-com usability

So, how were your e-sales this year? E-com usability may be easier to discuss in recent hindsight, after the dust has settled and the wishful thinking has stopped. How well did you do? What sort of ROI (return on investment) are you seeing?

E-commerce success can be measured in many ways; gross sales numbers are just a start. A more critical number in our view: Abandonment rates. How many people abandoned a shopping cart? Do you know exactly when and where it was abandoned? Most importantly - Do you know why someone abandoned a transaction?

The UK firm userfocus recently discussed "Nine tests for a usable checkout" which certainly merit a look. It's great to see another firm reiterate what we've been observing - and telling our clients - for years. "Ask the minimum number of questions. Use consistent and standard form controls. Place error messages next to the entries that need correcting." Truly commonsensical advice.

While the Nine Tests checklist is a great contribution, we still need to understand our users on a more specific level. The Amazon one-click book shopper is not the Playboy online club member, who is not the engineer joining IEEE online. Mental models, and the reasons for our purchases, play a role. The Amazon book shopper has learned to trust one-click ordering and needs little else. The Playboy user may like the added thrill of the chase that results from "teaser" screens. The engineer at IEEE will grumble if date entry fields don't enforce formats (he's an engineer!).

User experience is about seeing the world through the user's eyes, even if briefly. The personal interaction that comes from user involvement - be it as advisor to the design team, or simply as usability test subject - is irreplaceable. Feel-good aside, sincere understanding of - and concern for - your user is the key to successful business online.

You can and should know how many transactions were abandoned; that's tactical. Understanding that user and designing to his needs is strategic.

Coda: I was about to join an association I really admire last week. They lost me at unenforced date format for my date of birth. Why did they lose me? Because I know they're smarter than that.