There's no replacement for a good research plan

Yes, you should test your Web properties, especially the transactional ones (e-commerce, subscriptions, digital libraries); we routinely recommend usability testing combined with other techniques such as multivariate testing or A/B testing. Anchor Wave's Anthony Rivera (@ant1832) asked us about The Ultimate Comparison of Multivariate Testing, originally tweeted by always-on-top-of-it Smashing Mag - which solution do we recommend? Good question, since startup costs range from free (Google Website Optimizer) to $33,000+ (Accenture). Where to begin?

Here's the short answer: We recommend selecting the best fit for your company. (You need to decide whether you need both multivariate and A/B testing; check out Avinash Kaushik's simple explanation of the difference. Or check out the handy glossary from Web Analytics Blog.) Essentially, multivariate testing swaps out components of a Web page, while A/B testing swaps out two (or more) versions of a Web page. Both are valuable and merit consideration.

Basic criteria to consider when choosing a solution:
- Scale: How extensive is your digital product?
- Flexibility: Can you modify your test targets?
- Self-service: Do you need or want hand-holding?
- Return on investment: If your digital product is a major source of revenue, shouldn't you invest in testing it with something more than a free solution?

There are two basic fallacies to avoid in considering multivariate testing or A/B testing:

The first fallacy is the idea that testing one page is enough. User experience consists of multiple steps (or screen views) through a Web site, Web application, or kiosk. Whether you select multivariate or A/B testing, the tool you choose must account for task sequences - the steps that users take through your digital product.

This is not to say that one page can't make a huge difference. Our usability testing shows that common, simple missteps - such as an interface change at a critical transaction point - will discourage users from completing a purchase. But the larger point is that measurement must be designed to span the entire process - especially because conventional metrics may fall off the map when the user proceeds to a different URL.

The second fallacy is the idea that ANY product replaces a considered research plan. In our practice, we rarely encounter well-planned research, where the organization routinely measures success against defined goals. (A notable exception: Playboy, where measurement is a regular activity conducted by research professionals.) As with usability testing results, well-planned research is most valuable when shared within the organization. The purpose of conducting research, after all, is to inform your business activities.

If you have not established a research plan - even at a back-of-the-napkin level, with definition of user profiles at a minimum - it's probably too soon to engage in multivariate or A/B testing. Because you really don't know what you need yet.

Are you an enterprise, a medium-sized business, or a small business? Your research initiatives should map to the size of the business. The enterprise MUST invest in professional research (and if it does not, it will eventually fail or under-perform, which amounts to the same thing). The small business can take advantage of free or almost-free tools. The medium-sized business can find solutions somewhere in the middle. Regardless of budget, a basic research plan must be in place before solutions are selected. It's a basic part of your Web strategy.

Who exactly is taking my money? Poor usability = reduced online sales

It's ironic that online check-out - the point in ecommerce at which users are most sensitive - is the point where many Web sites fail to provide the brand consistency that users need to feel confident about entering personal information and credit card numbers.

Abrupt interface changes (from the main Web site to the ecommerce application) are the most common - and most destructive - usability problems on ecommerce sites. A startling new interface with different branding, combined with a new URL, will give pause to any user, regardless of technical literacy. They wonder: Am I on a different site? Is the privacy policy different? Who exactly is taking my money?

When someone decides to purchase from your Web site it's because they trust you and your brand. Our own research and usability testing has shown scores of users halting and not completing transactions when they encounter an unfamiliar screen at the point of interaction.

What caused this problem? Initially, many Web sites did not have ecommerce functionality, so it was bolted on as an afterthought. Third-party applications were used as the ecom stopgap. Brands were not allowed to customize these interfaces. The result: A bumpy ride for the user, reduced sales for the company. See examples below of the inconsis

The solution: Bring ecom under your primary Web property so users trust you - leading to increased online sales. Plan for ecommerce within your information architecture.

Example of a third-party payment system

Clicking on conEdison's "Pay by credit card" option takes you to a third-party company that handles the payment. The transition is jarring for several reasons:

• New URL doesn't mention conEdison
• NCO logo replaces conEdison's in the upper left
• New color scheme
  
• New privacy statement.
  

The payment agency does responsibly provide a link to their own privacy policy and disclaimer that they are a third party that handles payments. Yet, other than the inserted conEdison logo, it's difficult to tell that this page relates at all to conEd's parent site.

Ecommerce is alive, usability problems still rampant

Clearly ecommerce is far from dead - it leads growth in retail even during this global economic downtown. Yet usability problems, which negatively affect sales, are rampant. Why has this problem remained with us since ecommerce first began?

In August of 2001, the dot-crash was roaring, about to be amplified by the events of 9/11. Early obits for the Web were popular. So it was that usability eminence Jakob Nielson asked, "Did poor usability kill ecommerce?" At the time, Nielsen conducted usability testing of 496 users on 20 US-based sites; failure rates were 56%. Nielsen also estimated that 79% of the sites could increase sales by improving usability. What has changed? Well, ecommerce is no longer an experiment. From Internet Retailer: "Sales of the Top 500 online retailers grew 11.7% to $115.85 billion in 2008 from $103.69 billion in 2007 while Internet Retailer estimates the total retail sales market grew 1.4%."

So ecommerce usability problems have gone away, right? Wrong. In many of our usability tests (2000 - 2009), we've observed all kinds of users attempting to purchase online, and we've watched them abandon transactions for the same old reasons:

• Clunky or incomprehensible checkout process
• Brand inconsistency between the parent site and the ecommerce application
• No evident privacy policy
• Confusion returning to shopping after placing a purchase in the shopping cart
• Failure of shopping carts to retain user choices
• Asking users to complete extensive forms prior to checkout

These are user task path problems, with a dose of emotional intelligence in design sorely missing; look at ecommerce interfaces for a myriad of more straightforward (and unacceptable) UI problems that undermine users and impact sales.

There is no replacement for information architecture, user interface design, and user-centric task sequence design as processes required for usable ecommerce platforms. Companies that took the time to stabilize their ecom platform before adding a plethora of products are the winners.

If your business model relies on online sales, plan to increase those sales by conducting usability testing and investing in solid information design. Time to stop guessing.

Do bad things happen when developers design user interface?

We work with some incredibly talented development teams around Web sites, Web applications, and software projects. Unfortunately, teams missing a user experience specialist often find that they cannot come to consensus on what the user interface should look like, or what it should do. This leads to a sub-optimal user experience, with the common result being that powerful software functions are invisible to the user. That wastes money - and obscures the excellent work of your team.

The cost to business of skipping the critical planning stages - which include user profiles, usability testing, information architecture, user interface design and task sequence design - is incalculable. Skipping these steps means endless hours of expensive meetings where everyone has an opinion, but few have real data.

Stop for a moment and consider the areas of expertise your employees own today. Is there someone on staff with formal training in user interface design? Information architecture? Task sequence design? If the answer is "no" or "sort of," it's time to start adding up the cost of all the hours spent in meetings with no consensus, so you can reallocate those funds to usability training or professional services that will bridge the gap.

CDC test of social media: Complexity makes usable design a challenge

Check out the concise summary of "social media usability testing" conducted by the CDC. Covering MySpace, blogs, widgets, e-Cards, and podcasts, the study points to the unsurprising conclusion that complex products - such as widgets - must meet a higher standard of usability and credible user experience.

I'll bracket that statement with two statistics from the study. First the e-Cards: "100% participants worry about viruses/spam." Which, of course, influences the decision to open the e-Card. This is what our Usable Times 5 methodology would describe as a Permission issue - as in, "I did not give you permission to send viruses or spam to my computer." Of course the Web site does not explicitly say it will send viruses or spam. But it doesn't reassure users that they won't have to worry about them, either.

Second, the widget, which only 45% if respondents were able to share and download, caused users to "struggle." Again, a reduction in usage, for different reasons. The loss here is in the development time for that widget, which is then under-utilized due to poor task sequence design. The presentation of user interactivity required for interaction with the widget is not straightforward enough for this user base. Interactivity is another Usable Times 5 factor.

The usability test report also discusses problems with adjacency of controls (instructions for use of podcasts too far from the action button, counter-intuitive "save this file" labeling). This is a common problem easily revealed in usability testing, and solved in user interface design.

The best news: "Visitors who use our new media products are more satisfied
with their visit to CDC’s Web site." That encourages all of us to continue the development of the many diverse expressions that currently fall under the rubric of social media. But these findings also point out the significant gap between the products we create and the user's ability to understand them without us there to explain them.

The need for information design on all digital products remains unmet.

The study, created by Sarah Greer, Catherine Jamal, Mike Peltz
and Cari Wolfson of the CDC's Division of eHealth Marketing, gets kudos for its frank look at CDC's current usage of media tools. No surprise to find FDA's Sanjay Koyani thanked at the end - he was among the first to create Web user experience standards in 2003 with the Research-Based Web Design and Usability Guidelines.