Developing Your Action Plan

Testing should not be an off-the-cuff activity: you should have a methodical plan to approach it. Ironically, the author presents a lot of advice in a very un-methodical way - grouped by topics, not in any particular order.

Begin by understanding the business objectives and understand how the Web site supports them. There are many things you can "improve," but unless this improvement helps the business achieve its goals, it's not a worthwhile investment of time and effort.

In particular, understand the financial impact. If you're going to ask for budget dollars, you'll find them easier to come by if you can promise revenue dollars as a result. Btu again, be conservative in your promises.

Once you have a notion of what you want to do, determine the individuals who need to be involved - from whom you will need permission, who will need to contribute their efforts, or whose demesne of responsibility will be impacted - and get their buy-in.

In addition to determining what to change (what pages), consider whom to change it for (traffic sources, audience demographics).

Determine what constitutes "success" in specific terms that can be objectively measured.

It may be more effective to begin by identifying a situation as a "problem" (a drop-off rate on a specific page) and addressing that, rather than testing to implement a solution to a problem that doesn't exist.

Make your test self contained - it should be temporary, with the understanding that it will end on a specific date - and all test code is backed out on that date. If the test is successful, there should be a separate plan to implement a permanent solution.

Of primary importance: a test should collect as much data as possible, even data that you may not expect is important, and the data should be analyzed after the test has concluded.