2 - What Is Conversion Optimization?
A significant amount of emphasis is placed on improving conversation rates - and that's all good and well because it has an immediate and significant impact on short-term revenues. But the immediate conversion-rate lift is a starting point to more long-term strategic changes within the organization.
Conversion Optimization Requires Controlled Testing
There's a bit of unnecessary campaigning for the scientific method - developing a hypothesis and testing it against a control group on the assumption that only the factor that was changed is responsible for any difference in outcomes. He paints a picture of marketers who resist the scientific method as silly and old-fashioned
(EN: "science" has earned itself a poor reputation in the past several decades, particularly in psychology and the social sciences in which poorly designed or even intentionally fraudulent experiments have led to specious conclusions and highly expensive disasters in the marketing field in particular. The most idiotic schemes have been fobbed off under the name of science, and it's likely that proponents of science would do well to start with an apology for past misdeeds rather than an expectation of instantaneous credibility.)
The use of a control group is essential to the scientific method - to make a change and see what happens before and after fails to account for other events that may have occurred during the same time. As such, one group must be shown the test version of a page at the same time another group is shown the control version (which doesn't include the changes made to the test version) if it can be said that the change in outcome is due to the change in the page rather than any other factor.
This is not to say that only one test may be run at a time. For example, the audience can be split into fourths, with one group seeing the control version and the other three each seeing a different test version. The version that achieves the highest conversion rate then becomes the new control for the next round of testing.
The author mentions the issue of sampling, as test results can be skewed if the audience is split in such a way that their characteristics skew the outcome. For example, if the test version is shown to teenagers and the control to elderly visitors, then the outcome is as likely to be a reflection of the differences in the behaviors of those two different groups rather than due to the different versions.
Most testing involves a random sample - which does not mean that the results are not skewed, only that no attempt was made to purposefully skew them. It might just so happen that every person who was randomly selected happened to be a teenager. It would be highly unlikely, but not impossible, something of that sort might happen. If even the potential for happenstance to skew results is to be eliminated, then marketers often construct a structured sample to ensure that an equal proportion of people of the same characteristics (age, gender, ethnicity, etc.) are in the audience for each test version and control. However, given the anonymous nature of the Internet, this is seldom possible.
There's a bit of nuts-and-bolts details about testing: splitting incoming traffic between variations, using a cookie to ensure that visitors see the same variation if they leave and revisit, tracking page-to-page conversions, ensuring you set a high enough level of statistical significance, etc.
It's also noted that unless you do testing, you are merely imitating practices on the assumptions that they will apply to your own site and audience, and that many of the maxims that are based on research will not always bear out: any suggestion that something always or never works is a generalization that may not hold true for you, even if it is supported by valid research elsewhere.
Said another way, even if a design suggestion sounds reasonable and comes from a respected source, you should always test it. Sometimes your test will bear out that the advice was sound, other times you will be quite surprised.
The author returns to the "before and after" method of testing, where a change is made for the entire audience and the numbers are watched to see if they go down or up. The problem, as noted previously, is that other events may be occurring at the same time that will influence the results (your own marketing, your competitors, stories in the news, a change in the weather, etc.). Another problem is that it subjects 100% of your audience to a potentially disastrous choice rather than testing it on a small group before risking its effect on a larger population.
The author then sets his sights on usability testing in the lab, which is fraught with problems - small sample size, people knowing they are being observed, a "make-believe" transaction with no real consequences, and the like. It's generally good for formulating hypotheses but not a substitute for optimization testing.
(EN: It's worth noting that usability testing is much abused and misinterpreted in this way. The sole outcome of a usability test is an answer to the question "what difficulties to users encounter when attempting to do this task?" I'm concerned that the author is dismissing usability testing as worthless because it do not do things it was never intended to do - and ignoring that they are very good at doing what they were actually designed to do.)
The author goes on to speak to the limitations of other marketing research tools such as customer surveys, traffic analysis, and Web page heat maps. Each of these yield interesting information that is loosely connected to customer behavior, but is not designed to gather the same kinds of information that optimization tests do.
Who Are Your Target Audiences?
The author provides a superficial overview of target market descriptors and the personas that are created to give marketers a sense of who their customers are. However, this leads to a problem of stereotyping - the persona is essentially a fiction character, and the way that it is used more often pertains to stereotypes than actual human behavior (the way you assume a fifty-year old wealthy white male would act is not actually the way that he does act).
More to the point, personas are used to guide the development of a new channel or product, when there is no existing audience to examine or to draw upon and the firm is making assumptions about whom its customers will eventually be, what they value, and how they would act. Admittedly, optimization testing is of no use at this point because there is no actual audience to test.
However, once a site has been built, optimization testing can take over - to observe the actual behavior of an actual audience, without suppositions.
Setting Goals
Obviously, the goal of convection is to generate more revenue for an organization, but the author suggests that there is a lot more involved in goal-setting than simply looking at sales, and that there are a wide array of activities that deliver business results.
(EN: Here, I beg to differ again. Particularly in business, it is all about profitability. The author suggests that goals such as improving brand image, getting people to "like" you on social media, and similar goals are independent. Each of these things is only important insofar as it results in increased sales or decreased expenses. Unless there is a proven correlation between an activity and profitability, then it is wasteful ritual to pursue it. He goes on for quite a while about this, but is entirely wrongheaded. As such I'm preserving nuggets of sense in the rough.)
- Conversion optimization should test goals through to a meaningful activity, which relates to revenue. You can test how many people "like" your Facebook page as a result of a campaign, but also test how many of those people eventually become customers.
- Most web site analytics have nothing to do with optimization.
- Just because something seems meaningful does not mean it should be a goal for conversion optimization.
- Be careful of optimizing your bounce rate. If you get someone to click a link and they leave immediately afterward, you have accomplished nothing.
- Broaden your horizon. A customer who does not buy immediately in the same channel is not lost. If people bounce off your Web site but place more orders by phone, it's still a "win" for your bottom line.
- Deepen your horizon. A customer who does no buy immediately in the same channel; may also return at a later time to complete his purchase, reclaiming his previously abandoned shopping cart.
Case Study: Tourism British Columbia
Tourism BC noted a direct correlation that suggested that the more content a user consumed on their Web site, the more likely they would plan a trip to the province (EN: Which smells like a cross-wiring of cause and effect, but I'll carry on.)
The site offered a brochure, which visitors were forced to register before downloading, and found that many left immediately after downloading the brochure, not interacting with the site further. It was believed that if they could maintain engagement, it would increase the probability of bookings.
In this instance, the conversion test focused on the "thank you" page for registration, with a goal of getting visitors to continue engaging with the site even after downloading the brochure. A test was done with a page that showed an "interactive map" of British Columbia against one that had a list of cities. The results after two rounds of testing was a 44% lift in visitor engagement after the brochure was downloaded.
(EN: Interesting, but what is lacking here is follow-through on the study to determine whether interacting with more site content in the same visit actually had any positive effect on travel bookings. Likely this is a situation in which it is impossible to know, because I suspect that people go to a different Web site to book travel, and the tourism board would never be aware of it.)
The Continuous Improvement Cycle
Much of human behavior follows a trial-and-error pattern in which we do something differently (intentionally or otherwise), observe the results, and decide either to carry on as before or adopt the change to our behavior. Organizations, including businesses, can evolve in the same ways, though they tend to be meticulous in planning changes and focused on specific outcomes (profitability).
It's also noted that change is not once-and-done. We make a change to achieve better results, and then seek to make additional changes to achieve even better results. In a long-term sense, this is progress. Over a shorter time period, it is called "continuous improvement." Conversion optimization takes the same approach: it is not a one-time project but an ongoing processed of seeking to achieve better results, capitalizing on success and learning from failure.
To this end, the author suggests a seven-step cycle:
- Begin with an analysis of the current flow. Web analytics, usability studies, surveys, and other observation method can identify areas for improvement
- Develop hypotheses that speculate how conversion could be improved
- Consider how those hypotheses can be tested and develop a test plan
- Leverage designers and writers to create test versions of the pages based on the hypothesis. It's important to note that this is the fourth step, not the first - a scientific test is targeted on outcomes, not random changes.
- Run an experiment in a testing tool that compares behavior of users of a test version, compared to the control (current) version
- Wait for statistical significance of the results to be achieved and stave off those who wish to jump to conclusions
- Analyze the data, confirm the hypotheses, and implement changes permanently
The author also suggests starting with the largest changes and then progressing to the smallest - much in the way that a sculptor hacks off the largest blocks of stone before removing smaller and smaller bits to create a finished sculpture.
In terms of conversion optimization, the "large chunks" are generally the flow of pages, then the common elements of page layout. Then, the individual elements within each page can be tested, beginning with the pages for which there is the most dramatic drop-out.