9: Analyzing Site Performance
The intent of the present chapter is to consider the nuts-and-bolts details of analyzing data collected from a Web site.
A distinction is drawn between analysis and reporting: analysis is the act of examining the numbers to draw conclusions, and reporting the task of making those conclusions meaningful to others. The distinction is clear when raw analytics are presented in lieu of a report - the people who get the report cannot make sense of the numbers, or draw their own conclusions based on their reaction to raw information.
The author also briefly mentions the danger of "rear-view mirror syndrome" - as analytics pertain only to the past, and focusing on raw analytics gives a firm a good idea of where it has been, but no clue as to where it is going. (Driving down the street with your gaze locked in the rear-view mirror is not a wise practice).
Don't Blame Your Tools
Many companies seek to purchase a plug-and-play analytics solution and find that it does them no good at all. And naturally, the reaction is to state that the tool is no good, and we need to buy a better one (which also turns out to deliver nothing). The author cites personal experience in consulting with firms who were looking to find a better tool, only to show them that they are overlooking the meaningful information (often, exactly what they feel they are not getting) in the output of their existing one.
Largely, it goes back to the fundamental problems of not approaching analytics with a clear sense of what information you seek to glean from it, and the data fog of so much information being available. There's also the matter of configuration and knowing how to use the tool for your specific audience and purposes.
That's not to say that all analytics tools are good - there are some lousy ones out there - but even the best tool yields bad results if it is not used property and effectively.
Examples of Analysis
The author asserts that it's difficult to discuss analysis conceptually, and indicates he will use "real-world examples" to illustrate the notion of effective analysis.
One example of analysis is the task funnel - measuring the number of people who enter a task flow, the number of people who complete it, and the number that drops out at each step along the way. It's also important to consider what happens to the drop-outs (whether they exit the task and go elsewhere on your site or leave the site altogether) and follow-on behavior (do they come back later and complete the task, or go to a different channel). Especially in e-commerce sites where multiple items can be ordered, it's also possible the user has "bailed" on the ordering funnel only to add another item to their cart and finish the order afterward.
Analytics can also be foiled by their assumption of a linear process: assuming the user goes from the home page, to an article, to the comment form, to the newsletter sign-up with no variations. People don't navigate in a linear fashion: they may enter on a "deeper" page, click about through various sections on the site, jump into a flow and out of it, tap the back button, open a second window and look at something else, etc. If you assume they march in an orderly fashion along the shortest route to where you want them to be, you are mistaken. (EN: and if you attempt to march them down that path, you will find they seek ways to break out, or bail completely from your site.) Your analytical tools must consider these variations to produce accurate results.
Time interval is also significant to analytics, especially when assessing customer engagement. Not all visitors to a site will purchase immediately, and some will visit a site a number of times before making a purchase, and even regular customers will drop in to browse sometimes. In terms of engagement, these no-purchasing visits are significant as a "phase" of customer engagement - and if analytics considers weekly behavior, or even monthly behavior, it may not reflect increasing customer engagement, which is itself a goal and a sign of success, or at least progress toward success.
Connectedness between activities can also be missed. If a purchase is considered in isolation, you will miss important details - such as the item that is "purchased" but the order is later cancelled, or the item is later returned. Or the act that a person who buys a given item is more or less likely to return to the store in future than those that did not buy that item.
Likewise, details of the audience can be lost. If your site experiences an increase in traffic, it may be because of advertising, or a link on a different site, or another reason. The new visitors will behave differently than your "normal" traffic, and their behavior should be segregated to avoid getting a false impression of a widespread trend or change.
Analyzing on-site search is an area all its own. The raw count of search terms entered is the tip of the iceberg, which can only be appreciated if you correlate their searches to more meaningful behavior: searching may be evidence that users are getting lost on pages, or that a page they visited did not give them the information they needed. A person who conducts a search may be more or less likely to undertake another action (such as purchasing an item after searching for it). The results of the search may be helpful, or people may leave if the search doesn't tell them what they need.
There are likewise a lot of specialized analyses of home page traffic - including the number of site visitors who enter the site at points other than the intended home page. Navigation paths from the home page can be analyzed to determine whether a given sequence of pages is more or less likely to result in a purchase.
(EN: This goes on for quite a while - but it's more of the same. General suggestions of the kinds of things that could be measured based on the nature and content of the site, and which can be missed by analytical tools that are not configured to look for the "right" things.)
Segmenting Traffic to Identify Behavioral Differences
It is helpful to be able to classify site visitors to consider their behavior in light of who they are - as treating every site visitor as part of the same, homogenous mass will lead to bad analysis and bad decisions. It's often accepted that the Web is an anonymous medium, that the site visitor is a complete unknown and it's impossible to segment - but this is not so.
(EN: Also, it's often suggested that segmentation is a "new" capability of the Web, but it's actually an old practice in sales and retailing. A good salesman can size up a prospect quickly, to assess whether he's more or less likely to buy, and to decide what approach would be most effective in interacting with the customer.)
Some of the basic segmentations of Web site traffic can include:
- First-time visitor versus returning visitor
- Categories of "regular" visitors (who visit X times per time period)
- New customer (never purchased) versus returning customer
- Visitors who were referred from a specific source (site, ad, campaign)
- Visitors who chose a specific action (separate job-seekers from customers)
Segmentation can be arbitrary, but it should be driven by the site goals and the organization's goals in terms of the relationships it has and seeks to foster with its audiences and those who fall into defined phases/stages of engagement with the firm.
There is no standard solution for doing this, as it depends on the organization, but the author provides a "case study" (EN: Which, as usual, illustrates the concepts but adds no new insight.)
Analyzing Drivers to Offline Conversion
Much of analytics focus on e-commerce sites, but there are a number of sites that are not commerce driven, and as such cannot use online sales figures to monetize the value of analytics. Such sites may still be "commercial," but the purchase transaction does not happen on the site itself - the customer may be handed off to another site or another channel to close.
Tracking handoffs in the online channel is fairly straightforward: you can track the behavior on your own site, up to and including the hand-off, and as the visitor is handed off, provide a token that enables the receiving site to identify them so their behavior or that site can be reported back to you. This should enable you to determine whether the visitors who are handed off complete the sale, so that you can assess what actions can be taken on your own site to improve the quality and quantity of potential buyers who are handed off.
(EN: This largely depends on the receiving site's ability to track this information, and their willingness to report it back to you. It's worth noting that many firms do not have their own houses in order - even those that you would think are large and experienced in the channel continue to have extremely primitive capabilities in terms of Web analytics. The only approach I can think of is to ensure that reporting requirements are well documented in any written agreement, to review those requirements in detail with technical staff of the other firm to ensure they understand, to ensure the agreement enables you to audit their tracking, and to hold their feet to the fire when necessary.)
Tracking visitors from the online channel to other channels is more difficult, as there is not a reliable way of passing a "token" to ensure that their behavior in different channels can be aggregated.
The notion of providing a specific phone number of a Web site to track calls that are referred from the Web channel has previously been suggested. While it's not entirely foolproof, it does enable some connections to be made and should not be discounted because it isn't perfect. The author attests that clients who have done this are often surprised at the number of calls that originated from the Web site, and are better able to tune their site to reduce call volume (by addressing more issues online), reduce average call length (by making the caller better prepared), or to increase the conversion rate from referred callers (by better screening via the online channel).
Tracking handoffs to the brick-and-mortar channel, including physical stores or live salesmen, is more difficult. In the retail model, promotional codes can be useful in tracking sales to the source. In the sales model, using an online form to gather information and request a sales contact helps to make the connection (and again, a web-specific phone number for the sales office can do the same).
The author also speaks from personal experience with a firm that sold real estate, and used the company Web site to provide more information to leads, and gather more information from them, as a means to improve conversion rates - in effect, screening customers to ensure that the (costly) time of sales agents was spent with customers who were likely to close, and to provide a level of information that expedited the time to close.
In sum, even when it seems like it is impossible to track activity from the Web site to other channels, it's "worth the trouble to find a way."
Delayed Conversion
Another phenomenon that is difficult to assess, but important to consider, is delayed conversion. In an e-commerce situation, many analytical tools are restricted (by capabilities or the way they are used) to the duration of a single session. If the web site visitor doesn't make a purchase during that visit, this reflects poorly on the conversion rate of the site - even if the customer makes a purchase on a subsequent visit.
Certain types of transaction may require a long period of engagement before a purchase is made. The example of mortgages is given: a customer generally does a lot of research and product comparison before applying for a mortgage (and even then, they may get a better offer, or decide not to buy - the company makes no money until they close their real-estate purchase and start making payments).
And while these activities don't result in an immediate sale, they have a strong correlation to whether a sale is eventually made. Returning to the example of mortgages, if a bank has no information at all on line, it cannot be considered by customers who use the Internet to research options. If the information is online and easy to find, it will likely be considered by more customers, and likely sell more loans as a result.
With this in mind, Web analytics is important, even if there are no online sales, to track the relationship with prospects, from the moment of first contact to the moment a sale is made (and then, to their ongoing transactions and interactions as customers).