2 - Measuring Measurement
A nice example: is a man who stands five-foot-nine tall or short? The measurement means nothing in and of itself. In Japan, he's quite tall. In Sweden, he is quite short. And in the end, does it really matter how tall he stands - if he can't reach something he wants, he's not tall enough. And so, unless you know what your goal is, you have no way of measuring yourself against it.
Profit is what matters in business: revenues less costs. They're very simple to measure, and largely self-explanatory. You can look at ROI, ROA, and ROE - it all comes down to the same thing.
But in many instances, they're very elusive, and there's a Gestalt: if you removed all the light bulbs in a store, people wouldn't shop there - so can you claim that the light bulbs are responsible for 100% of the revenues? Measured that way, any empty room, well-lit, should generate a handsome profit. But it doesn't work that way.
The costs of technology are easy to measure: the dollars sent on hardware, software, and labor (both initial and recurring) - but the benefits derived from technology are more difficult to measure.
If you're talking about an online store, you can measure the revenue generated through online product sales - but for all the rest of the sites, which communicate information to various parties about the company and its products, the dollar-value of the benefit is harder to derive. How many people bought your product at a store because they read about it online? It cannot be measured to the penny, or even estimated with much accuracy at all.
Measuring What Matters
Some basic principles of Web metrics:
- Define a few key variables rather than a multitude of meaningless ones.
- Any measurement that cannot be linked to the needs of the business (key business drivers) are not meaningful
- Measurement must take place over time - a momentary snapshot is meaningless
- Measurements should be based on the needs of stakeholders (customers, shareholders, etc.)
- Measures should be relevant to the "top" of the organization
- A single index for all measurements provides a better overall assessment of performance than a multitude of various factors
- Measurements should account for changes in your strategy and the environment
- Goals and targets should only be based on metrics if there is a clear causal correlation between them.
Keeping Measurement In Mind
The key to measurement is knowing what you are seeking to accomplish, then using measurements to tell how well you're accomplishing it. If your goal is to enable customers to download software updates, your key metric is the number of software updates downloaded. In a fundamental sense, it's that simple.
Measurement can also impact how a site is designed. For example, if you want to measure which items in a FAQ are of the greatest interest to site visitors, then having the entire FAQ in a single page prevents you from measuring that. (EN: there's a careful balance here - you don't want measurability to get in the way of usability.)
What, Exactly, Do You Want to Know?
In the end, metrics are designed to answer questions - but you have to know what the questions are before you can answer them. Otherwise, the numbers are meaningless.