Why does it matter?
The analysis above wasn't especially difficult, but for some it may be painful enough that they just don't bother. Let's see why it's important to do the analysis.
Suppose that for some block of time we have 1,000,000 page impressions, and that the sample proportions above are an accurate reflection of the real population proportions. (So in other words, in the long run, 4.4% of the blue button impressions on the home page will result in a click, etc.) Also, though we didn't address it above, let's assume that people visit the home page twice as often as they click on the programs page. So 667K impressions will be for our home page and 333K impressions will be for the programs page. Finally, let's say that each CTA click has a value of $5 to the company.
(I'm just making all these numbers up to show how to do a first pass at the analysis; obviously you'll need to do the research that allows you to plug in your own numbers.)
Based on these assumptions, let's examine the effect of different approaches on button colors:
Approach 1: Always use blue buttons
((667K home page impressions * 4.4% CTR) + (333K program page impressions * 5.6% CTR)) * $5 = $239,980
Approach 2: Always use red buttons
((667K home page impressions * 3.2% CTR) + (333K program page impressions * 10.8% CTR)) * $5 = $286,540
Approach 3: Use blue buttons on the home page and red buttons on the program page
((667K home page impressions * 4.4% CTR) + (333K program page impressions * 10.8% CTR)) * $5 = $326,560
The difference between approach 3 (the optimized approach) and approach 1 (the approach that our initial naïve analysis would have suggested) in particular is striking. Approach 3 represents a 36.1% improvement in revenues over approach 1. Not too shabby for an analysis based on middle school mathematics.
Analyses like this are clearly very useful to increasing revenues. They're also useful in providing objective data to design processes that are usually subjective in nature. (For example, the branding committee or maybe the web team's star graphic designer may simply like blue buttons better.) That doesn't mean that the quantitative analysis always wins out—generating revenues may not be an important priority (though that's probably rare in business settings), or it may be that people think that adopting a specific set of brand/design guidelines will have a better overall long-term effect on revenues. But it helps to have the analysis so you can make a conscious decision rather than a haphazard one.
