Branding through Metrics

Every month several research services report on “search market share.” Although their numbers for queries performed and percentages of queries per search service never agree in detail, Google consistently ranks first among the major search engines. The data sources these monthly reports draw upon include user click data provided by Internet Service Providers. A reasonable third-party assumption is that each service acquires its data directly and that they may not all be overlapping sources.

Compete leverages its custom toolbar data into its reporting. At the very least, therefore, we know that Compete has two distinct data sources to work with and can (hopefully) map trends against data sets.

Hitwise reveals the least amount of information in its public interface.

Nielsen/NetRatings offers more information than Hitwise, less than Compete.

comScore offers a report about comparable to Nielsen’s.

These are the four sources most relied upon in the search engine optimization industry for comment and analysis of which search engines provide the most coverage. The problem is that very little of the data actually looks at anything beyond number of queries performed on a search service. The quality of the metrics is thus extremely low and not very useful.

At least two of the services estimate number of visitors per month. Yahoo! consistently dominates the list, although Google is now a close second. The Microsoft network provides a challenge since MSN and Live are tracked separately, but if you assume that there may be overlap between the Live.com visitors and MSN visitors, you still see Microsoft placing a solid third place.

Ask receives the fewest visitors and fewest queries per month among the major services. Out of curiosity, I occasionally look at data for Lycos, which is the last of the “old school” search services that still makes a play for traffic in any capacity.

If I wait another week to discuss this topic I can give you some numbers for March 2007, but let me talk about February. Compete estimated queries at 6.5 billion. comScore put the number at 6.9 billion. Nielsen estimated 6.4 billion queries.

comScore estimated February unique visitors for the search services as: Yahoo! 128 million, Time/Warner (AOL) 117 million, Google 114 million, Microsoft 114 million, Ask 48 million, and Lycos 22 million. (As a basis for comparison, Compete’s March 2007 data shows 112 million visitors for Yahoo!, 109 million for Google, 74 million for MSN, 72 million for AOL, 62 million for Live, 26 million for Ask, and 12 million for Lycos.)

By themselves, neither number of visitors per month nor number of queries per month tell you anything really useful. We can look at the ratios and see where people seem to spend the most time searching. This type of data is not reported, so to demonstrate the model I have to illustrate with some averaged results that are not statistically valid. This is just an illustration of how I’d like to evaluate search engine performance.

I want to look at searches performed per visitor and searches performed per visit. Since MSN is now primarily concerned with providing content and it has rebranded search under Live, let me work with 51.9 million as Live’s visitor count for February.

Since there is no agreement between the reporting services, let me use approximate averages for query share: Google 55.3%, Yahoo! 23.2%, Live 9.7%, AOL 4%, and Ask 3.3%.

Still, we show about 521 million visitors for Yahoo!, MSN, Google, AOL, and Ask in February. That gives Yahoo! 24.5% of this micro market space of “visitors”, AOL 22.45%, Google and Microsoft both 21.88%, and Ask 9.21%. In terms of estimated visitors, Yahoo! and AOL still outperform Google and Microsoft is running neck-and-neck with Google. Even Ask looks pretty healthy.

So now we can look at an average 6.6 billion queries performed and divide them according to our averaged percentages. That gives Google 3.64 billion queries, Yahoo! 1.53 billion queries, Live 640 million queries, AOL 264 million queries, and Ask 217 million queries. That works out to 32 queries per visitor for Google, 12.3 queries per visitor for Microsoft, 11.9 queries per visitor for Yahoo!, 4.5 queries performed per visitor for Ask, and 2.25 queries per visitor for AOL.

Number of queries per visitor implies that people are either very satisfied with their Google results are very unsatisfied. It also implies that the chance of people seeing advertising on a Google search results page are greater than for any other search service.

One last piece of data I’d like to see, but for which there is no reporting mechanism whatsoever, is number of click-throughs on organic search results. It would help marketers to know how many results per query people click through on.

Still, the overwhelming perception in the industry is that Google dominates search, even if that perception is based solely on the total number of queries performed. What we see here is that each search service has a core audience that repeatedly uses the service, but the overall health of a search service may be indicated by the ratio of queries-per-visit. That particular value is not reasonably available to us. Yahoo!, Time Warner, and Microsoft all offer a great deal of non-search related content. Is it possible to adjust these metrics to show that Yahoo! and Time Warner actually produce higher ratios closer to Google and Microsoft respectively?

If search industry analysts looked at more useful data like these ratios, Google’s dramatic lead would be reduced to a bold front position. People would then rightly be led to ask whether quality of search is a measurable metric and if it is indeed a metric that should be measured (I think so, although this analysis does not make the case). The fading stars of Microsoft, AOL, and Ask would not seem quite so tarnished under this kind of more detailed scrutiny.

While Ask and Lycos (for whom I had too little data to incorporate into this example) both seem to be losing traffic according to various metrics, I think it would behoove all the search services other than Google to talk about the quality of their users’ experience. They have lost the war on number of queries performed. That metric screams GOOGLE every month. Yahoo! and Microsoft, at least, look much better if they focus on what happens when people use their search tools. That is where brand wars on the Internet can change directions. Proposing a reasonable, more informative but alternative metric to the one where you’re not performing well at least offers you a fighting chance to make your case before you really do start to lose market share.

I’m yet to be convinced that Google controls search as much as many analysts imply it does. The numbers just don’t add up. Not yet.