Search Engine Effectiveness - Practical Test [Digg.com This!]

I've been wondering recently how effective alternate search engines are compared to Google. As a small test I decided to take 25 searches I had performed in the last week and search on Google, Yahoo and MSN Live Search to compare results.

The following table shows the results. A score of zero means the target info was found on the search page itself (I didn't have to click through to the site). A number 1-10 is the position of the search result with the target info. A blank score means the target info was not found on the first page of links.

The score is a sum of each search position negated from 11. As so:

Target info on search page: 11-0 = 11
Target info on page result 4: 11-4 = 7
Target info on page result 10: 11-10 = 1
Target info not found: 0



Conclusions:

I was pretty suprised Google was so far ahead compared to Yahoo and MSN. Google won because it was better at finding more niche searches, whereas Yahoo and MSN failed completely. For more obvious searches all three performed flawlessly and may do for day-to-day searches.

Google tended to either have the result first, or not at all. Yahoo had a few useful results in the top five. MSN had some results nearer the bottom of the top 10. This is significant for MSN because much lower results seriously increases the time taken to find what you want.

Some oddities included Yahoo having numerous entries that were highly irrelevant and MSN having a curious slant to open source results, particularly source code.

Where is the mobile phone market going? [Digg.com This!]

To answer this question let's think back around 25 years. These were the days when computers used proprietary hardware and proprietary operating systems. Companies like Commodore, Sinclair and Apple were selling their combined hardware and software solutions. The market was largely fragmented, and the systems available varied widely in functionality.

It was only in 1982 that we saw the start of what now represents the vast majority of computer sales: clones. Compaq Computer Corp introduced the first IBM PC clone. As a result, by luck, Microsoft's operating system was able to run on computers from two hardware companies; it was no longer at the mercy of a single manufacturer. Soon this would become three, then four, then many. Whilst hardware manufacturers were able to concentrate on improving hardware technology and manufacturing process, Microsoft was able to concentrate on improving the operating system.

The above is relevant because we see the same story playing out today in the mobile phone marketplace. Manufacturers with proprietary operating systems are moving to those that are common.

A few years ago mobile manufacturers such as Nokia, Motorola and Ericsson used proprietary firmware. They did this because, just like the Commodore PET, the firmware performed such a simple function that writing it was no major overhead on development.

In the last few years we've seen mobile phone power and functionality increase dramatically, and as a result the use for a common, complex operating system has become more attractive. Quite simply, mobile manufacturers have an increasing burden to create what has gone from being a simple firmware, to a rich operating system.

Microsoft, much as they did 25 years ago, has been one of the companies to provide a common operating system for multiple hardware manufacturers. Unfortunately for Microsoft, not all hardware manufacturers were as naive as computer manufacturers when they first seized the computer OS market.

One very real competitor to Microsoft Windows Mobile is Symbian OS, originally developed by Psion and now co-owned by Nokia, Ericsson, Panasonic and others. In July 2006, it had 67% of the 'smart mobile device' market. This operating system achieves by design exactly what Microsoft achieved by luck: being a common OS capable of running on many manufacturers' hardware.

With Microsoft a very competitive player, Symbian OS having a strong grip on the market, and Apple joining the party with the market awareness and sex appeal to have an impact, only time will tell if in 25 years time, the mobile phone market will have played out exactly as the computer market has. This time around, however, Microsoft has some strong competitors to reconcile.


Of course I could also have mentioned Linux as another competitor, but I'll save this for another post.