Disruptive Proactivity

[Note: most of this was originally written in summer 2005, with some minor tweaks and published in February 2006]

Work done by Government is generally one of 3 things; doing
work people don’t see, doing work people see, or moving
something from one category to the other. Search engines and
portal tools are often used to help citizens do what they
can’t find. Vast quantities of money, goodwill and time are
wasted because of poor re-implementations of a moderately
hard problem.

In a direct comparison early in 2005, 75% of users, when
shown results of search queries side by side, selected a
result from Google (limited to .gov.uk domains) over the
specialised £4.4m portal direct.gov.uk[2].

Users submitted a term to a webpage, and were shown, side by
side, the results of their search in the direct.gov.uk
engine and in google.

When the term was submitted, a request was sent to the
direct.gov.uk search engine, and the resulting HTML parsed
and the results extracted and displayed. Google’s knowledge
of .gov.uk sites was then searched[1] and similarly displayed.

The results were shown in the order that the search engine returned them,
and the default number of results for each engine shown side
by side (20 results for direct.gov.uk and 10 for Google). When a
user clicked on a page they wanted to look at, they
were first taken to a script which recorded their search term,
preferred url and which search engine returned that link, before bouncing
them to where they wanted to go. Users who opened multiple links for the
same search term (for whatever reason) generated multiple records.

Initially, we displayed google results in the left, and
Direct.gov.uk results on the right. After a month of usage,
these were switched to look into whether the earlier display
of google (on the left) was disadvantaging direct.gov.uk.
When the direct.gov.uk results were shown first, 28% of
people a small (3%) decrease in the percentage who clicked a
google result when it was shown second rather than first.

People generally searched using “keywords”, rather than a
sentance or phrase. Where longer search terms were submitted
(ie four words or more), these were generally for very
specific searches (e.g. “national spatial address
infrastructure”, or “freedom of information request
exemption”) , rather than english phrasing around what was
wanted (e.g. “how much council tax is band e in selsey”).

Direct.gov.uk “improved” searching

In the summer of 2005, partially as a result of criticism,
direct.gov.uk loudly announced a new, improved search engine
in August. Taking only results from September 2005 onwards,
repeating the above comparison, the newer engine made
no difference – the direct.gov.uk was still selected 28% of
the time (we still showed direct.gov.uk results first).

Footnote

1. used the Google API which returns identical results to www.google.com but in a more computer readable format

2. These stats were generated in a 3 month period in 2005.