Disruptive Proactivity

Government makes available large amounts of numbers about other levels of (local) Government. It locks them up in boring spreadsheets, with dull tables, in much the same way it buries useful information behind the clunky direct.gov.uk portal.

Some of this data is dull, some of it is quite interesting. And when it’s all made available in machine readable formats, it becomes easier to fiddle with it and do nice things which it wasn’t actually ever intended to be used for – such as be the input to a card game. The aim is to win all the cards, by selecting a better metric than your opponent (the computer). Thanks to Matthew for the code

Why do this? Because we can (and I needed an example for the NeSS API although didn’t end up using it due the lack of a postcode lookup file for each LA) and because it’s useful to learn about the country as it is, rather than how we think it is (or we thought it was). There’s much more than can be done in this area.

For that reason, I’ve also done one based on country information (see the iQuango announcement post (which escaped when I pressed the wrong button, so is incomplete – sorry)).

There are often conversations about what Government organisations could do to make their data more available. This data is already made available in multiple forms; most websites have a version covering the latest fad when it was being designed (the varying “mobile” access has been a constant for the last few years, although the actual methods have varied – think WAP).

What would happen if a site was designed to be reused, as well as used?

It would probably involve outputting XML or CSV or similar, rather than HTML, but the content may be substantially similar. Other projects go from HTML to XML…

One project I’ve recently been working on involves reusing data from Neighbourhood Statistics.

Neighbourhood statistics has many constituent parts, but it’s main aim is for you to give it an area, and it tell you statistics about that area. It can do that in multiple different ways, as data, as descriptions and with pretty maps, but the core is the data. All easily accessible in nice friendly ways. In fact, NeSS is a good example of what most sites should do about making data accessible to people.

Although for my purposes, I didn’t want to type 140,000 postcodes into NeSS. I’m lazy and that type of work is what we should get the computer to do. The data is well formed – we can just scrape it.

While on a singular basis, that worked (results will be available soon), what about something for next time?

NeSSY is designed to make it easier to make repeated requests for specific information (or find out what is available for an area) from Neighbourhood Statistics. It operates the same way as the NeSS web interface does, but you can tell it which links you’d like it to follow to get the data you want.

Retrofitting this on has significant downsides, as significant amounts of context are lost in the conversion from the database into HTML, and putting it back requires both vast amounts of care (getting it wrong is bad), and more knowledge/time, I’ve given it so far (about 8 hours of work). There may be areas that it doesn’t cope with at all.

What it does show is that there are relatively simple and fast ways of redisplaying information that Government could add to their systems which make reuse far easier.

Multiple choice games could be “fun” to see how much people know about their local areas… The possibilities of data reuse are endless. If anyone is interested in working on further development, the code is here, and please drop me an email to let me know what you’re working on