National Hack the Government Day

Paul Battley & I attempt to make the website for the Office for National Statistics more useful.

I spent yesterday in the shiny new Guardian offices at Rewired State – National Hack the Government Day – which was a lot of fun, despite the early start. There was a really friendly atmosphere and lots of interesting projects going on.

ONS Graphs

I teamed up with Paul to work on some improvements for the Office for National Statistics website. We’d noticed that while there are many timeline datasets available on the site, the only way to view graphs of the data is to download a Windows application called Navidata. Although it’s free, this application is pretty clunky and hard to use and doesn’t generate very clear graphs. Also, rather obviously, it isn’t much use to Mac or Linux users.

Update: More about our hack from Paul – GreaseMonkey vs the ONS at Rewired State

So for the first part of our hack, in a fine example of post-modern programming, we decided to use JavaScript to generate a graph in the browser using the HTML table form of the data. Thanks to an article by Rebecca Murphey, we decided to use jQuery together with Rebecca’s graphTable jQuery plugin which uses the flot jQuery plugin to actually draw the graphs. We used the GreaseMonkey, the Firefox add-on, to write a little script to pull everything together.

Here’s an example of a graph generated by the GreaseMonkey script :-

So now anyone can use this script to view graphs of ONS timeline datasets :-

Paul presenting our hack at the end of the day :-

ONS Navigation

For the second part of our hack, we wanted to make a better way of navigating to the datasets. The current system uses a series of forms in a wizard-like experience which is quite hard to understand. Also, because of the form submissions (i.e. HTTP POSTs) required to access the datasets, there are no permanent URLs available to navigate directly to a dataset. We decided to try to remedy this by using Mechanize to scrape the timeline data from the website and then build a simplified static website using only links (i.e. HTTP GETs). However, we hit a snag in automatically submitting the last form in the wizard and ran out of time. I might try and have a go at this in the future if I get time.

We did notice that this last form had a hidden field containing what appeared to by a SQL query, making it ripe for SQL injection attack, but fortunately we managed to restrain ourselves.

Thanks to Ben Dodson & Paul Battley for the photos.