I spent yesterday in the shiny new Guardian offices at Rewired State – National Hack the Government Day – which was a lot of fun, despite the early start. There was a really friendly atmosphere and lots of interesting projects going on.
I teamed up with Paul to work on some improvements for the Office for National Statistics website. We’d noticed that while there are many timeline datasets available on the site, the only way to view graphs of the data is to download a Windows application called Navidata. Although it’s free, this application is pretty clunky and hard to use and doesn’t generate very clear graphs. Also, rather obviously, it isn’t much use to Mac or Linux users.
Update: More about our hack from Paul – GreaseMonkey vs the ONS at Rewired State
Here’s an example of a graph generated by the GreaseMonkey script :-
So now anyone can use this script to view graphs of ONS timeline datasets :-
Paul presenting our hack at the end of the day :-
For the second part of our hack, we wanted to make a better way of navigating to the datasets. The current system uses a series of forms in a wizard-like experience which is quite hard to understand. Also, because of the form submissions (i.e. HTTP POSTs) required to access the datasets, there are no permanent URLs available to navigate directly to a dataset. We decided to try to remedy this by using Mechanize to scrape the timeline data from the website and then build a simplified static website using only links (i.e. HTTP GETs). However, we hit a snag in automatically submitting the last form in the wizard and ran out of time. I might try and have a go at this in the future if I get time.
We did notice that this last form had a hidden field containing what appeared to by a SQL query, making it ripe for SQL injection attack, but fortunately we managed to restrain ourselves.