If you follow Scrape_No10 on twitter you will be receiving all the meetings, gifts and hospitalities received at No.10 by ministers, special advisers and permanent secretaries. If you follow #Scrape10 then you should be getting those as well as all the tweets relating to any item of data. The current database should be tweeted out by July.
The information contained in each tweet comes from the data published by the Cabinet Office. I scraped each data set and sorted the aggregated information chronologically, so that it can be tweeted out in the order in which the events happened (not every day was given). The links to the original data source and the scraped data are given at the bottom of this post.
With respect to what I have done, I would like to remind people that Freedom of Information does not equate to contextual knowledge regarding the information or making it useful. I am a data journalist. Data is my beat. But data is a public right. Not just your data but also the data of the people who work for you – government data.
Journalism involves information but also conversation. Each data entry now has the ability to start a conversation. Just use #Scrape10. If a tweet is interesting or someone somewhere has added a piece of news relating to the tweet, then #Scrape10 should trend and the tweet should be sent around to the community where the information matters. That’s the theory.
Information is now socially enabled and should be socially enabling. What you would like to know, what matters to you or your wider social community should not only be made available to you but should be made useable in a way that matters.
I am an experimental data journalist, playing with code. All my source code for collecting this data is open and you can download the entire dataset. The code to get it onto twitter is not available, as the publication of the authorisation keys would allow people to hack into the account. I have also written a scraper to store #Scrape10 tweets into a database everyday, so you can catch them all here if you want to.
You can also read a previous post on the Special Advisers’ gifts and hospitality dataset here.
Data Miner UK 
What a great idea.
Would it be possible to link to the source in each tweet? I note some of the tweets are (necessarily) truncated and this would allow people to see the full details.
>The code to get it onto twitter is not available, as the publication of the authorisation keys would allow people to hack into the account.
Could you publish the code with the authorisation keys redacted or extracted into configuration?
Here’s a blog post I wrote for Scraperwiki and you’ll find the link to the template. Would you like to help me figure out how to get a link to the source?
[…] Scrape No.10 converts data published by the Cabinet Office about all the meetings, hospitality and gifts received at No.10 into a twitter stream. In order to get this going Nicola had to write a scraper to collect information from various web-published data sets, incorporate them into a central spreadsheet and trickle them out slowly via. Twitter. More information is available on her blog. […]
[…] Posts AboutScrape No.10Data Journalism: The Story So FarData Harvesting in BrusselsThe Best Things in Life are Free […]
[…] ran this as an experiment, explaining all on my own blog. I was surprised when a certain ScraperWiki star wanted the Twitter bot code so I’ve copied […]
[…] meetings, gifts and hospitalities is back up and tweeting. You can read the post about its creation here and download all the data the account contains. This account needs more coding maintenance than the […]
[…] my journey to seek validation. I have acted to develop my business acumen, creating point stories, Twitter accounts and […]
[…] social media which I want to use to generate stories that might not necessarily be put on it. In my previous post I wrote how I am tweeting out all the Ministers’, Special Advisers’ and Permanent […]