Posts Tagged ‘media’

So I’m in the US, preparing to roll out events. To make decisions as to where to go I needed to get data. I needed numbers on the type of people we’d like to attend our events. In order to generate good data projects we would need a cohort of guests to attend. We would need media folks (including journalism students) and people who can code in Ruby, Python, and/or PHP. We’d also like to get the data analysts, or Data Scientists, as they are known as, particularly those who use R.

So with assistance from Ross Jones and Friedrich Lindenberg, I scraped the locations (websites and telephone numbers if they were available) of all the media outlets in the US (changing the structure to suit its intended purpose i.e. why there are various tables), data conferences, Ruby, PHP, Python and R meetups, B2B publishers (Informa and McGraw-Hill) and top 10 journalism schools. I added small sets in by hand such as HacksHackers chapters. All in all, nearly 12,800 data points. I had never used Fusion Tables before but I have heard good things. So I mashed up the data and imported it into Fusion Tables. So here is it (clicki on the image as sadly wordpress.com does not support iframes):

Click to explore on Fusion Tables

Sadly there is a lot of overlap so not all the points are visible. Google Earth explodes the points on the same spot however it couldn’t handle this much data when I exported it. Once we decide where best to go I can hone in on exact addresses. I wanted to use it to pinpoint concentrations, so a heat map of the points was mostly what I was looking for.

Click to explore on Fusion Tables

Using Fusion Tables I have then break down the data for the hot spots. I’ve looked at the category proportions and using the filter and aggregate, made pie charts (see New York City for example). The downside I found with Fusion Tables is that the colour schemes cannot be adjusted (I had to fix them up using Gimp) and the filters are AND statement (no OR statement option). The downside with US location data is the similarity of place names across states (also having a place and state name the same), so I had to eye up the data. So here is the breakdown for each region where the size of the pie chart corresponds to the number of data points for that location. It is relative to region not across.

Of course media outlets would outnumber coding meetups, universities and HacksHackers Chapters, but they would be a better measure of population size and city economy.

What I’ve learnt from this is:

  1. Free tools are simple to use if you play around with them
  2. They can be limiting for visual mashups
  3. The potential of your outcome is proportional to your data size, not your tool functionality (you can always use multiple tool)
  4. To work with different sources of data you need to think about your database structure and your outcome beforehand
  5. Manipulate your data in the database not your tool, always keep the integrity of the source data
  6. To have data feed into your outcome changes your efforts from event reporter to source
This all took me about a week between doing other ScraperWiki stuff and speaking at HacksHackers NYC. If I were better at coding I imagine this could be done in a day no problem.

The road to No.10 is paved with advisers, they lead you in, they open doors. Often for themselves. Previous advisers include Alastair Campbell, Ed Balls and the Miliband brothers. Until they’re in the door they generally don’t command the political spotlight. That is, unless they’re on the way out like Andy Coulson. What they do command is fine wining and dining.

The Cabinet Office publishes Special Advisers’ gifts and hospitality in various Excel sheets that are filled-in depending on how much coffee the civil servant had that morning i.e. inconsistently. They weren’t even consistent with the appointed minister the adviser falls under. So I scraped it and put all the files into one download which covers May to September 2010. You can get it all by hitting the ‘Download spreadsheet (CSV)’ link here.

The Trends:

Here are the advisers listed according to the amount of hospitality they received:

Note that Nick Clegg’s chief adviser, Jonny Oates, has been taken out the most followed by the then PM’s communications chief, Andy Coulson. Most hospitality is provided by media organisations (see table below) and by using Google Refine I dug deeper into the data to look for a bias between advisers for the Prime Minister and Deputy Prime Minister (seeing as there’s a party split). It turns out the BBC only court Cameron’s advisers (15 times in 5 months). The same is true of the Daily Mail. Whereas The Financial Times dine only with those close to Clegg. The Guardian similarly enjoy Lib Dem company, inviting them to their table twice as many times as they did the Tories.

What’s very noticeable from this information is that Special Advisers are wined and dined mostly by media organisations. Here is a list of the top 10 hospitality givers:

If you add up all of Rupert Murdoch’s empire, they account for 20 occasions split 13:7 Cameron’s to Clegg’s.

The close relationship between advisers and media organisations (this is all within a five month period) makes me wonder: when a ‘No.10 insider’ or ‘someone close to the Prime Minister’ is quoted, how often is that piece of information plucked from the lips of these well-fed advisers? A lot I imagine.

The Outliers:

In fact, media and PR are so predominant in hospitality for advisers, I’ve decided to list the rest of the givers in order of how many times they appear in the data: Bell Pottinger (mostly business clients, Airbus, Sky, Unilever, etc), News Corporation, Tetra Strategy (clients include the Government of Dubai and the jailed Russian billionaire,
Mikhail Khodorkovsky), The Daily Telegraph, The Mail on Sunday, The Sunday Times, The Telegraph, Alexander Kutner, Baron Wolfson of Aspley Guise (Conservative life peer and CEO of Next), Business in the Community, Center for Court Innovation (New York think-tank), Citi, Connect Communications (lobbying), Demos (think-tank), General Sir Richard Dannatt, ITN, ITV, Ian Osborne and Partners, Institute for Public Policy Research (think-tank), Islamic Relief, James Kempton, Lansons Communications (clients include J.P. Morgan, Lloyds TSB and Barclays), London Palladium (Whoopi Goldberg), Malaria No More, Martyn Rose, Medley Global Advisors (“provider of macro policy intelligence service for the world’s top hedge funds, institutional investors, and asset managers”), News International, Lawn Tennis Association, Not to Scale, Open Road, Pakistan International Airlines, Policy Exchange (think-tank), RSA, Ramesh Dewan, Richard Thaler, Royal Bank of Scotland, SAB Miller (De Klerk Foundation Event), Save the Children, Taxpayers’ Alliance, The Daily Telegraph and The Daily Mirror, The Economist, The Evening Standard, The Spectator, The Sun, The Sunday Express, UK Music, Wall Street Journal and Wellington College.

Bell Pottinger and Martyn Rose are now with the Big Society Network.

Only six entries weren’t lunch or dinner dates. Steve Hilton was given champagne from Not to Scale, Steve Chatwin received concert tickets from Malaria No More, Naweed Khan got his flights upgraded by Pakistan International Airlines, Andy Coulson was given theatre tickets by Whoopi Goldberg and a bottle of wine by a one Alexander Kutner.

The Anomalies:

Now the only Alexander Kutner I can find happens to have been the Vice President and Principal Engineer of Software Development at Electronic Evidence Discovery. They reduce the risk of electronic discovery, a process which involves digital forensics analysis for recovering evidence.  See comment below regarding the identity of Alexander Kutner

Also, Ian Osborne and Partners, who dined Tim Chatwin, has no existence according to Google.

The MetaData:

What’s missing is what went on at these meals. Who attended. What was said, or agreed upon. Who was being represented. What goes on is not an entry in the data sheets and it never will be. But this data should make you more aware of the existence of these meals on deals.

You can find a list of Special Advisers and their salaries here.