To Scrape or not to Scrape

Posted: June 28, 2011 in Data Journalism, My Data Journey, Open Data Movement
Tags: , ,

I’m in Berlin for the Open Knowledge Conference which you’ll be hearing about. For the last few days I’ve found myself with a mixed bunch of open data hackers and some (data) journalists. It’s the first time I’ve been away from the ScraperWiki family and seen coding in the wild. One thing that surprises me is the diversity of geeks. No one person has the same experience/background. Lots of people with no experience have jumped into it out of interest. The one realization that delights and alarms me is: I’ve been throw in at the deep end. Only a tiny amount of programmers have delved deeply into the scraping soup of the web. And journalists refuse to wander far into this level of ‘difficulty’.

I was speaking to my Canadian counter part (doppelganger), Momoko Price, from BuzzData. They’re a kind of data social network. She left the journalistic platform to join a developer platform, delving into the dirty world of data. She’s learning to code having started her data journey with more experience than myself. Yet, even though we’re on the data journalism path and have a frighteningly similar road map, our coding environment has evolved two very different species. I am chasing the needle in the haystack and not visualizing/making the haystack interactive. We both see the need for this, thankfully. Scraping is helping me evolve this speciality. More so than tinkering with software.

It is the road less travelled by and that’s making all the difference. Stay tuned for more tales from the road.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s