How to backup your twttr
UPDATE (16 Feb 2013) twttr now allows you to download your twttr archive
I’ve been a twttr user since July 14, 2006. It recently dawned on me that my twttr archive is an (albeit mudane) running commentary of my life. I’ve never been one to keep a diary so I thought it important that I have a backup — this is what I did.
Currently, you can’t get more than ~3000 updates from the twttr API. It is what it is. To get to the point where I own all 19,477 of my twttr updates took a bit of time. Your mileage may vary.
Request to access your personal data
When I realised that there was no feasible way of getting access to my historical twttr updates, I found Privacy International’s handy guide which walks you through the convoluted process of filing a request to Twitter for all the data they hold on a specific account if you’re a citizen of the EU.
I sent my request on April 18, 2012 and received a ZIP file containing a bunch of text files on May 11, 2012.
When I received my ZIP file I was thrilled, skimmed through my twttr updates then promptly forgot about it. Dave Winer’s Twitter is a Corporate API motivated me to make my twttr archive robust and automatic.
Getting my last 3000 updates
Before I could look at the automation bit I needed to get hold of all the updates from May 10 (the latest update in the
rey-tweets.txt file from the ZIP) up to my latest update (July 9).
After setting it up, running the following command gave me more than I needed:
t timeline @rey --csv --number 3000 > rey-twttrs.csv
So I now have all my updates from the beginning of time!
There’s probably a zillion different ways of automagically backing up your updates (eg. jphpsf’s blog post about backing up with t has a handy shell script) but it took me 30 seconds to find an ifttt recipe which sticks all your new updates in a single Evernote er, note.
- Implement a daily shell script backup deal to replace ifttt
- Whack the entirety of my updates into a single database
- Sweet data analysis (How often do I have a hangover on a Thursday?)