Download all tweets to a file
Sun 30 May 2010 17:34
I found a post on archiving twitter data with python that would download all old tweets into a file in either json or xml format. Alas it didn't work right away.
Here it is adapted for Python 3 and the new twitter API
import urllib.request
username = "username"
password = "password"
format = "xml" # json or xml
filename = "tweet_archive.xml" # filename of the archive
tweets = 700 # number of tweets
#pages = (int(float(tweets)/float(80)))+1
pages = 35
auth = urllib.request.HTTPPasswordMgrWithDefaultRealm()
auth.add_password(None, "http://api.twitter.com/", username, password)
authHandler = urllib.request.HTTPBasicAuthHandler(auth)
opener = urllib.request.build_opener(authHandler)
urllib.request.install_opener(opener)
i = 1
response = ""
print
print ("Downloading tweets. Note that this may take some time")
while i <= pages:
request = urllib.request.Request("http://api.twitter.com/1/statuses/user_timeline." \
+ format + "?page=" + str(i))
response = response + urllib.request.urlopen(request).read().decode('utf8')
i = i + 1
handle = open(filename,"w")
handle.write(response)
handle.close()
print ("Archived " + str(tweets) + " of " + username + \
"\'s tweets to " + filename)
No feedback yet
Leave a comment
| « The Argos Fiasco - Computers Using Computers | The Argos Fiasco - The House by the Cemetery » |