Barry Barry - 1 year ago 72
Python Question

Trying to Gather all tweets from the past 24hours and put them into a CSV file

Im trying to gather all the tweets from the last 24 hours and put them into a CSV file

When i do this i get

_csv.Error: iterable expected, not datetime.datetime

As an error

Can anyone help tell me how to get rid of this error and any other improvements that could possibly be made to the code

def get_all_tweets(screen_name):
# Twitter only allows access to a users most recent 3240 tweets with this method

# authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_token, consumer_secret)
auth.set_access_token(access_token, access_secret)
api = tweepy.API(auth, wait_on_rate_limit=True)

# initialize a list to hold all the tweepy Tweets
alltweets = []

# make initial request for most recent tweets (20 is the maximum allowed count)
new_tweets = api.home_timeline (screen_name=screen_name, count=20)

# save most recent tweets

# save the id of the oldest tweet less one
oldest = alltweets[-1].id - 1

page = 1
deadend = False

while len(new_tweets) > 0:
print ("getting tweets before %s" % (oldest))

# all subsiquent requests use the max_id param to prevent duplicates
new_tweets = api.home_timeline(screen_name=screen_name, count=20, max_id=oldest, page = page)

# save most recent tweets

# update the id of the oldest tweet less one
oldest = alltweets[-1].id - 1

print ("...%s tweets downloaded so far" % (len(alltweets)))

for tweet in alltweets:

if ( - tweet.created_at).days < 1:

# transform the tweepy tweets into a 2D array that will populate the csv
outtweets = [tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")]

deadend = True
if not deadend:
page += 1

# write the csv
with open('%s_tweetsBQ.csv' % screen_name, 'w') as f:
writer = csv.writer(f)
writer.writerow(["id", "created_at", "text"])

print ("CSV written")

if __name__ == '__main__':
# pass in the username of the account you want to download


(most recent call last):
File "C:\Users\Barry\workspace\TwitterTest\Test1\", line 77, in <module>
File "C:\Users\Barry\workspace\TwitterTest\Test1\", line 70, in get_all_tweets
_csv.Error: iterable expected, not datetime.datetime

Answer Source

outtweets only ever contains a single row of data. writer.writerows() expects a list of rows, that is, a list of lists:

  [columns, in, row, 1],
  [columns, in, row, 2],

You are setting outtweets like this:

outtweets = [tweet.id_str, tweet.created_at, tweet.text.encode("utf-8")]

That's just a single row. To pass this to writerows, you need to accumulate each row of data into a list, and then pass that list to writerows.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download