Cherona Cherona - 5 months ago 18
Python Question

Looping through and concatenating log files

So I have a small python script that will ask the user for a username then it will rip their logs for a certain twitch.tv channel from overrustlelogs.net

At the moment I just have 2 lists, one for months one for the year because the log URL takes the format of


http://overrustlelogs.net/Channel%20chatlog/MonthName%20Year/user.txt


months = ['September','October','November','December','January','February','March','April','May','June','July','August','September','October','November','December','January','February','March','April','May','June','July','August']
years = ['2014','2014','2014','2014','2015','2015','2015','2015','2015','2015','2015','2015','2015','2015','2015','2015','2016','2016','2016','2016','2016','2016','2016','2016']


As you can see my way of using a for loop and taking the index of each loop is pretty stupid because I have to manually edit the list each month.

What's a better way to do this? I want the program to download the logs from September 2014 to the current Month and Year

Answer

Here's a simple script that will help you.

  1. It creates a list with days between certain dates using datetime.timedelta().

  2. Then it format these days using only month name and year (%B-%Y)

  3. And finally remove duplicates by using an if not in condition while looping through the calculated days between t0 and t1.


Code:

t0 = '21-06-2014'
t1 = '07-01-2016'

#formatting in the proper way to use timedelta()
start = datetime.datetime.strptime(t0, "%d-%m-%Y")
end = datetime.datetime.strptime(t1, "%d-%m-%Y")

#generating dates using list comprehension and timedelta()
date_generated = [start + datetime.timedelta(days=x) for x in range(0, (end-start).days)]

#removing duplicates using loop and 'if not in'
dates = []

for date in date_generated:
    s = str(date.strftime("%B-%Y"))
    if s not in dates:
        dates.append(s)

#printing dates    
for date in dates:
    print date

Output:

June-2014
July-2014
August-2014
September-2014
October-2014
November-2014
December-2014
January-2015
February-2015
March-2015
April-2015
May-2015
June-2015
July-2015
August-2015
September-2015
October-2015
November-2015
December-2015
January-2016

With the dates list generated, we may now create the urls using:

for date in dates:
    MonthName = date.split('-')[0]
    Year = date.split('-')[1]
    url = 'http://overrustlelogs.net/Channel%20chatlog/' + MonthName + '%20' + Year + '/user.txt'
    print url

Output:

http://overrustlelogs.net/Channel%20chatlog/June%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/July%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/August%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/September%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/October%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/November%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/December%202014/user.txt
http://overrustlelogs.net/Channel%20chatlog/January%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/February%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/March%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/April%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/May%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/June%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/July%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/August%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/September%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/October%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/November%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/December%202015/user.txt
http://overrustlelogs.net/Channel%20chatlog/January%202016/user.txt