Technolohic27 Technolohic27 - 1 year ago 58
Python Question

How to make for loop run faster?

Here is my for loop. It takes very long to run over 100,000 iterations.

for urlt in xrange(0, len(clean_url_data)):
print urlt

Answer Source

Rather than extending the website_title list using website_title.append(urlt) and then immediately overwriting that urlt item with your title data, you should just append the title data directly to the list.

Also, it's better to iterate directly over the clean_url_data list, rather than using an integer loop to index clean_url_data. If you need an index you can use enumerate. like this:

website_title = []
for i, url in enumerate(clean_url_data):
    print i

If you don't really need that index, you can make things simpler. Here's an approach using a list comprehension, which is a little faster than using append in a loop.

website_title = [g.extract(url).title for url in clean_url_data]

The list comprehension uses a special LIST_APPEND bytecode, which is basically an operator, so it's faster than looking up the list's .append method then calling it; calling a Python function is relatively slow compared to executing an operator.