marco marco - 4 months ago 12
Python Question

How to speed up nested for loops in Python

I have the following Python 2.7 code:

listOfLists = []
for l1_index, l1 in enumerate(L1):
list = []
for l2 in L2:
for l3_index,l3 in enumerate(L3):
if (L4[l2-1] == l3):
value = L5[l2-1] * l1[l3_index]
list.append(value)
break
listOfLists.append(list)


with the L1,L2,L3,L4,L5 lists being:

L1 = [[0.60, 0.95, 0.38, 1.02, 0.29, 0.43], [0.40, 0.09, 0.87, 0.85, 0.70, 0.46], [0.67, 0.91, 0.66, 0.79, 0.86, 0.06], [0.59, 1.81, 0.05, 1.88, 0.20, 0.48], [0.64, 0.34, 0.37, 1.39, 0.56, 0.27], [0.56, 0.34, 0.68, 2.79, 0.18, 0.42], [0.42, 1.67, 0.04, 0.44, 0.25, 0.94], [0.32, 1.92, 0.95, 2.85, 0.95, 0.96], [0.50, 0.68, 0.84, 1.79, 0.35, 0.09], [0.34, 0.66, 0.85, 0.35, 0.38, 0.59], [0.50, 0.79, 0.45, 2.93, 0.50, 0.92], [0.11, 0.11, 0.93, 1.11, 0.81, 0.49]] # a list of 12 sublists
L2 = [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
L3 = [480, 120, 35, 0, 520, 300]
L4 = [120, 120, 120, 0, 300, 35, 35, 520, 300, 480, 120, 480, 0, 35, 0, 0, 300]
L5 = [5.4, 2.83, 1.16, 6.9, 0.76, 2.15, 5.61, 3.12, 1.57, 0.08, 5.36, 0.2, 1.2, 1.4, 2.9, 2.1, 3.5]


These are only examples; in reality the lists contain hundreds of thousands of numbers. The interpreter takes tens of seconds to calculate the three nested
for
loops.

Is it possible to somehow speed up this code, e.g. using
itertools
or any other module/function?

EDIT: I can not use non-standard python 2.7 modules (numpy, scipy...)

Answer

Since you said the readability is not important as long as it speeds up the code, this is how you do the trick:

[[L5[l2 - 1] * sl1 for sl1, l3 in zip(l1, L3)
     for l2 in L2 if L4[l2 - 1] == l3]
 for l1 in L1]

This code is 25% faster than for loop. But trust me I will shoot him whoever wrote this in my code.