Michael Michael - 5 months ago 7
Python Question

Are Python's map and filter really so ridiculously fast or are my measurements wrong?

I do some speed tests for operations on vectors/lists. Suprisingly,

map
and
filter
seem to beat
numpy
by a factor of 5-10x. See the following short code sample with times given (Full code below.):

n = 10000000
a = np.random.rand(n)
b = np.random.rand(n)
c = a + b # time = 0.07 s
d = a[a < 0.3] # time = 0.09 s

a = [random.random() for x in range(0, n, 1)]
b = [random.random() for x in range(0, n, 1)]
c = map(lambda x, y: x + y, a, b) # time = 0.006s
d = filter(lambda e: e < 0.3, a) # time = 0.001s


Is it really possible that
map
and
filter
are that much faster than the
numpy
operations? Or are my measurements flawed? You can see the full code below:

import numpy as np
import time
import random

class StopWatch:

def __init__(self, str):
self.str = str
self.t = time.time()

def stop(self):
t = time.time()
print("time = " + str(t - self.t) + " s for " + self.str)

n = 10000000
a = np.random.rand(n)
b = np.random.rand(n)

sw = StopWatch('numpy')
c = a + b
sw.stop()

sw = StopWatch('numpy')
d = a[a < 0.3]
sw.stop()

a = [random.random() for x in range(0, n, 1)]
b = [random.random() for x in range(0, n, 1)]

sw = StopWatch('list')
c = map(lambda x, y: x + y, a, b)
sw.stop()

sw = StopWatch('list')
d = filter(lambda e: e < 0.3, a)
sw.stop()


If my measurements are correct, WHY is it that much faster?

Answer

My guess is that c = map(lambda x, y: x + y, a, b) is actually not calculated. In python 3, map and filter are evaluated lazy, and therefore not before they have to be evaluated.

You can verify this by adding a list(c) before you stop the timer, though this might affect the time a little more for the list creation.

Comments