math4tots math4tots - 2 months ago 23
Python Question

Python numpy array vs list

I need to perform some calculations a large list of numbers.

Do array.array or numpy.array offer significant performance boost over typical arrays?

I don't have to do complicated manipulations on the arrays, I just need to be able to access and modify values,


import numpy
x = numpy.array([0] * 1000000)
for i in range(1,len(x)):
x[i] = x[i-1] + i

So I will not really be needing concatenation, slicing, etc.

Also, it looks like array throws an error if I try to assign values that don't fit in C long:

import numpy
a = numpy.array([0])
a[0] += 1232234234234324353453453

On console I get:

a[0] += 1232234234234324353453453
OverflowError: Python int too large to convert to C long

Is there a variation of array that lets me put in unbounded Python integers?
Or would doing it that way take away the point of having arrays in the first place?

Answer Source

Your first example could be speed up. Python loop and access to individual items in a numpy array are slow. Use vectorized operations instead:

import numpy as np
x = np.arange(1000000).cumsum()

You can put unbounded Python integers to numpy array:

a = np.array([0], dtype=object)
a[0] += 1232234234234324353453453

Arithmetic operations compared to fixed-sized C integers would be slower in this case.