nhawkins nhawkins - 8 months ago 46
Python Question

Iterating on data in two 3D arrays python

I'm trying to perform a number of functions to get some results from a set of satellite imagery (in the example case I am performing similarity functions). I first intended to iterate through all the pixels simultaneously, each containing 4 numbers, then calculating a value for each one based off these too numbers then write it to an array e.g scipy.spatial.distance.correlation(pixels_0, pixels_1).

The issue I have is when I run this loop I am having issues getting it to save to an array 1000x1000 giving it a value for each pixel.

array_0 = # some array with dimensions(1000, 1000, 4)
array_1 = # some array with dimensions(1000, 1000, 4)

result_array = []

for rows_0, rows_1 in itertools.izip(array_0, array_1):
for pixels_0, pixels_1 in itertools.izip(rows_0, rows_1):
results = some_function(pixels_0, pixels_1)
print results
>>> # successfully prints desired results
>>> # unsuccessful in creating the desired array

I am getting the results I want to get printing down the run window but I don't know how to put it back into an array which I could manipulate in a similar manor. Are my for loops the issue or is this a simple issue with appending it back to arrays? Any explanation on speeding it up would also be great too as I'm very new to python and programming all together.

a = np.random.rand(10, 10, 4)
b = np.random.rand(10, 10, 4)

def dotprod(T0, T1):
return np.dot(T0, T1)/(np.linalg.norm(T0)*np.linalg.norm(T1))

results =dotprod(a.flatten(), b.flatten())
results = results.reshape(a.shape)

This now causes ValueError: total size of new array must be unchanged,
and when printing the first results value I receive only one number. Is this the fault of my own poorly constructed function or in how I am using numpy?


The best way is to use Numpy for your task. You should think in vectors. And you should write your some_function()to work in a vectorized manner. Here is an example:

array_0 = np.random.rand(1000,1000,4)
array_1 = np.random.rand(1000,1000,4)
results = some_function(array_0.flatten(), array_1.flatten()) ## this will be (1000*1000*4 X 1)
results = results.reshape(array_0.shape)   ## reshaping to make it the way you want it.