Kendall Weihe Kendall Weihe - 3 months ago 9
Python Question

NumPy matrix not indexing range correctly [0,0,0,5:53].shape = 43

I have a rather large numpy matrix, and I'm thinking since it is so large this is happening. It has to be some sort of pointer issue.

Anyways, I have the following matrix

(Pdb) input_4d_volume.shape
(26010, 48, 48, 48)


When I index like the following, no problem:

(Pdb) input_4d_volume[0,0,0,0:48].shape[0]
48


But when I try to do a range, the max always max's out at
48


For example, take these two cases:

(Pdb) input_4d_volume[0,0,0,1:1+48].shape[0]
47
(Pdb) input_4d_volume[0,0,0,10:10+48].shape[0]
38


The expected outputs should be:

(Pdb) input_4d_volume[0,0,0,1:1+48].shape[0]
48
(Pdb) input_4d_volume[0,0,0,10:10+48].shape[0]
38


Since,
(1+48)-1 = 48
and
(10+48)-10 = 48
(see below examples as proof)

As proof this works, I have another matrix that works perfectly fine

(Pdb) entire_volume.shape
(99, 396, 396)
(Pdb) entire_volume[0,0,0:100].shape[0]
100
(Pdb) entire_volume[0,0,10:10+100].shape[0]
100


Is this because
input_4d_volume
is too large?

Answer

Python will let you index up to a number larger than the length of the object. For example,

import numpy as np
a = np.arange(0, 10)
a[5:100]
# array([5, 6, 7, 8, 9])

If your 4-D input array is of shape (26010, 48, 48, 48), and you are slicing on the 4th dimension, you cannot end up with an 1D array large than that dimension's size. For example,

a = np.zeros((100,30,30,30))
a[0, 0, 0, 0:5].shape
# (5L,)

a[0, 0, 0, 28:32].shape
# (2L,)

In your proof that it works example, you are taking a slice on dimension three of indices 10 to 109 with entire_volume[0,0,10:10+100].shape[0]. These are within the dimension's size of 396, so you get what you expect.

If you instead did entire_volume[0,0,390:390+100].shape[0], the array returned is the same as if you sliced with entire_volume[0,0,390:396].shape[0]. Python lets you go above the max, but it can't return data that doesn't exist.