lilla lilla - 17 days ago 21
Python Question

How can I open a 3D .npy file in R or MATLAB?

I have .npy files of the shape[512, 512, x] with x going up to maximally 400. In these arrays (is that the right term? I'm not sure about the right terminology), there are floating point numbers between zero and one. They are grey levels of a noise image. The image is 512x512 pixels and per npy file I have a different number of images that I saved.

I now would like to do some computations with the noise images but have been unable to import the arrays in R or MATLAB:

In R I tried the RcppCNPy - package and got the error "unsupported dimension in npyLoad".

In MATLAB, I tried the readNPY - function and got the error "This file does not appear to be NUMPY format based on the header." But my file has no header. Could it be that the function only works for 2D arrays?

How could I access the 3D-array that I saved before?

Any help in MATLAB or R would be very much appreciated!

Answer

npy is a unique numpy file format. And yes, it does have a header block that contains information like version, dimensions, strides, and dtype.

A quick glance at the readNpy shows it is relatively new and experimental. Check with them about capabilities.

np.savetxt writes text CSV files. This is widely used (look at the number of SO question about np.genfromtxt and np.loadtxt), but by the very nature of 'paper' is 2d - lines of columns. I don't know of any standard to write higher dimensional arrays - reshaping to 2d and back is probably the simplest.

scipy.io.savemat can write MATLAB compatible .mat files. This (and loadmat) is capable of reading/writing higher D arrays, and MATLAB structures and cells. This is the best tested file interchange between numpy and MATLAB.

A newer MATLAB save format uses HDF5 files. The Python h5py package can read and write those as well. There have been a scattering of SO questions about reading this type of file as produced by MATLAB. Python-created HDF5 dataset transposed in Matlab

Keep in mind that MATLAB dimension ordering is equivalent to the numpy 'F' order.

Compare:

xf=np.arange(12).reshape(3,4,order='F')   # saved with savemat

>> xf(:).'      # in octave
ans =
   0   1   2   3   4   5   6   7   8   9  10  11

without the order=F, octave gives

 0   4   8   1   5   9   2   6  10   3   7  11

and

>> xf
xf =
   0   3   6   9
   1   4   7  10
   2   5   8  11

>> x
x =
   0   1   2   3
   4   5   6   7
   8   9  10  11

np.asfortranarray(x) doesn't seem to make a difference.

Comments