MRocklin MRocklin -4 years ago 176
Python Question

How to flatten a memoryview?

I have a

memoryview
with non-trivial strides like the following:

>>> mv.strides
(96, 32, 8)


I want to write this
memoryview
to a socket but my networking library seems to expect memoryviews with
mv.strides == (1,)
. Is there a way in Python to flatten this
memoryview
?

>>> flatten(mv).strides
(1,)


Ideally this would neither affect the underlying bytes nor require a copy. I could do this with NumPy, but I'd rather keep things general if possible.

Edit: Here is some code that produces such a memoryview



In [1]: import numpy as np
In [2]: x = np.ones((2, 3, 4))
In [3]: x.data
Out[3]: <memory at 0x7f371aa849a8>

In [4]: x.data.strides
Out[4]: (96, 32, 8)

Answer Source

Just for clarification, you probably know this but I think it's better to make sure:

  • The length of the strides tuple represents the number of dimensions, so (1, ) and (8, ) are both one dimensional and (10, 2) and (20, 1) are both two-dimensional.
  • For C-contiguous arrays the last element in the strides tuple represents the itemsize of the items in your memoryview. That's not always correct: sometimes the values are padded then it will be bigger than the actual itemsize - but in most cases it represents the itemsize.

So you don't only want your memoryview flattened but it should be flattened and have an itemsize of 1.

In Python 3.3 the memoryview.cast method was added that makes flattening your array trivial:

cast(format[, shape])

Cast a memoryview to a new format or shape. shape defaults to [byte_length//new_itemsize], which means that the result view will be one-dimensional. The return value is a new memoryview, but the buffer itself is not copied. Supported casts are 1D -> C-contiguous and C-contiguous -> 1D.

The destination format is restricted to a single element native format in struct syntax. One of the formats must be a byte format (‘B’, ‘b’ or ‘c’). The byte length of the result must be the same as the original length.

So it only works if you cast to char (c), unsigned char (B) or signed chars (b) and it's C-contiguous.

>>> import numpy as np

>>> memview = memoryview(np.ones((2, 3, 4)))
>>> memview.cast('b').strides   # or 'B' or 'c'
(1, )

However this is flattened and interpreted as 1-byte values. If you just want it flattened you need to cast it to the original type again:

>>> memview.cast('b').cast(memview.format)

That will be one-dimensional but it won't have strides of (1, ) because floats are 8 bytes (at least if it's float64):

>>> memview.cast('b').cast(memview.format).strides
(8, )
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download