Nate Reed Nate Reed - 3 months ago 29
Python Question

How can I mask elements of a record array in Numpy?

I understand how to create a masked array, and I would like to use masking in a record array so that I can access this data using named attributes. The masking seems to be "lost" when I create a record array from a masked array:

>>> data = np.ma.array(np.ma.zeros(30, dtype=[('date', '|O4'), ('price', '<f8')]),mask=[i<10 for i in range(30)])
>>> data
masked_array(data = [(--, --) (--, --) (--, --) (--, --) (--, --) (--, --) (--, --) (--, --)
(--, --) (--, --) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0)
(0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0) (0, 0.0)],
mask = [(True, True) (True, True) (True, True) (True, True) (True, True)
(True, True) (True, True) (True, True) (True, True) (True, True)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)
(False, False) (False, False) (False, False) (False, False) (False, False)],
fill_value = ('?', 1e+20),
dtype = [('date', '|O4'), ('price', '<f8')])

>>> r = data.view(np.recarray)
>>> r
rec.array([(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0),
(0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0), (0, 0.0)],
dtype=[('date', '|O4'), ('price', '<f8')])


When I access a record the data is not masked:

>>> r.date[0]
0


Unlike in the original array:

>>> data['date'][0]
masked_array(data = --,
mask = True,
fill_value = 1e+20)

fill_value = 1e+20)


What can I do? Does the record array not support masking? Browsing on the web I have seen some code examples that seem to suggest otherwise, but it wasn't very clear. Hoping I can get a good answer here.

Answer

I haven't found much documentation on numpy.ma.mrecords.MaskedRecords, except for a brief mention here. You can find some examples on how to use it by studying the unit tests that come with numpy. (e.g. /usr/lib/python2.6/dist-packages/numpy/ma/tests/test_mrecords.py).

import numpy as np
import numpy.ma.mrecords as mrecords

data = np.ma.array(
    np.ma.zeros(30, dtype=[('date', '|O4'), ('price', '<f8')]),
    mask=[i<10 for i in range(30)])

r = data.view(mrecords.mrecarray)

print(r.date[0])
# --
Comments