ArcAngel ArcAngel - 7 months ago 21
Python Question

Python, looking to extract values from a masked array, then rebuild an array

I’m currently writing something that involves a lot of noise I’m attempting to remove, but in order do this I initially used masks, but the way in which I’m analysing the data breaks using a mask.

The masking is done, I’m looking to extract the data that is not masked, run analysis on this, then rebuild the array with the original order.

array([[3, 0, 3],
[6, 7, 2],
[2, 5, 0],
[2, 1, 4]])


Make Mask

array([[-, -, -],
[6, 7, 2],
[-, -, -],
[2, 1, 4]])


Extract Values

array([[6, 7, 2],
[2, 1, 4]])


Do analysis

Rebuild Array

array([[-, -, -],
[6, 7, 2],
[-, -, -],
[2, 1, 4]])


I’m hoping for an efficient way of doing this as I’m dealing with 100 million data points. Any suggestions are appreciated.

Answer

You could use

masked[~masked.mask] = analyzed.ravel()

to reassign the analyzed values to the masked array.


import numpy as np

arr = np.array([[3, 0, 3],
                [6, 7, 2],
                [2, 5, 0],
                [2, 1, 4]])

masked = np.ma.masked_array(arr, mask=False)
masked.mask[::2, None] = True

extracted = np.ma.compress_rows(masked)

analyzed = extracted*10

masked[~masked.mask] = analyzed.ravel()

print(masked)

yields

[[-- -- --]
 [60 70 20]
 [-- -- --]
 [20 10 40]]
Comments