jlconlin jlconlin - 25 days ago 16
Python Question

Using scipy.stats.chisquare with masked arrays

I need to calculate the chi-squared fit to a set of arrays (

observed
and
expected
). The arrays are the same size, but some of the elements of
expected
are
None
because I don't know the expected value.

I would like to use the
scipy.stats.chisquare
to calculate the chi-squared fit of the data. Since some of my data is
None
, I get the error:

TypeError: unsupported operand type(s) for -: 'float' and 'NoneType'


I've tried to get around this by only including the data that is not
None


import numpy.ma
import scipy.stats

expected = [None, 2,3,4]
observed = [1,2.2,3.3,4.4]

expMasked = numpy.ma.masked_where(expected == None, expected)
obsMasked = numpy.ma.masked_where(expected == None, observed)

chi2 = scipy.stats.chisquare(obsMasked, f_exp=expMasked)


This just give me the same error.

How can I use the
scipy.stats.chisquare
function with my data?

Answer

While it seems logical that numpy.ma.masked_where(expected == None, expected) would in fact mask expected where None occurs, numpy.ma does not recognize None as comparable to expected, so None must be cast to an np.array. Your code will work like this

import numpy as np
import scipy.stats

expected = [None, 2,3,4]
observed = [1,2.2,3.3,4.4]

expMasked = np.ma.masked_where(expected == np.array(None), expected)
obsMasked = np.ma.masked_where(observed == np.array(None), observed)

expMasked[expMasked.mask] = (VALUE YOU WANT MASKED VALUES TO BE REPLACED WITH)
obsMasked[obsMasked.mask] = (VALUE YOU WANT MASKED VALUES TO BE REPLACED WITH)

chi2 = scipy.stats.chisquare(np.array(obsMasked), f_exp=np.array(expMasked))
Comments