keflavich keflavich - 13 days ago 5
Python Question

numpy FloatingPointError: invalid value encountered in subtract - not reproducible?

I'm encountering

FloatingPointError: invalid value encountered in subtract
in a piece of test code. The exception started being raised without any changes being made in the code itself, so I'm having a great deal of trouble understanding it.

My question: What causes the
invalid value encountered in subtract
exception? Why would it behave differently on different installs of python+numpy?

DETAILS:

This MWE does not raise a
FloatingPointError
:

>>> import numpy as np
>>> np.__version__
'1.6.1'
>>> x = np.arange(5,dtype='float64')
>>> y = np.ones(5,dtype='float64')
>>> x[2]=np.nan
>>> x-y
# array([ -1., 0., nan, 2., 3.])


However, deep within a piece of code, I subtract two
np.float64
ndarray
objects, and get a floating point exception. The arrays causing the exception contain some pretty enormous and tiny numbers (e.g., 1e307 and 1e-307) and some
nan
s, but I haven't made any combination of these numbers result in an exception testing on my own.

Much more disturbingly, I have a large grid of Jenkins tests running the exact same code with many versions of numpy, matplotlib, python, and scipy, and NONE of them raise this exception. I'm lost at this point - I don't know if there is a bug, or if there is, how to track it down.

In case you're morbidly curious, the code in question is pyspeckit and the test is failing on line 20 of
test_hr2421.py
.

EDIT: Follow-up - I think this little snippet:
np.seterr(invalid='raise')
was being called in a module I was importing, specifically pymc, and a pull request has since prevented this change from being made.

Answer

Numpy has configurable behaviour as to how errors are treated. By default some errors are ignored, others cause a warning. For each category you can change this behaviour. Someone must have set it to raising errors, without changing it back.

You can suppress this exception by calling numpy.seterr(invalid='warn'), or, alternatively, invalid='ignore'. For a full list of possible errors, read through the documentation of numpy.seterr.

You can also use a context-manager:

In [12]: x = np.arange(-5, 5,dtype='float64')

In [13]: with np.errstate(divide="raise"):
    print(1/x)
   ....:     
---------------------------------------------------------------------------
FloatingPointError                        Traceback (most recent call last)
<ipython-input-13-881589fdcb7a> in <module>()
      1 with np.errstate(divide="raise"):
----> 2     print(1/x)
      3 

FloatingPointError: divide by zero encountered in true_divide

In [14]: with np.errstate(divide="warn"):
    print(1/x)
   ....:     
/home/users/gholl/venv/stable-3.5/bin/ipython3:2: RuntimeWarning: divide by zero encountered in true_divide

[-0.2        -0.25       -0.33333333 -0.5        -1.                 inf
  1.          0.5         0.33333333  0.25      ]

In [15]: with np.errstate(divide="ignore"):
    print(1/x)
   ....:     
[-0.2        -0.25       -0.33333333 -0.5        -1.                 inf
  1.          0.5         0.33333333  0.25      ]

I tend to wrap my entire code inside a with np.errstate(all="raise") block, and then use a context-manager ignoring a particular condition if I am sure that the problem is not hiding a bug — it usually is, though.

Comments