Dhruv Ghulati - 3 months ago 17
Python Question

# Deal with errors in parametrised sigmoid function in python

I am trying to convert a set of numbers into sigmoids:

``````actualarray = {
'open_cost_1':{
'cost_matrix': [
{'a': 24,'b': 56,'c': 78},
{'a': 3,'b': 98,'c':1711},
{'a': 121,'b': 12121,'c': 12989121},
]
},
'open_cost_2':{
'cost_matrix': [
{'a': 123,'b': 1312,'c': 1231},
{'a': 1011,'b': 1911,'c':911},
{'a': 1433,'b': 19829,'c': 1132},
]
}
}
``````

Where each number in each list of dicts in each
`cost_matrix`
gets normalised by different sigmoid functions:

``````def apply_normalizations(costs):

def sigmoid(b,m,v):
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1 #Taken from http://web.stanford.edu/class/psych252/tutorials/Tutorial_LogisticRegression.html

def normalize_dicts_local_sigmoid(bias, slope,lst):
return [{key: sigmoid(bias, slope,val) for key,val in dic.iteritems()} for dic in lst]

for name, value in costs.items():
if int((name.split("_")[-1]))>1:
value['normalised_matrix_sigmoid'] = normalize_dicts_local_sigmoid(0,1,value['cost_matrix'])

apply_normalizations(actualarray)
``````

However, when I run this, I get:

`````` RuntimeWarning: overflow encountered in exp
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1
RuntimeWarning: invalid value encountered in double_scalars
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1
``````

And the array becomes:

``````{
'open_cost_2': {
'cost_matrix': [
{
'a': 123,
'c': 1231,
'b': 1312
},
{
'a': 1011,
'c': 911,
'b': 1911
},
{
'a': 1433,
'c': 1132,
'b': 19829
}
],
'normalised_matrix_sigmoid': [
{
'a': 1.0,
'c': nan,
'b': nan
},
{
'a': nan,
'c': nan,
'b': nan
},
{
'a': nan,
'c': nan,
'b': nan
}
]
},
'open_cost_1': {
'cost_matrix': [
{
'a': 24,
'c': 78,
'b': 56
},
{
'a': 3,
'c': 1711,
'b': 98
},
{
'a': 121,
'c': 12989121,
'b': 12121
}
]
}
}
``````

Note, every cost is always more than 0, hence I multiply by 2 and subtract 1 in my sigmoid function.

How can I adapt this to not have this error?

As the warning states, the exponential in your implementation of the sigmoid function is overflowing. When that happens, the function returns `nan`:

``````In [3]: sigmoid(1000, 1, 1)
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: overflow encountered in exp
if __name__ == '__main__':
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: invalid value encountered in double_scalars
if __name__ == '__main__':
Out[3]: nan
``````

Instead of writing your sigmoid function in terms of `exp`, you can use `scipy.special.expit`. It handles very large arguments correctly.

``````In [5]: from scipy.special import expit

In [6]: def mysigmoid(b, m, v):
...:     return expit(b + m*v)*2 - 1
...:

In [7]: mysigmoid(1000, 1, 1)
Out[7]: 1.0
``````

Check that it returns the same as your `sigmoid` function in cases where it doesn't overflow:

``````In [8]: sigmoid(1, 2, 3)
Out[8]: 0.99817789761119879

In [9]: mysigmoid(1, 2, 3)
Out[9]: 0.99817789761119879
``````

See Numpy Pure Functions for performance, caching for my answer to another question about the sigmoid function.