Dhruv Ghulati Dhruv Ghulati - 1 year ago 95
Python Question

Deal with errors in parametrised sigmoid function in python

I am trying to convert a set of numbers into sigmoids:

actualarray = {
'open_cost_1':{
'cost_matrix': [
{'a': 24,'b': 56,'c': 78},
{'a': 3,'b': 98,'c':1711},
{'a': 121,'b': 12121,'c': 12989121},
]
},
'open_cost_2':{
'cost_matrix': [
{'a': 123,'b': 1312,'c': 1231},
{'a': 1011,'b': 1911,'c':911},
{'a': 1433,'b': 19829,'c': 1132},
]
}
}


Where each number in each list of dicts in each
cost_matrix
gets normalised by different sigmoid functions:

def apply_normalizations(costs):

def sigmoid(b,m,v):
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1 #Taken from http://web.stanford.edu/class/psych252/tutorials/Tutorial_LogisticRegression.html

def normalize_dicts_local_sigmoid(bias, slope,lst):
return [{key: sigmoid(bias, slope,val) for key,val in dic.iteritems()} for dic in lst]


for name, value in costs.items():
if int((name.split("_")[-1]))>1:
value['normalised_matrix_sigmoid'] = normalize_dicts_local_sigmoid(0,1,value['cost_matrix'])


apply_normalizations(actualarray)


However, when I run this, I get:

RuntimeWarning: overflow encountered in exp
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1
RuntimeWarning: invalid value encountered in double_scalars
return ((np.exp(b+m*v) / (1 + np.exp(b+m*v)))*2)-1


And the array becomes:

{
'open_cost_2': {
'cost_matrix': [
{
'a': 123,
'c': 1231,
'b': 1312
},
{
'a': 1011,
'c': 911,
'b': 1911
},
{
'a': 1433,
'c': 1132,
'b': 19829
}
],
'normalised_matrix_sigmoid': [
{
'a': 1.0,
'c': nan,
'b': nan
},
{
'a': nan,
'c': nan,
'b': nan
},
{
'a': nan,
'c': nan,
'b': nan
}
]
},
'open_cost_1': {
'cost_matrix': [
{
'a': 24,
'c': 78,
'b': 56
},
{
'a': 3,
'c': 1711,
'b': 98
},
{
'a': 121,
'c': 12989121,
'b': 12121
}
]
}
}


Note, every cost is always more than 0, hence I multiply by 2 and subtract 1 in my sigmoid function.

How can I adapt this to not have this error?

Answer Source

As the warning states, the exponential in your implementation of the sigmoid function is overflowing. When that happens, the function returns nan:

In [3]: sigmoid(1000, 1, 1)
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: overflow encountered in exp
  if __name__ == '__main__':
/Users/warren/miniconda3/bin/ipython:2: RuntimeWarning: invalid value encountered in double_scalars
  if __name__ == '__main__':
Out[3]: nan

Instead of writing your sigmoid function in terms of exp, you can use scipy.special.expit. It handles very large arguments correctly.

In [5]: from scipy.special import expit

In [6]: def mysigmoid(b, m, v):
   ...:     return expit(b + m*v)*2 - 1
   ...: 

In [7]: mysigmoid(1000, 1, 1)
Out[7]: 1.0

Check that it returns the same as your sigmoid function in cases where it doesn't overflow:

In [8]: sigmoid(1, 2, 3)
Out[8]: 0.99817789761119879

In [9]: mysigmoid(1, 2, 3)
Out[9]: 0.99817789761119879

See Numpy Pure Functions for performance, caching for my answer to another question about the sigmoid function.