Fcoder - 1 month ago 21

Python Question

Im doing a Neural network task with

`Sigmoid`

`MNIST`

`28*28`

`N*784`

`Sigmoid`

`expit()`

numbers until 30 resulting near 1 in

`expit()`

`expit(28)`

`0.99999999`

`expit(29)`

`1.0`

`29`

`1`

`30`

`1`

What i have to do?

`Sigmoid's`

`29`

Answer

As discussed in comments section, the real problem turned out to be using `sigmoid`

itself, which is not suited for such cases. In any finite-precision calculations one will face the described problem, one system with 29, on other with 38.

One way to tackle the problem would be use `softmax`

activation function, which is less susceptible to such issues. Mind that with cost function you might encounter similar challenges.

Slightly off-topic, ut you might want to check how the problem is resolved with e.g. tensorflow. It has some nice tutorials for beginners.