Im doing a Neural network task with
As discussed in comments section, the real problem turned out to be using
sigmoid itself, which is not suited for such cases. In any finite-precision calculations one will face the described problem, one system with 29, on other with 38.
One way to tackle the problem would be use
softmax activation function, which is less susceptible to such issues. Mind that with cost function you might encounter similar challenges.
Slightly off-topic, ut you might want to check how the problem is resolved with e.g. tensorflow. It has some nice tutorials for beginners.