lya lya - 1 month ago 12
C# Question

Inconsistent Convert.ToDouble(decimal) results

I upgraded a calculation program and found the results changed unexpectedly.
I've located the problem, but have no idea how this can happen or how to solve it.

The result difference is caused by this statement

Convert.ToDouble(0.0000000035141287671294348266M)


When I open two Visual Studio Community 2015 instances to run both the new and the old program, set a break point before the statement and evaluate it in the Immediate Window, I can get two results.
One is

3.514128767129435E-09


and the other is

3.5141287671294345E-09


It seems the former is correct (4348266 rounded to 435), but if I break the former program immediately after it is started and evaluate again, the result becomes the latter.

It's not a huge difference but the behavior scares me. Which one should be the correct result? How can I avoid the problem?

Answer

The problem lies within what the guranteed precision of a double actually is.

According to MSDN it is 15-16 digits, and regarding your example, the difference is only visible at the 16th digit.

Comments