C5H8NNaO4 C5H8NNaO4 - 2 days ago 4x
Javascript Question

Compute the double value nearest preferred decimal result

Let N(x) be the value of the decimal numeral with the fewest significant digits
such that x is the

value nearest the value of the numeral.

values a and b, how can we compute the
value nearest N(b)-N(a)?


  • If a and b are the
    values nearest .2 and .3,

    • the desired result is the
      value nearest .1,

      • 0.1000000000000000055511151231257827021181583404541015625,

    • rather than than the result of directly subtracting a and b,

      • 0.09999999999999997779553950749686919152736663818359375.


As a baseline: In Java, the Double.toString() provides the N(x) function described in the question, returning its value as a numeral. One could take the strings for a and b, subtract them with the elementary-school method, and convert the resulting string to double.

This demonstrates solving the problem is quite feasible using existing library routines. This leaves the task of improving the solution. I suggest exploring:

  • Is there a function D(x) that returns the number of significant digits after the decimal place for the numeral described in N(x)? If so, can we multiply a and b by a power of ten determined by D(a) and D(b), round as necessary to produce the correct integer results (for situations where they are representable as double values), subtract them, and divide by the power of ten?
  • Can we establish criteria for which b-a or some simple expression can be quickly rounded to something near a decimal numeral, bypassing the code that would be necessary for harder cases? E.g., could we prove that for numbers within a certain range, (round(10000*b)-round(10000*a))/10000 always produces the desired result?