C5H8NNaO4 - 3 months ago 11

Javascript Question

Let N(x) be the value of the decimal numeral with the fewest significant digits

such that x is the

`double`

Given

`double`

`double`

- If a and b are the values nearest .2 and .3,
`double`

- the desired result is the value nearest .1,
`double`

- 0.1000000000000000055511151231257827021181583404541015625,

- rather than than the result of directly subtracting a and b,

- 0.09999999999999997779553950749686919152736663818359375.

- the desired result is the

Answer

As a baseline: In Java, the `Double.toString()`

provides the N(x) function described in the question, returning its value as a numeral. One could take the strings for a and b, subtract them with the elementary-school method, and convert the resulting string to `double`

.

This demonstrates solving the problem is quite feasible using existing library routines. This leaves the task of improving the solution. I suggest exploring:

- Is there a function D(x) that returns the number of significant digits after the decimal place for the numeral described in N(x)? If so, can we multiply a and b by a power of ten determined by D(a) and D(b), round as necessary to produce the correct integer results (for situations where they are representable as
`double`

values), subtract them, and divide by the power of ten? - Can we establish criteria for which
`b-a`

or some simple expression can be quickly rounded to something near a decimal numeral, bypassing the code that would be necessary for harder cases? E.g., could we prove that for numbers within a certain range,`(round(10000*b)-round(10000*a))/10000`

always produces the desired result?