Code Complete Code Complete - 3 years ago 181
Java Question

What is precision of double/float comparison (relation operator) with integral types?

What is precision of double/float comparison (relation operator) with integral types?

I understand that it is not recommended to use floating-point in any comparisons. But so far that this operation is allowed, here is the question:

int x = 90;
float y = 90.00_001f;

if (x < y) {
System.out.println("it works!"); // gets printed
}

// now just add one zero to lessen the precision a bit...

int x = 90;
float y = 90.000_001f;

if (x < y) {
System.out.println("it works!"); // not printed
}

Answer Source

The precision of a IEEE 754 single-precision floating point is between 6 and 9 significant decimal digits (source).

For double-precision you get between 15 and 17 significant decimal digits.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download