pizzafilms pizzafilms - 4 months ago 37
C++ Question

C++: Strange floating point math results

I've been struggling to find a crazy bug in some C++ code and narrowed it down to this small section. I placed into a simple main.c to debug it and can't figure out why the floating point math is rounding when it shouldn't.

// setup the variables for this simple case
int writep = 672;
float offset = 672.000122;
int bufferSize = 2400;
float bufferSizeF = (float)bufferSize;

float outPointer = (float)writep - offset; // outPointer: -0.000122070313
if(outPointer < 0.0f){
printf("outPointer: %.9f \n", outPointer); // outPointer: -0.000122070313
outPointer += bufferSizeF; // outPointer SHOULD be: 2399.9998779296875
printf("outpointer: %.9f \n", outPointer); // outPointer: 2400.000000000

Someone...please explain. Thanks.


2400.000000000 and 2399.9998779296875 are too close for a standard float to differentiate them. Try this:

int main() {
    std::cout << (float)2399.9998779296875 << "\n";

It will probably give 2400 as output.

An IEEE 754 single precision float can only hold about 7 to 8 significant decimal digits. If you need a higher number of significant digits use a double precision double.