janlindso janlindso - 2 months ago 9
Android Question

Calculation gives different results

I'm trying to calculate a formula in Android:

percent = absence/(week*(37/period))*100;


The value for
absence
is
4
, week is
2
, and period is
2
.
percent
returns
11.111111
.

But if I do the same in Excel with formula
4/(2*(37/2))*100
, it returns
10.81
.

Can anyone explain me what might be wrong?

Answer

It's because of integer division.

The value for absence is 4, week is 2, and period is 2. percent returns 11.111111.

This is because 37 / 2 is 18.5, and the fractional part is thrown out because of integer division. That gives 18, which multiplied by 2 is 36. Next we divide 4 by 36, which is 0.111... That multiplied by 100 is 11.111....

But if I do the same in Excel with formula 4/(2*(37/2))*100, it returns 10.81.

This is because Excel doesn't use integer division. 37 / 2 is 18.5, 18.5 * 2 is 37. 4 / 37 is 0.108108108... Thus multiplying by 100 gives approximately 10.81.


Explanation

The reason it's doing integer division is because you have an integer period. Java sees that there are two integers and are being divided so it uses integer division. This causes all fractional parts to be erased, meaning the 0.5 from 18.5 resulting from 37 / 2 is removed, making the results different.

Fix

You can 'fix' this by making at least one of the operands in the division float or double:

double percent = absence/(week*((double) 37/period))*100;

What this will do is achieve floating-point division giving 10.8108...

Comments