Flip - 1 year ago 56
Ruby Question

# Why is Ruby adding precision/digits when calculating a mod?

I need to make some calculations with coordinates and came across
this (at least for me) weird behaviour.
Can someone explain why this happens?

``````\$ long
=> 49.0126760222489
\$ long % long.floor
=> 0.012676022248896857
``````

I was expecting the last line to evaluate to just the digits after the period, from
`long`
`6857`
tacked onto the end.

Where is the information coming from?

I suppose we simply see float's imprecions. See i.e. `0.0126760222489 % 1.0` and `1.0126760222489 % 1.0`. You'd think the result should be the same, but no - IEEE754 floats/doubles don't guarantee perfect results, and by default these are used in Ruby for storing floating point values.

It's even somewhat shown in the docs

``````6543.21.modulo(137)      #=> 104.21
6543.21.modulo(137.24)   #=> 92.9299999999996
``````

you can see that the second result has a small error. Actually, on Ruby 2.3.1 I ran the first line and got:

``````pry(main)> 6543.21.modulo(137)
=> 104.21000000000004
``````

It's not necessarily related to modulo, and not always visible:

``````[30] pry(main)> 10.0126760222489 - 0.0
=> 10.0126760222489
[31] pry(main)> 10.0126760222489 - 1.0
=> 9.0126760222489
[32] pry(main)> 10.0126760222489 - 2.0
=> 8.0126760222489
[33] pry(main)> 10.0126760222489 - 3.0
=> 7.0126760222489
[34] pry(main)> 10.0126760222489 - 4.0
=> 6.0126760222489
[35] pry(main)> 10.0126760222489 - 5.0
=> 5.0126760222489
[36] pry(main)> 10.0126760222489 - 6.0
=> 4.0126760222489
[37] pry(main)> 10.0126760222489 - 7.0
=> 3.0126760222489004
``````

Every piece of software that uses standard floats needs to account for these small errors. If you cannot handle that for some reason, then you could use `bigdecimal` (should be already included in your Ruby), fixed-point, or some similar numeric library

``````require 'bigdecimal'

BigDecimal('6543.21').modulo(137).to_s
=> "0.10421E3"

BigDecimal('6543.21').modulo(137).to_f
=> 104.21
``````

Keep in mind that 'bigdecimal' may be slower and may use more memory.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download