Sylar Sylar - 25 days ago 8
Ruby Question

Prevent answer from rounding

The title alone has many answers for different languages but I'm having a weird behavior. I get correct figures in the rails console but not so with Rails.

Console:

a = -42.57
b = -43.47
a - b = 0.8999999999999986


Rails' answer is
1.0
. This slight error makes a huge difference in my application.

answer = (a-b)
answer < 1 ? 'do something' : 'do other thing'


I need the exact calculation. Why it's not so in Rails controller but correct in the console? There's no database involved, just basic math in a method.

Controller

def step_3
a = 112.53 - 155
b = 112.53 - 156
a - b
end

Answer

This is not a Rails problem. You are using an inappropriate data type.

If you need precise results, you could do it like this:

require 'bigdecimal'

a=BigDecimal.new('-42.57')
b=BigDecimal.new('43.47')
c=a-b
puts(c.to_s('F')) # prints -86.04