Sylar Sylar - 1 year ago 78
Ruby Question

Prevent answer from rounding

The title alone has many answers for different languages but I'm having a weird behavior. I get correct figures in the rails console but not so with Rails.


a = -42.57
b = -43.47
a - b = 0.8999999999999986

Rails' answer is
. This slight error makes a huge difference in my application.

answer = (a-b)
answer < 1 ? 'do something' : 'do other thing'

I need the exact calculation. Why it's not so in Rails controller but correct in the console? There's no database involved, just basic math in a method.


def step_3
a = 112.53 - 155
b = 112.53 - 156
a - b

Answer Source

This is not a Rails problem. You are using an inappropriate data type.

If you need precise results, you could do it like this:

require 'bigdecimal''-42.57')'43.47')
puts(c.to_s('F')) # prints -86.04
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download