[SLUG] Math Precision

From: Paul M Foster (paulf@quillandmouse.com)
Date: Sun Feb 06 2005 - 03:11:23 EST


Programmers:

I'm using SQLite for some payroll database work, and seeing a lot of
rounding errors, or what are really errors caused by trying to store
decimals as binaries. Things like 2.45 being stored and then getting
2.4500000001 back out. I'd like an uncomplicated way of avoiding this,
_and_ avoiding math/rounding errors when dealing with decimals. I asked
this question a long time ago, and it got answered, but for some reason
I didn't keep the replies. I can't seem to find the right terms to
google for exactly what I want. I'm not doing fibonaccis or cosines.
Just simple add, subtract, multiply and divide, and I need the results
to be exact (it's payroll, folks).

Any help?

Paul

-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS). Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 18:38:49 EDT