Re: [SLUG] Math Precision

From: Christopher Hotchkiss (christopher.hotchkiss@gmail.com)
Date: Sun Feb 06 2005 - 08:18:14 EST


On Sun, 06 Feb 2005 06:24:45 -0500, Kwan Lowe <kwan@digitalhermit.com> wrote:
> Paul M Foster wrote:
> > Programmers:
> >
> > I'm using SQLite for some payroll database work, and seeing a lot of
> > rounding errors, or what are really errors caused by trying to store
> > decimals as binaries. Things like 2.45 being stored and then getting
> > 2.4500000001 back out. I'd like an uncomplicated way of avoiding this,
> > _and_ avoiding math/rounding errors when dealing with decimals. I asked
> > this question a long time ago, and it got answered, but for some reason
> > I didn't keep the replies. I can't seem to find the right terms to
> > google for exactly what I want. I'm not doing fibonaccis or cosines.
> > Just simple add, subtract, multiply and divide, and I need the results
> > to be exact (it's payroll, folks).
> >
> > Any help?
> >
> > Paul
>
> You'd probably want to use some for of binary coded decimal. It stores
> the exact number rather than a decimal approximation.
Or for the lazy, stick it in as a VARCHAR.

-- 
Christopher Hotchkiss
(813)960-9273
http://www.post227.org
-----------------------------------------------------------------------
This list is provided as an unmoderated internet service by Networked
Knowledge Systems (NKS).  Views and opinions expressed in messages
posted are those of the author and do not necessarily reflect the
official policy or position of NKS or any of its employees.



This archive was generated by hypermail 2.1.3 : Fri Aug 01 2014 - 18:39:30 EDT