"ajp166" <ajp166_at_bellatlantic.net> wrote:
> come to mind. This led to things like NS* basic and HP using BCD to
> avoid
> the SQRT(16)=3.99999!
Hi
When I was in school we proved that 0.99999... was equal
to 1.00000... So, the number 3.99999 is enough equal to
4.00000 that I would not consider it a problem. The number
0.00001 is within a reasonable range of error for the size
numbers being displayed.
The first thing to realize is that all rounding and truncation
are ERRORS! Still, if we'd been using a binary
system all along, the banks would insist that we use binary
and not BCD in our calculations. The reason banks like BCD
is that it can specify the rules for rounding that will work
the same on every computer.
It is interesting that even how we normally do rounding can
cause serious problems ( decimal or binary ). The method of
rounding towards zero can cause all kinds of problems in control
systems ( I've seen this one in action ).
Dwight
Received on Mon Apr 23 2001 - 12:00:27 BST