BCD accuracy (was: HP 85/Rocky Mountain BASIC replacement)

From: Jim Battle <frustum_at_pacbell.net>
Date: Sat Apr 21 02:37:49 2001

At 12:47 AM 4/21/01 -0400, Allison Parent wrote:
>From: Jim Battle <frustum_at_pacbell.net>
> >Unless you are doing financial work where the fractional numbers tend to be
> >inherently decimal, BCD arithmetic, for a given number of bytes of storage,
> >is less accurate than binary. As a BCD byte can represent only 100 states
> >vs 256 for binary, you are going to lose more than one bit of accuracy per
>
>Sinppage....
>
>Never confuse accuracy with resolution or range.

Yes, they are three different things. Range is not in question here (which
is determined mostly by the exponent and base of the exponent). So we are
left with accuracy and resolution (back in high school I was taught the
terms accuracy and precision.). I don't know how your (accurate) statement
clarifies the matter at hand. I claim that a floating point representation
specifies precision and has nothing to say about accuracy (other than
accuracy is limited by precision).

To clarify, say a number is 175.136985. This represents my weight very
precisely, but whether it is accurate or not is a different matter.

>Most BCD systems were less prone to truncation, rounding and other
>cumulative errors within their range. Binary for the number of bits gave
>more resolution but sometimes at the expense of accuracy.

Again, I must be missing something, or perhaps it is a definitional
thing. I've given my definition for precision (or what you call
resolution) and accuracy. I claim accuracy is largely outside the scope of
a number representation. Perhaps you and others have a different
definition than I do and are also internally consistent. I'd like to hear
it, since I keep running into it and don't understand it.

As for truncation, this is exactly the example I give below. 0.1 (decimal)
is a non-repeating fraction in binary, and so might show up as
0.09999999999999 when converted back to a decimal output string. It is
very conveniently and exactly representable in BCD and so won't suffer this
type of problem. This is kind of unfair, because it just so happens that
this example has chosen a number that matches the base. Also, although it
might look wickedly different from 0.1, the error is only 10^-14.

If, on the other hand, you want to know what is sin(0.1), I claim the
binary representation is more precise (and thus allows more accuracy;
accuracy here depends on the implementation of the trig algorithms and not
inherently in the number system).

The counter example is that 0.11111 (base 16) [that is, 2^-4 + 2^-8 + 2^-12
+ 2^-16 + 2^-20] is exactly representable in a binary number system, but
not in an 8B BCD number representation because there isn't enough precision
to hold all the digits.

> >So, OK, 0.1 (base 10) can't be exactly represented in a binary format, but
> >0.11111 (base 16) can't be represented exactly in an 8B BCD representation.
>
>If you meant .1 and got .11111 that would be a significant error! same for
>say meaning .1 and getting .09999.

nope, I didn't mean that at all.

maybe I should give up and take this to the alt.ieee.vs.bcd.floating-point
newsgroup. :-)


-----
Jim Battle == frustum_at_pacbell.net
Received on Sat Apr 21 2001 - 02:37:49 BST