Calculation discrepancies

[Dermot]

I have a POS program written in LBB, and I have the problem where calculations (very rarely) do not appear to be exact.
I show below the sort of thing that happens, and I have used the 'notice' command to display the values

For example, value1 and value2 are the result of calculations involving currency (2 decimals)

the code:
notice str$(value1) + "-" + str$(value2)

results in the following display:

70.52-70.52

as expected, and they appear to be the same, but the code:

if value1 <> value2 then notice str$(value1 - value2)

resuts in the following display:

-6.9388939E-18

ie they look the same but they are not exactly the same, causing the program to take the incorrect course of action

Why does this happen, and how can one prevent it happening?

Dermot

[guest]

Why does this happen,

I am surprised you should ask, because I thought (by now) this characteristic - of virtually every programming language - was well known. But if you have not encountered the issue before, it happens because numbers are stored internally as floating-point binary, not as decimal. So a number like '70.52', which has a precise value in decimal, does not have a precise value in binary. In fact, should you be interested, the closest LBB can get to that value is:

70.519999999999999996946886682280819513835012912750244140625

That's pretty damn close, but it's not exact. LB 4 has an even greater error, because it works internally with a lower precision than LBB. The closest LB 4 can get to 70.52 is:

70.5199999999999960209606797434389591217041015625

and how can one prevent it happening?

To avoid it happening you should do all your calculations in integers, which are precise. So for example when working with currency values (such as pounds and pence or dollars and cents) you should do all your calculations in integer numbers of pence or cents, and then when you want to display the amount in pounds or dollars you simply divide by 100. Since you're converting to a non-integer value purely for the purposes of display, errors cannot accumulate.

Richard.

[bencooper]

When searching for an integral solution to a mathematical procedure (for example, a brain teaser), it's not sufficient to use the algorithm
IF (NUM-INT(NUM))= 0 THEN .......,
because if the calculated NUM for an integer NUM includes rounding errors (say 2.000000000001 instead of integer 2) then the comparison fails.

Usually, a slight modification to the above algorithm such as
IF ABS(NUM-INT(NUM))<1E-6 THEN ..... traps the integer while avoiding rounding errors (the ABS function accepts positive
and negative rounding alike).

[guest]

Usually, a slight modification to the above algorithm...

As you say, "usually". But it's risky, because you are not determining precisely whether a value is an integer but only that it is very close to an integer, and that might not always be good enough. It's far better, when possible, to devise a method that uses only integer calculations and avoids converting to a floating-point value at all; that will maintain 'exact' precision throughout the entire process.

Liberty BASIC and LBB are quite well suited to this kind of calculation, because they both support a wider range of integers than 'typical' BASICs do. LB4 supports 'arbitrary precision' integer arithmetic so you can use values as large as you like with no loss of precision. LBB doesn't, but its integer range (−18,446,744,073,709,551,615 to +18,446,744,073,709,551,615) is still very large.