High-precision scientific notation
Posted: Mon Apr 09, 2018 1:24 pm
(copied from the old forum)
LB 4 (including 4.5.1) performs floating-point arithmetic internally with a precision of approximately 15 to 16 significant figures, which should be good enough for most applications. But, strangely, if the calculation returns a value that must be represented in scientific (exponential) notation - because it is too large or too small for fixed-point format - LB will print the result only to a precision of 8 significant figures!
For example suppose you want to calculate 123456789^-20; LB knows what the answer is to an accuracy of at least 15 digits but the best you can print (easily) is:
This limitation arises because, uniquely in my experience, the LB 4 implementation of the USING() function does not give you the opportunity of specifying that scientific notation should be used. This omission is fixed in LBB:
which gives:
(all the digits are significant).
Here's a program for printing all the powers from +42 to -42 (only a fraction of the range that LBB is capable of):
Richard.
LB 4 (including 4.5.1) performs floating-point arithmetic internally with a precision of approximately 15 to 16 significant figures, which should be good enough for most applications. But, strangely, if the calculation returns a value that must be represented in scientific (exponential) notation - because it is too large or too small for fixed-point format - LB will print the result only to a precision of 8 significant figures!
For example suppose you want to calculate 123456789^-20; LB knows what the answer is to an accuracy of at least 15 digits but the best you can print (easily) is:
Code: Select all
0.14780886e-161
Code: Select all
print using ("###.###################^^^^", 123456789^-20)
Code: Select all
1.4780885631555544717E-162
Here's a program for printing all the powers from +42 to -42 (only a fraction of the range that LBB is capable of):
Code: Select all
a = 123456789
for n = 42 to -42 step -1
b = a^n
print n;tab (10); using ("###.###################^^^^",b)
next n
end