LibreOffice Module tools (master) 1
|
Inaccurate cancellation for a fraction.
Inaccurate cancellation for a fraction.Clip both nominator and denominator to said number of bits. If either of those already have equal or less number of bits used, this method does nothing.
nSignificantBits | denotes, how many significant binary digits to maintain, in both nominator and denominator. |
(8) has an error <1% [1/2^(8-1)] - the largest error occurs with the following pair of values:
binary 1000000011111111111111111111111b/1000000000000000000000000000000b = 1082130431/1073741824 = approx. 1.007812499
A ReduceInaccurate(8) yields 1/1.