Ah, yes, yes, sending #asScaledDecimal to a Float is too late, the precision is already lost. It makes total sense now. Thank you, Werner, for the explanation.
On 13 Jan 2020, at 16:13, werner kassens <wkassens@libello.com> wrote:
Hi Sven,
if you dont transform floats, that are not always exactly what they seem to be, you get what you would expect: 0.09560268s8 - 0.005s8 = 0.09060268s8. "true" "of course" a:=0.09560268 asScaledDecimal. 0.09560268s8=a . "false" "you could eg do this:" b:=a roundTo: 0.00000001s8. "then" 0.09560268s8=b . "true"
but if you do something like this generally in Float>>asScaledDecimal you could loose some precision that could be unwanted in some other circumstances (or perhaps not).
werner
On 1/13/20 2:13 PM, Sven Van Caekenberghe wrote:
0.09560268 asScaledDecimal - 0.005 asScaledDecimal = 0.09060268s8. "false"