tl;dr: 1.2000000000000002 *is* the correct answer for 1.19 roundTo: 0.1. (1.19 roundTo: (1/10))asFloat gives a different answer because it's rounding to a multiple of exactly 0.1, not the Float nearest to 0.1. On 10/31/2016 08:35 AM, test wrote:
oops, make that original number 1.19 in the example.
OK, so the examples are now 1.19 round: 1. 1.19 roundTo: 0.1. (1.19 roundTo: (1/10))asFloat I should note that the final example *should* result in a Float without having to have #asFloat sent to it. Per ANSI and traditional Smalltalk practice, any operation where the receiver *or* the argument is a Float produces a Float. Since 0.1 is not quite equal to 1/10, the correct answer may not be the same for the latter two examples. Whether you define #round: to be one or the other is an interesting question. I've learned the hard way not to trust the results from Smalltalk math libraries too far, so let's manually figure out what the correct answer is for each of these, and see if Pharo's doing it right. To start, every Float (ignoring infinities, NaNs, and subnormals) is (by definition) a Fraction constrained to have a numerator in the range [2**52..2**53) and a denominator that is a power of two. Pharo says: 0.1 asFraction ==> (3602879701896397/36028797018963968) Is this correct? The numerator is less than 2**52, so this Fraction has been reduced. Let's restore it to what it would be in its Float form by multiplying numerator and denominator by, in this case, 2: 7205759403792794 ----------------- 72057594037927936 It's easy to see that this fraction is very nearly equal to 1/10, and to see that there could be no Float closer to 1/10. --- But I haven't shown the conversion from the string '0.1' to the actual Float instance is correct. It's possible that there's some rounding error there, which is reversed by a rounding in a different direction by the #asFraction. The Float for '0.1' is two words with values, 1069128089 and 2576980378. Convert to hex and combine them, you get 16r3FB999999999999A The numerator of our fraction is the low-order 52 bits of this, with a 1 added on the most-significant end, so 16r1999999999999A, which in decimal is 7205759403792794. Which is what we got above, so we're good so far. ======== Doing the same analysis on 1.19 gives a numerator of 16r130A3D70A3D70A, and a denominator 1/16th the previous one, or 5359283556570890 ---------------- 4503599627370496 This one's harder, I can't just look at it and say it's the closest possible to 1.19. But we know that 1.19 = 119/100. 119/100 * 4503599627370496/4503599627370496 = 535928355657089024 ------------------ 450359962737049600 and is still exactly 1.19. But to make a representable Float we have to divide numerator and denominator by 100, rounding the numerator if necessary. And we can see that the numerator is correctly rounded, so Pharo also converts the string '1.19' into a Float correctly. How does Pharo do on 1.19 asFraction? (2679641778285445/2251799813685248) Once again, this has been reduced by dividing numerator and denominator by two, but it's numerically correct. ====== Now that we know that we have the correct Fractions for the floats, we can check the rounding functions. 1.19 roundTo: 0.1 should ideally be equivalent to (1.19 asFraction roundTo: 0.1 asFraction) asFloat. In Pharo 5, 1.19 asFraction roundTo: 0.1 asFraction ==> (10808639105689191/9007199254740992) For this to correct, (10808639105689191/9007199254740992) must be a multiple of (3602879701896397/36028797018963968), since that's 0.1 asFraction. Is it a multiple? We can align the denominators by multiplying (10808639105689191/9007199254740992) by 4/4, and we get 43234556422756764 ----------------- 36028797018963968 which would be a multiple of 3602879701896397 ----------------- 36028797018963968 iff 43234556422756764 is a multiple of 3602879701896397. Looking at the original problem (1.19 roundTo: 0.1) we'd expect the multiple to be 12. Sure enough, 3602879701896397 * 12 ==> 43234556422756764. So we're correct so far. Now we have to convert our answer to the nearest representable Float. Our denominator is already a power of two, but our numerator is out of range -- it's too large. It's even, so we can cut it in half, getting 21617278211378382 Still too large, still even. Cut it in half again: 10808639105689191 Just a bit too large. Cut it in half again: 5404319552844595.5 Now it's in range [2**52..2**53) but it's not an integer, so we must round. Our number is exactly halfway between two representable numbers, so by IEEE754 we round to the even one, which is 5404319552844596 The Float we get back from 1.19 roundTo: 0.1 is 16r3FF3333333333334 and the numerator part of that is 16r13333333333334, or 5404319552844596 Which agrees with what we computed before. So this *is* the correct Float result. And it prints as 1.2000000000000002. This print string implies that there is a Float nearer to 1.2 than this. 1.2 is represented as 16r3FF3333333333333, so one representable Float less than the result of the #roundTo:. But, as we've shown, 1.2000000000000002 is *correct*. (Which was not what I expected when started this -- I suspected there might be rounding error in the computation of #roundTo:. But for this example, there isn't). I haven't done the full analysis of it, but I expect that (1.19 roundTo: (1/10))asFloat gives the Float with representation 16r3FF3333333333333 because it's rounding to a multiple of exactly 0.1, not the Float nearest to 0.1. Regards, -Martin