On Mon, Oct 31, 2016 at 11:10 PM, test <wkassens@libello.com> wrote:
Hi, just a simple numerical example using nicolas' implementation: 1.199999999999999999 round: 1. "1.2" i'd guess this result is more or less what a beginner would expect with this line. how would he do it with roundTo:? perhaps this way? 1.199999999999999999 roundTo: 0.1. "1.2000000000000002" i'm not so sure that he would immediately see that he'd need to do it this way: (1.199999999999999999 roundTo: (1/10))asFloat. "1.2" werner
On Tue, Nov 1, 2016 at 4:58 AM, Martin McClure <martin@hand2mouse.com> wrote:
tl;dr:
1.2000000000000002 *is* the correct answer for 1.19 roundTo: 0.1. (1.19 roundTo: (1/10))asFloat gives a different answer because it's rounding to a multiple of exactly 0.1, not the Float nearest to 0.1.
So then why don't we *really* help the naive programmer make "x roundTo: 0.1" an error? Conceptually... Number>>roundTo: quantum quantum < 1 ifTrue: [ quantum isFraction ifFalse: [ SubtleArithmeticError signal ]. ^(self / quantum) rounded * quantum SubtleArithmeticError >> defaultAction self inform: 'Decimals have an inexact Float representation, so rounding may not give you what you expect. Us Fractions instead.'. cheers -ben