Rounding in Floats
Hi, Continuing with my issues with decimals, I'm having one issue that is not clear to me why it happens. If I do: (9.1 + (-2.0)) roundTo: 0.1. "7.1000000000000005" I expect to get a single decimal Float (rounded with whatever precision, but a single decimal). Even if I do something like this: 7.1 roundTo: 0.1 It gives the wrong result. In VW and VAST it provides the right result. (9.1 + (-2.0)) roundTo: 0.1 "7.1" In Dolphin it also returns the wrong result, it seems to use the same algorithm to round it. Is this a bug? Esteban A. Maringolo
On 6 Sep 2020, at 16:06, Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
Continuing with my issues with decimals, I'm having one issue that is not clear to me why it happens.
If I do: (9.1 + (-2.0)) roundTo: 0.1. "7.1000000000000005"
I expect to get a single decimal Float (rounded with whatever precision, but a single decimal).
Even if I do something like this: 7.1 roundTo: 0.1
It gives the wrong result.
In VW and VAST it provides the right result. (9.1 + (-2.0)) roundTo: 0.1 "7.1"
In Dolphin it also returns the wrong result, it seems to use the same algorithm to round it.
Is this a bug?
Maybe. But I would not approach the problem of rounding like that. You probably want to control how numbers are printed. I would keep the numbers themselves at maximum internal precision and only do something when printing them. 1 / 3 asFloat printShowingDecimalPlaces: 1. Since you like Seaside, the following is even much more powerful (has *many* options): GRNumberPrinter new precision: 1; print: 1/3 asFloat. Check it out.
Esteban A. Maringolo
Hi Sven, On Sun, Sep 6, 2020 at 11:57 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 6 Sep 2020, at 16:06, Esteban Maringolo <emaringolo@gmail.com> wrote:
(9.1 + (-2.0)) roundTo: 0.1. "7.1000000000000005"
Is this a bug?
Maybe.
But I would not approach the problem of rounding like that. You probably want to control how numbers are printed.
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result. And then it fails because of the difference above when it is "semantically" correct.
I would keep the numbers themselves at maximum internal precision and only do something when printing them.
I do. I'm implementing the full WHS specification [1], and it mentions that most calculations should preserve the maximum internal precision and even introduces some weird situation where you have an "index" that is rounded and another one that is called the same way but must not be rounded because is going to used to compute some other value. For most calculations it specifies it must round to 1 decimal or even to no decimal, based on a round up if the decimal is greater or equal than zero (e.g. 1.5 becomes 2), except for negative numbers, in which case it rounds "down" (-1.5 becomes 1).
1 / 3 asFloat printShowingDecimalPlaces: 1.
Since you like Seaside, the following is even much more powerful (has *many* options): GRNumberPrinter new precision: 1; print: 1/3 asFloat.
I'm already using the GRNumberPrinter and also the Date printer to print these values. Regards! [1] https://en.wikipedia.org/wiki/Handicap_(golf)#World_Handicap_System
Hi Esteban,
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result. And then it fails because of the difference above when it is "semantically" correct.
If you want reliable precision tests with IEEE floats, you should use rounding to powers of 2, not 10. Round to 1/8 or 1/16, but not to 1/10. The fundamental issue is that 1/10 does not have an exact finite representation in base 2. No matter how I/O libraries handle the conversion between base 2 (in the binary representation of IEEE floats) and base 10 (in the text representation), there will always be bad surprises because the expectations we have from working in base 10 cannot all be met with an internal representation in base 2. So if your goal is not fundamentally related to I/O, forget about I/O and design your code to work the binary level, in particular for testing. However, if your problem specification explicitly requires precision guarantees in base 10 (which seems to be your case, from a quick glance at the Wikipedia page you cite), then it's best to avoid binary floats completely. Rationals (fractions) are one option, scaled integers another one. Konrad.
On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001. Floats should always be compared using an epsilon (precision) value in tests, not using equality.
BTW why closeTo: is initialized by default with a value of 0.0001 ? https://github.com/pharo-project/pharo/issues/3067 Sent from my iPhone
On 7 Sep 2020, at 20:10, Sven Van Caekenberghe <sven@stfx.eu> wrote:

On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends.
(9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001.
Floats should always be compared using an epsilon (precision) value in tests, not using equality.
Hi Sven, On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.
I remembered about it, and implemented my own #assertDifferential:equals: which I first did convert to ScaledDecimals and then compares with #roundedByScale.
Floats should always be compared using an epsilon (precision) value in tests, not using equality.
Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it? Regards! Esteban A. Maringolo
Esteban, You can instead use `7.1 round: 1` to get 7.1. It's not the problem of the rounding algorithm. It's because IEEE float can't express the exact value of 0.1. --- tomo 2020å¹´9æ7æ¥(æ) 21:28 Esteban Maringolo <emaringolo@gmail.com>:
Hi Sven,
On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.
I remembered about it, and implemented my own #assertDifferential:equals: which I first did convert to ScaledDecimals and then compares with #roundedByScale.
Floats should always be compared using an epsilon (precision) value in tests, not using equality.
Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it?
Regards!
Esteban A. Maringolo
Hi Tomohiro, Thanks, that certainly is the simplest solution that does exactly what I need. Maybe roundTo: should be considered harmful when using Floats. (and it is very unlikely anybody is going to round to something other than a power of 10) Best regards! Esteban A. Maringolo On Mon, Sep 7, 2020 at 9:51 AM Tomohiro Oda <tomohiro.tomo.oda@gmail.com> wrote:
Esteban,
You can instead use `7.1 round: 1` to get 7.1. It's not the problem of the rounding algorithm. It's because IEEE float can't express the exact value of 0.1. --- tomo
2020å¹´9æ7æ¥(æ) 21:28 Esteban Maringolo <emaringolo@gmail.com>:
Hi Sven,
On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.
I remembered about it, and implemented my own #assertDifferential:equals: which I first did convert to ScaledDecimals and then compares with #roundedByScale.
Floats should always be compared using an epsilon (precision) value in tests, not using equality.
Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it?
Regards!
Esteban A. Maringolo
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it. 7.1 IS NOT 71/10. 0.1 IS NOT 1/10. Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. #include <math.h> #include <stdio.h> int main(void) { printf("%.18f\n", 7.1); printf("%.18f\n", round(7.1 / 0.1) * 0.1); return 0; } % a.out 7.099999999999999645 7.100000000000000533 Does this help? "7.1" is a bit LESS than 7.1. "7.1 roundTo: 0.1" is a bit MORE than 7.1. This is the best that the computer's hardware can do. The systems that print 7.1 are LYING to you, and Pharo is not. On Tue, 8 Sep 2020 at 00:28, Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi Sven,
On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote:
It is not for printing but for testing. I want to assert that a certain calculation gives the expected result.
Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.
I remembered about it, and implemented my own #assertDifferential:equals: which I first did convert to ScaledDecimals and then compares with #roundedByScale.
Floats should always be compared using an epsilon (precision) value in tests, not using equality.
Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it?
Regards!
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't). Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise". In any case, the method proposed by Tomohiro solved my issues. Regards, Esteban A. Maringolo
If you are going to use floating point numbers in any programming language, Pharo included, you need to understand something about them. Things like - the relevant precision is *fixed*; a Float in Pharo is *always* a 64-bit IEEE floating-point number, which amongst other things means (to a first approximation) 54 binary digits of precision. It doesn't make the slightest amount of difference how many decimal digits you use when writing the number, the precision is *always* 54 binary digits. So you *did* get the precision out of rounding that you put in, it's just that the precision you put in was not what you thought it was. - almost all modern architectures offer BINARY arithmetic, not decimal. (The main exceptions are IBM z/series and IBM POWER, which offer binary *and* decimal.) Rounding a number to n decimals is ALWAYS problematic using binary floats. This has little or nothing to do with Smalltalk: it's the hardware. (This is why you are advised to keep money in cents, not dollars.) - Again, this is NOT a language issue, it is a hardware issue. IF you understand how binary floating point arithmetic works, then the arithmetic you get in C or R or Smalltalk or Matlab does not surprise you at all. If you DON'T understand how binary floating point arithmetic works, the hardware will *get* you sooner or later. - ScaledDecimal in Pharo didn't have to violate the principle of least surprise, but it does, big time. - It would be really nice if Pharo offered true decimal arithmetic as an option. There may already be a package for this. On Tue, 8 Sep 2020 at 15:46, Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
I'm coming back to this because I've been bitten by these floating points things again. If in Pharo [1] you do: a := 6.7 + (32.8 - 35) It will produce: 4.499999999999997 Which, when rounded, will produce 4. In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5. I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before. The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output. In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time. Is there any way to "configure" the floating point calculation to behave as the "other systems"? What is the best way to anticipate these situations, am I the only one being bitten by these issues? Thanks in advance for any hints about these problems. Best regards, [1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried Esteban A. Maringolo On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
On 14 Jun 2021, at 22:44, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
But, a roundTo: 0.1 "=> 4.5"
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
BTW, I recently wrote my own float printer, as an experiment. NeoJSONFloatPrinter lowPrecision print: a. => 4.5 What I wanted was a human friendly, compact float printer, that tries to go for the shortest, simplest number. It prefers integers and goes to scientific notation when needed, while limiting the precision. Maybe it is interesting to look at the code. But I am far from an expert.
On 14 Jun 2021, at 23:23, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 14 Jun 2021, at 22:44, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
But,
a roundTo: 0.1 "=> 4.5"
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
Hi Sven, I accidentally skipped this. How is this different from the GRNumberPrinter? Where is the code available? Thanks! Esteban A. Maringolo On Mon, Jun 14, 2021 at 6:30 PM Sven Van Caekenberghe <sven@stfx.eu> wrote:
BTW, I recently wrote my own float printer, as an experiment.
NeoJSONFloatPrinter lowPrecision print: a.
=> 4.5
What I wanted was a human friendly, compact float printer, that tries to go for the shortest, simplest number. It prefers integers and goes to scientific notation when needed, while limiting the precision. Maybe it is interesting to look at the code.
But I am far from an expert.
On 14 Jun 2021, at 23:23, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 14 Jun 2021, at 22:44, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
But,
a roundTo: 0.1 "=> 4.5"
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
On 15 Jun 2021, at 16:19, Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi Sven,
I accidentally skipped this.
How is this different from the GRNumberPrinter?
It is similar, but different (it does several things to produce cleaner numbers). Basically, when I produced certain JSON with floats that were results of calculations, I got these very long, ugly numbers. Forcing them to a certain precision always added trailing zeros and made them floats. I also wanted integers to be printed when possible (1 instead of 1.0, or 0 instead of 0.0). I also have automatic switching to scientific notation outside a certain range.
Where is the code available?
It is part of NeoJSON, https://github.com/svenvc/NeoJSON Keep in mind that this is just an unfinished experiment.
Thanks!
Esteban A. Maringolo
On Mon, Jun 14, 2021 at 6:30 PM Sven Van Caekenberghe <sven@stfx.eu> wrote:
BTW, I recently wrote my own float printer, as an experiment.
NeoJSONFloatPrinter lowPrecision print: a.
=> 4.5
What I wanted was a human friendly, compact float printer, that tries to go for the shortest, simplest number. It prefers integers and goes to scientific notation when needed, while limiting the precision. Maybe it is interesting to look at the code.
But I am far from an expert.
On 14 Jun 2021, at 23:23, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 14 Jun 2021, at 22:44, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
But,
a roundTo: 0.1 "=> 4.5"
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
Hi Sven,
If in Pharo [1] you do: a := 6.7 + (32.8 - 35) It will produce: 4.499999999999997 Which, when rounded, will produce 4.
But, a roundTo: 0.1 "=> 4.5"
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output. (9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005" However, at this point I know that Pharo "does the right, raw, thing" (at least compared to other mainstream languages), but it still produces a surprise effect. In particular when people go to Excel to "check that your system does the right thing", uses the same formula and it gives a different result to them. Then open a calculator (or a physical one) and it gives the same number as Excel. So an hour after the fact you're arguing about floating point implementations to non-technical people :-) Regards! Esteban A. Maringolo
On 15/06/2021 01:03, Esteban Maringolo wrote:
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output.
(9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005"
However, at this point I know that Pharo "does the right, raw, thing" (at least compared to other mainstream languages), but it still produces a surprise effect.
That's the "floating point surprise" that everyone has at some point, no matter the language and runtime system. If that surprise is a problem for you, are you sure that floating-point arithmetic is what you really want? Maybe your needs are better served with integers and fractions. Konrad.
Have you considered using fixed-point arithmetic? For example: 7.1s2 roundTo: 0.1s2 The rule of thumb I stick to is to use FP only if I know the inaccuracies won't bite me. Funny enough, both 7.1 and 0.1 are already not accurately representable as floats. (And by coincidence, I prepared exam questions about floats for my students yesterday. ) Kind regards, Steffen Konrad Hinsen schrieb am Dienstag, 15. Juni 2021 07:02:30 (+02:00):
On 15/06/2021 01:03, Esteban Maringolo wrote:
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output.
(9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005"
However, at this point I know that Pharo "does the right, raw, thing" (at least compared to other mainstream languages), but it still produces a surprise effect.
That's the "floating point surprise" that everyone has at some point, no matter the language and runtime system. If that surprise is a problem for you, are you sure that floating-point arithmetic is what you really want? Maybe your needs are better served with integers and fractions.
Konrad.
-- Gesendet mit Vivaldi Mail. Laden Sie Vivaldi kostenlos von vivaldi.com herunter.
Typo: FP should read floating point and not fixed point. Steffen Märcker schrieb am Dienstag, 15. Juni 2021 08:47:24 (+02:00):
Have you considered using fixed-point arithmetic? For example: 7.1s2 roundTo: 0.1s2
The rule of thumb I stick to is to use FP only if I know the inaccuracies won't bite me. Funny enough, both 7.1 and 0.1 are already not accurately representable as floats. (And by coincidence, I prepared exam questions about floats for my students yesterday. )
Kind regards, Steffen
Konrad Hinsen schrieb am Dienstag, 15. Juni 2021 07:02:30 (+02:00):
On 15/06/2021 01:03, Esteban Maringolo wrote:
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output.
(9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005"
However, at this point I know that Pharo "does the right, raw, thing" (at least compared to other mainstream languages), but it still produces a surprise effect.
That's the "floating point surprise" that everyone has at some point, no matter the language and runtime system. If that surprise is a problem for you, are you sure that floating-point arithmetic is what you really want? Maybe your needs are better served with integers and fractions.
Konrad.
-- Gesendet mit Vivaldi Mail. Laden Sie Vivaldi kostenlos von vivaldi.com herunter.
-- Gesendet mit Vivaldi Mail. Laden Sie Vivaldi kostenlos von vivaldi.com herunter.
Well... fixed point numbers (aka ScaledDecimals) had their issues as well. And in this particular case, it might cause issues, since the "full precision" (whatever that means) must be retained when using such a number to derive another one. E.g. in my case: ch := 6.7 + (32.8 - 35). "course handicap" allowance := 0.85. "85%" ph := (ch * allowance) roundedHandicap. "playing handicap" The #roundedHandicap is like #rounded but rounds -0.5 towards -1 instead of toward 0. My first approach was to use fixed points, but then I hit issues where some results also produced "wrong" numbers (according to the reference), also in Pharo some ScaledDecimals compare as different even when they're printed as the same (which is not correct, IMO). I just tested using everything as fractions and converting to float (or ScaledDecimal) at the very last step and it produces the expected result. I'll review and convert whatever is needed (including database fields) to work with this. The good thing is that all the involved numbers only have one decimal as much. Thanks! Esteban A. Maringolo On Tue, Jun 15, 2021 at 3:48 AM Steffen Märcker <merkste@web.de> wrote:
Have you considered using fixed-point arithmetic? For example: 7.1s2 roundTo: 0.1s2
The rule of thumb I stick to is to use FP only if I know the inaccuracies won't bite me. Funny enough, both 7.1 and 0.1 are already not accurately representable as floats. (And by coincidence, I prepared exam questions about floats for my students yesterday. )
Kind regards, Steffen
Konrad Hinsen schrieb am Dienstag, 15. Juni 2021 07:02:30 (+02:00):
On 15/06/2021 01:03, Esteban Maringolo wrote:
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output.
(9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005"
However, at this point I know that Pharo "does the right, raw, thing" (at least compared to other mainstream languages), but it still produces a surprise effect.
That's the "floating point surprise" that everyone has at some point, no matter the language and runtime system. If that surprise is a problem for you, are you sure that floating-point arithmetic is what you really want? Maybe your needs are better served with integers and fractions.
Konrad.
-- Gesendet mit Vivaldi Mail. Laden Sie Vivaldi kostenlos von vivaldi.com herunter.
On 15/06/2021 16:05, Esteban Maringolo wrote:
And in this particular case, it might cause issues, since the "full precision" (whatever that means) must be retained when using such a number to derive another one.
To me, "full precision" means "no precision is ever lost in arithmetic operations". If that's what you need, your choices are 1) Integers 2) Rational numbers (fractions) 3) Computable numbers (often called "exact reals", a term I find misleading). Pharo has 1) and 2). Konrad
On Wed, Jun 16, 2021 at 9:13 AM Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
On 15/06/2021 16:05, Esteban Maringolo wrote:
And in this particular case, it might cause issues, since the "full precision" (whatever that means) must be retained when using such a number to derive another one.
To me, "full precision" means "no precision is ever lost in arithmetic operations". If that's what you need, your choices are
1) Integers
2) Rational numbers (fractions)
Pharo has 1) and 2).
Yeap, I'll switch to 2. What I'm not totally convinced of is whether to store numbers as integers (knowing beforehand the decimal precision, and converting it to fractions when read back) or if to store them as scaled decimals (which internally hold a fraction). Thanks!
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. I think that going for 1/10 fractions/precision is not going to help: you got a division by 113 in your formula [https://en.wikipedia.org/wiki/Handicap_(golf)], this will give smaller fractions. The problem is not the calculation (modern 64-bit floats as in Pharo are plenty accurate), it is how you handle results. You should just round it correctly and be done with it. Note that a roundTo: 0.00000000000001. (1e-14) still gives 4.5 it is only one step further that you hit the limit and get the ugly but correct result. I assume that most calculators always use a printing precision that is lower than their internal precision, hence they hide this reality. When computing with money, you would be inclined to put everything in cents (because you cannot divide them further). But once you start computing percentage discounts or taxes, you again get problems. At each of those steps you must make sure that no cents are lost.
On 16 Jun 2021, at 15:19, Esteban Maringolo <emaringolo@gmail.com> wrote:
On Wed, Jun 16, 2021 at 9:13 AM Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
On 15/06/2021 16:05, Esteban Maringolo wrote:
And in this particular case, it might cause issues, since the "full precision" (whatever that means) must be retained when using such a number to derive another one.
To me, "full precision" means "no precision is ever lost in arithmetic operations". If that's what you need, your choices are
1) Integers
2) Rational numbers (fractions)
Pharo has 1) and 2).
Yeap, I'll switch to 2.
What I'm not totally convinced of is whether to store numbers as integers (knowing beforehand the decimal precision, and converting it to fractions when read back) or if to store them as scaled decimals (which internally hold a fraction).
Thanks!
On 16/06/2021 15:52, Sven Van Caekenberghe wrote:
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.
Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free. Konrad
I didn't know there were decimal floats. Are they used elsewhere? Esteban A. Maringolo On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
On 16/06/2021 15:52, Sven Van Caekenberghe wrote:
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.
Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free.
Konrad
2Decimal floats are (a) part of the IEEE 758-2008 standard for floating-point arithmetic, (b) implemented as part of the instruction set in IBM z/Series and POWER computers, (c) allowed for in the current C standard, (d) partially supported by GCC https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Decimal-Float.html and (e) available for most machines via the "reference implementation" in C. The current COBOL standard basically defines its arithmetic in terms of this. Java of course has BigDecimal. While decimal floats became part of IEEE 754 in 2008, they were previously covered by IEEE 854. Reverting to ScaledDecimal, let us consider the 10.0s1 / 3 example. There is a huge problem here with no really satisfactory answer. In my library, it currently answers 3.3s1, exactly equal to 33/10. In Squeak/Pharo, it answers "I am really 10/3 but print me with one decimal", so you see 3.3a1 but it *behaves* like 10.3. In VisualAge Smalltalk you get 3.333333333333333333333333333333 If it were not for the fact that the ANSI Smalltalk standard appears to require that <scaled decimal> / <integer> yield a <scaled decimal>, I would prefer to answer 10/3. Decimals just plain are not closed under division (although they are closed under addition, subtraction, and multiplication). When it comes to scaled decimals, the ANSI Smalltalk standard is frankly a mess. Amongst other things, it defers the semantics of operations on them to the Language-Independent Arithmetic standard, which explicitly denies having anything to say about them! I understand why my library does what it does. I even tried to publish a paper about it. I understand why VisualAge Smalltalk does what it does. (I've read IBM/360 PrincOps.) I do NOT understand why Squeak and Pharo do what they do, and if anyone knows the rationale for their ScaledDecmal please tell me. On Thu, 17 Jun 2021 at 06:29, Esteban Maringolo <emaringolo@gmail.com> wrote:
I didn't know there were decimal floats.
Are they used elsewhere?
Esteban A. Maringolo
On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
On 16/06/2021 15:52, Sven Van Caekenberghe wrote:
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.
Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free.
Konrad
On 16 Jun 2021, at 16:20, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:
On 16/06/2021 15:52, Sven Van Caekenberghe wrote:
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.
Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free.
Konrad
mmm, this is interesting. It would be possible (and maybe it has already been done) to implement/add such a decimal floating point number to Pharo. It would require reimplementing all operations from scratch (esp. sin/cos/tan log/exp and so on), it would be slow, but interesting.
Sven Van Caekenberghe <sven@stfx.eu> writes:
It would be possible (and maybe it has already been done) to implement/add such a decimal floating point number to Pharo. It would require reimplementing all operations from scratch (esp. sin/cos/tan log/exp and so on), it would be slow, but interesting.
That would be an interesting experiment indeed. Either use one of the IEEE754 decimal float formats (https://en.wikipedia.org/wiki/Decimal_floating_point), or Douglas Crockford's DEC64 proposal (https://www.crockford.com/dec64.html, which strangely enough does not refer to the older IEEE standard). There are implementations of both for various other languages that can serve as inspiration. Konrad.
On Wed, Jun 16, 2021 at 10:52 AM Sven Van Caekenberghe <sven@stfx.eu> wrote:
I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.
I think that going for 1/10 fractions/precision is not going to help: you got a division by 113 in your formula [https://en.wikipedia.org/wiki/Handicap_(golf)], this will give smaller fractions.
And the reasons behind why they chose 113 are still unknown, because that number is used to get a "difficulty coefficient", so if the course is rated a 125, then 125/113 will give you roughly a ~1.1 coefficient. They could have used 100.
The problem is not the calculation (modern 64-bit floats as in Pharo are plenty accurate), it is how you handle results. You should just round it correctly and be done with it.
Note that a roundTo: 0.00000000000001. (1e-14) still gives 4.5 it is only one step further that you hit the limit and get the ugly but correct result.
I'm not doing that, and I only store the original number (which has a fixed decimal) and the "rounded" (to no decimals) number, all intermediate numbers are re-calculated (maybe cached) but not stored elsewhere. What I might be doing wrong is to store the original index as float instead of a Decimal number.
When computing with money, you would be inclined to put everything in cents (because you cannot divide them further). But once you start computing percentage discounts or taxes, you again get problems. At each of those steps you must make sure that no cents are lost.
And that includes adding a "rounding" item for any difference in all invoices, orders, etc. Regards! Esteban A. Maringolo
The problem is that while it is possible to read the ANSI Smalltalk standard as requiring ScaledDecimals to be fixed point numbers (which are a kind of exact rational number of the form x * 10**y, x and y integers), in Squeak and Pharo ScaledDecimal numbers are NOT fixed-point numbers. They are unlimited rational numbers with the scale being used for printing purposes only. This creates enormous confusion and I really don't see any point in Squeak/Pharo ScaledDecimal existing at all. There *is* point in having fixed-point decimal numbers as they are a perfect match for SQL data bases and several programming languages. I've now located a PDF of the USGA Rules of Handicapping. Wow. 128 pages. And with all that they couldn't explain the calculations precisely. Rounding arrives in more than one place. For example you might have to average the best 1-8 of the last (as many as are available up to 20) scores and then round to one decimal, but it is not stated how ties are to be broken (which can arise for the best 2, 4, 6, or 8). It is probably safe to assume rounding out. Number>>roundOut ^self negative ifTrue: [((self * 2 - 1) / 2) ceiling] ifFalse: [((self * 2 + 1) / 2) floor] On Wed, 16 Jun 2021 at 02:06, Esteban Maringolo <emaringolo@gmail.com> wrote:
Well... fixed point numbers (aka ScaledDecimals) had their issues as well.
And in this particular case, it might cause issues, since the "full precision" (whatever that means) must be retained when using such a number to derive another one.
E.g. in my case: ch := 6.7 + (32.8 - 35). "course handicap" allowance := 0.85. "85%" ph := (ch * allowance) roundedHandicap. "playing handicap"
The #roundedHandicap is like #rounded but rounds -0.5 towards -1 instead of toward 0.
My first approach was to use fixed points, but then I hit issues where some results also produced "wrong" numbers (according to the reference), also in Pharo some ScaledDecimals compare as different even when they're printed as the same (which is not correct, IMO).
I just tested using everything as fractions and converting to float (or ScaledDecimal) at the very last step and it produces the expected result. I'll review and convert whatever is needed (including database fields) to work with this. The good thing is that all the involved numbers only have one decimal as much.
Thanks!
Esteban A. Maringolo
On Tue, Jun 15, 2021 at 3:48 AM Steffen Märcker <merkste@web.de> wrote:
Have you considered using fixed-point arithmetic? For example: 7.1s2 roundTo: 0.1s2
The rule of thumb I stick to is to use FP only if I know the inaccuracies won't bite me. Funny enough, both 7.1 and 0.1 are already not accurately representable as floats. (And by coincidence, I prepared exam questions about floats for my students yesterday. )
Kind regards, Steffen
Konrad Hinsen schrieb am Dienstag, 15. Juni 2021 07:02:30 (+02:00):
On 15/06/2021 01:03, Esteban Maringolo wrote:
Sure, but what initiated this thread was a reference to roundTo: 0.1 which produced a "wrong" output.
(9.1 + (-2.0)) roundTo: 0.1 "=> 7.1000000000000005" 7.1 roundTo: 0.1 "=> 7.1000000000000005"
However, at this point I know that Pharo "does the right, raw,
thing"
(at least compared to other mainstream languages), but it still produces a surprise effect.
That's the "floating point surprise" that everyone has at some point, no matter the language and runtime system. If that surprise is a problem for you, are you sure that floating-point arithmetic is what you really want? Maybe your needs are better served with integers and fractions.
Konrad.
-- Gesendet mit Vivaldi Mail. Laden Sie Vivaldi kostenlos von vivaldi.com herunter.
On Wed, Jun 16, 2021 at 10:49 AM Richard O'Keefe <raoknz@gmail.com> wrote:
The problem is that while it is possible to read the ANSI Smalltalk standard as requiring ScaledDecimals to be fixed point numbers (which are a kind of exact rational number of the form x * 10**y, x and y integers), in Squeak and Pharo ScaledDecimal numbers are NOT fixed-point numbers. They are unlimited rational numbers with the scale being used for printing purposes only. This creates enormous confusion and I really don't see any point in Squeak/Pharo ScaledDecimal existing at all.
I only used decimals in other platforms, and only for accounting purposes, but to me if two scaled decimals print the same but answer false to #= then something is wrong conceptually. I can identify the issue, but the reasoning behind how it works, or even more, how to fix it, is above my level.
There *is* point in having fixed-point decimal numbers as they are a perfect match for SQL data bases and several programming languages.
That's the only use I gave it. So if we divided 10.0s1 / 3 ended up with 3.3s1 each, and we had to calculate the rounding errors (0.1s1 in this case) to make all balance.
I've now located a PDF of the USGA Rules of Handicapping. Wow. 128 pages.
That's just USGA, there is a "World System" but each "region" or local federation does a variation of the "world system". So Belgium does one thing, Australia another one, Argentina another one, England decided to only take one part... standards, they call it :-) I Implemented a couple of them.
And with all that they couldn't explain the calculations precisely.
Try to ask one federation how they calculate the PCC, and they won't tell you, it's like the coca-cola formula.
Rounding arrives in more than one place. For example you might have to average the best 1-8 of the last (as many as are available up to 20) scores and then round to one decimal, but it is not stated how ties are to be broken (which can arise for the best 2, 4, 6, or 8).
In this computation, the tie doesn't matter, because you start with an _already rounded_ number (called a "differential"), so it doesn't matter if you choose 11.4 from the 5th or 11.4 from the 19th. The rounding happens at the very end and determines a new rounded (to one decimal) number that's going to be used in other formulas (like the one that re-kicked this thread). The ties in competitions are solved differently. In real tournaments by playoff, or in amateur tournaments either by playoff or by a hole by hole comparison. If that even happens[*], there is the option to toss a coin to untie it. [*] In +400 tournaments managed by my software (involving ~15000 scorecards), that never happened. So to wrap up and bring it back to the topic, a proper ScaledDecimal implementation would help me storing the numbers as such guaranteeing no floating point funny thing happens, but since there is no such thing, I'll rather build the scaled decimals manually (e.g. ScaledDecimal fromFraction: 67/10) or not use scaled decimals and try to use the Fractions directly wherever possible. It's always good to learn new things.
Richard As I said it thousands of times, - first we are not good in math - second Pharo is what we have and we never said that it is the best system ever we just enhanced daily - third, I never saw a piece of code from you. - if you want to see Pharo improve (which I doubt about) just send us code and tests Iâm sad that you systematically ignore any of my emails on the topic. You could have an impact but you prefer to hide yourselves in your tower. Good luck with it. S.
"first we are not good in math" First, I am not criticising YOU. Pharo's weird ScaledDecimal is Squeak's weird ScaledDecimal and you have not changed it. Fine. "second Pharo is what we have and we never said that it is the best system ever" True, but irrelevant. "third I never saw a piece of code from you" That is not true. Why, in this thread alone I contributed #roundOut. I have provided several pieces of code in this mailing list over the years. You can say that my code is no good, or that you don't want it, or that you think there is a better way to do the (admittedly small) things that I have provided, but if you never saw them you must have shut your eyes. In addition, a snapshot of my system is available on the web for anyone to play with. I shall post here when my GitHub repository is populated; I just have to clear a few more things out of the local directory that I don't have the right to distribute. (For example, benchmarking my CSV library against NeoCSV meant having to have NeoCSV, but it's not mine to distribute.) On Thu, 17 Jun 2021 at 08:45, ducasse <stepharo@netcourrier.com> wrote:
Richard
As I said it thousands of times, - first we are not good in math - second Pharo is what we have and we never said that it is the best system ever we just enhanced daily - third, I never saw a piece of code from you. - if you want to see Pharo improve (which I doubt about) just send us code and tests
Iâm sad that you systematically ignore any of my emails on the topic. You could have an impact but you prefer to hide yourselves in your tower. Good luck with it.
S.
You are right! Lorenzo -----Messaggio originale----- Da: ducasse [mailto:stepharo@netcourrier.com] Inviato: mercoledì 16 giugno 2021 22:44 A: Any question about pharo is welcome <pharo-users@lists.pharo.org> Oggetto: [Pharo-users] Re: Rounding in Floats Richard As I said it thousands of times, - first we are not good in math - second Pharo is what we have and we never said that it is the best system ever we just enhanced daily - third, I never saw a piece of code from you. - if you want to see Pharo improve (which I doubt about) just send us code and tests Iâm sad that you systematically ignore any of my emails on the topic. You could have an impact but you prefer to hide yourselves in your tower. Good luck with it. S.
It's not quite clear what is happening in the other systems. Here is R
6.7 + (32.8 - 35) [1] 4.5 round(6.7 + (32.8 - 35)) [1] 4 The result *is* 4.499999.... but is *printed* as 4.5.
(+ 6.7 (- 32.8 35)) 4.499999999999997 (round (+ 6.7 (- 32.8 35)))
Here is Gambit Scheme 4. Here is astc Smalltalk 6.7 + (32.8 - 35) printOn: Transcript Transcript tab. (6.7 + (32.8 - 35)) rounded printOn: Transcript. Transcript cr. 4.499999999999997 4 And here is Python
x = 6.7 + (32.8 - 35) x 4.499999999999997 round(x) 4
For what it's worth, I expected ScaledDecimal to act like fixed point arithmetic, and implemented it that way in my own Smalltalk library, where two ScaledDecimals *do* print the same if and only if they are numerically exactly the same. What Squeak and Pharo do is exceedingly odd: a ScaledDecimal is an exact rational number (Integer or Fraction) combined with a precision that is used for printing, not for calculation. There really isn't any principle of least surprise when it comes to floating- point arithmetic. It's full of surprises and edge cases. Excel in particular is notorious for messing up due to trying to pretend all is well. In this particular case, the exact result is 4.5 There are at least three rules for rounding such numbers: rounding out (5), rounding in (4), and rounding in [banker'salgorithm] (4 here, but 5.5 -> 6). So you are pushing up against an edge case for exact hand calculation! I think you need to re-express your entire calculation to use exact arithmetic. That or get agreement on "de minimis non curat lex". On Tue, 15 Jun 2021 at 08:45, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
On Mon, Jun 14, 2021 at 10:37 PM Richard O'Keefe <raoknz@gmail.com> wrote:
For what it's worth, I expected ScaledDecimal to act like fixed point arithmetic, and implemented it that way in my own Smalltalk library, where two ScaledDecimals *do* print the same if and only if they are numerically exactly the same. What Squeak and Pharo do is exceedingly odd: a ScaledDecimal is an exact rational number (Integer or Fraction) combined with a precision that is used for printing, not for calculation.
I pointed out before the weird behavior of ScaledDecimals in Pharo, two ScaledDecimal that print the same are not equal. Giving some weird comparison issues as result.
There really isn't any principle of least surprise when it comes to floating- point arithmetic. It's full of surprises and edge cases. Excel in particular is notorious for messing up due to trying to pretend all is well. In this particular case, the exact result is 4.5
Well... for the end user, it produces what they'd expect. So it might mess up, but in a spreadsheet 6.7 - 2.2 should give 4.5.
There are at least three rules for rounding such numbers: rounding out (5), rounding in (4), and rounding in [banker'salgorithm] (4 here, but 5.5 -> 6). So you are pushing up against an edge case for exact hand calculation!
I implemented a Float>>#roundedHandicap method that does something like the banking algorithm, but rounds positive numbers towards the next integer and negative numbers towards zero. E.g. 0.49 -> 0 0.5 -> 1 -0.5 -> 0 -0.51 -> -1 Nothing uses more than one decimal, so a ScaledDecimal would work but the specification says that it should use all possible precision in intermediate calculations, so I cannot use it.
I think you need to re-express your entire calculation to use exact arithmetic.
I really don't know how to do this, any pointers? Nothing is more straightforward than addition and subtraction to me, 6.7 - 2.2 is the simplest it can get. The common formula here is: h * s / 113 + (r - p), but in this particular case s was 113 so it removed the "troubling" part.
That or get agreement on "de minimis non curat lex".
I had to search for that expression. Now I know, I agree. Regards, Esteban A. Maringolo
On Tue, 15 Jun 2021 at 08:45, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
Whether 6.7 - 2.2 gives 4.5 or something else is a subtle issue, starting with the fact that 6.7 and 2.2 cannot be represented precisely in floating-point format. In this particular case, the errors happen to match up. What you *can* rely on in Smalltalk is that (67/10) - (22/10) = (9/2). Let's just consider h * s / 113 + (r - p). If h, s, r, and p are all exact numbers (Integers or Fractions) the result will be an exact number. So what kinds of number are h, s, r, and p, and *why* are they like that? If you are reading them from a file, consider that Fraction readFrom: '6.7' readStream >>> (67/10) Is there a specification you could show us? Some test data? On Tue, 15 Jun 2021 at 14:24, Esteban Maringolo <emaringolo@gmail.com> wrote:
On Mon, Jun 14, 2021 at 10:37 PM Richard O'Keefe <raoknz@gmail.com> wrote:
For what it's worth, I expected ScaledDecimal to act like fixed point arithmetic, and implemented it that way in my own Smalltalk library, where two ScaledDecimals *do* print the same if and only if they are numerically exactly the same. What Squeak and Pharo do is exceedingly odd: a ScaledDecimal is an exact rational number (Integer or Fraction) combined with a precision that is used for printing, not for calculation.
I pointed out before the weird behavior of ScaledDecimals in Pharo, two ScaledDecimal that print the same are not equal. Giving some weird comparison issues as result.
There really isn't any principle of least surprise when it comes to floating- point arithmetic. It's full of surprises and edge cases. Excel in particular is notorious for messing up due to trying to pretend all is well. In this particular case, the exact result is 4.5
Well... for the end user, it produces what they'd expect. So it might mess up, but in a spreadsheet 6.7 - 2.2 should give 4.5.
There are at least three rules for rounding such numbers: rounding out (5), rounding in (4), and rounding in [banker'salgorithm] (4 here, but 5.5 -> 6). So you are pushing up against an edge case for exact hand calculation!
I implemented a Float>>#roundedHandicap method that does something like the banking algorithm, but rounds positive numbers towards the next integer and negative numbers towards zero. E.g. 0.49 -> 0 0.5 -> 1 -0.5 -> 0 -0.51 -> -1
Nothing uses more than one decimal, so a ScaledDecimal would work but the specification says that it should use all possible precision in intermediate calculations, so I cannot use it.
I think you need to re-express your entire calculation to use exact arithmetic.
I really don't know how to do this, any pointers?
Nothing is more straightforward than addition and subtraction to me, 6.7 - 2.2 is the simplest it can get.
The common formula here is: h * s / 113 + (r - p), but in this particular case s was 113 so it removed the "troubling" part.
That or get agreement on "de minimis non curat lex".
I had to search for that expression. Now I know, I agree.
Regards,
Esteban A. Maringolo
On Tue, 15 Jun 2021 at 08:45, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
Hi Richard,
Whether 6.7 - 2.2 gives 4.5 or something else is a subtle issue, starting with the fact that 6.7 and 2.2 cannot be represented precisely in floating-point format. In this particular case, the errors happen to match up. What you *can* rely on in Smalltalk is that (67/10) - (22/10) = (9/2).
Let's just consider h * s / 113 + (r - p). If h, s, r, and p are all exact numbers (Integers or Fractions) the result will be an exact number. So what kinds of number are h, s, r, and p, and *why* are they like that? If you are reading them from a file, consider that Fraction readFrom: '6.7' readStream >>> (67/10)
I'm not reading them from a file, although some numbers come from an API as a JS number (float). Using Fractions wherever possible seems to be the right solution to this. And given that I already know the precision (it's always 1 decimal as much) I can turn everything into that and store them as DECIMAL in the database. I'm reluctant to perform such a change because everything has been working fine and this is just an edge case, but I think it is the right thing to do.
Is there a specification you could show us?
There is no specification other than the formulas: https://www.usga.org/content/usga/home-page/handicapping/roh/2020-rules-of-h... Section III -> Rules 6.1 and 6.2 There is only a remark after 6.1b that you must retain full precision in intermediate calculations. "the full calculated value is retained and rounding occurs only after the Playing Handicap calculation."
Some test data?
Not that I could find, I've been adding cases to my tests to assert it behaves properly. Thanks! Esteban A. Maringolo On Tue, Jun 15, 2021 at 4:13 AM Richard O'Keefe <raoknz@gmail.com> wrote:
On Tue, 15 Jun 2021 at 14:24, Esteban Maringolo <emaringolo@gmail.com> wrote:
On Mon, Jun 14, 2021 at 10:37 PM Richard O'Keefe <raoknz@gmail.com> wrote:
For what it's worth, I expected ScaledDecimal to act like fixed point arithmetic, and implemented it that way in my own Smalltalk library, where two ScaledDecimals *do* print the same if and only if they are numerically exactly the same. What Squeak and Pharo do is exceedingly odd: a ScaledDecimal is an exact rational number (Integer or Fraction) combined with a precision that is used for printing, not for calculation.
I pointed out before the weird behavior of ScaledDecimals in Pharo, two ScaledDecimal that print the same are not equal. Giving some weird comparison issues as result.
There really isn't any principle of least surprise when it comes to floating- point arithmetic. It's full of surprises and edge cases. Excel in particular is notorious for messing up due to trying to pretend all is well. In this particular case, the exact result is 4.5
Well... for the end user, it produces what they'd expect. So it might mess up, but in a spreadsheet 6.7 - 2.2 should give 4.5.
There are at least three rules for rounding such numbers: rounding out (5), rounding in (4), and rounding in [banker'salgorithm] (4 here, but 5.5 -> 6). So you are pushing up against an edge case for exact hand calculation!
I implemented a Float>>#roundedHandicap method that does something like the banking algorithm, but rounds positive numbers towards the next integer and negative numbers towards zero. E.g. 0.49 -> 0 0.5 -> 1 -0.5 -> 0 -0.51 -> -1
Nothing uses more than one decimal, so a ScaledDecimal would work but the specification says that it should use all possible precision in intermediate calculations, so I cannot use it.
I think you need to re-express your entire calculation to use exact arithmetic.
I really don't know how to do this, any pointers?
Nothing is more straightforward than addition and subtraction to me, 6.7 - 2.2 is the simplest it can get.
The common formula here is: h * s / 113 + (r - p), but in this particular case s was 113 so it removed the "troubling" part.
That or get agreement on "de minimis non curat lex".
I had to search for that expression. Now I know, I agree.
Regards,
Esteban A. Maringolo
On Tue, 15 Jun 2021 at 08:45, Esteban Maringolo <emaringolo@gmail.com> wrote:
I'm coming back to this because I've been bitten by these floating points things again.
If in Pharo [1] you do: a := 6.7 + (32.8 - 35)
It will produce: 4.499999999999997
Which, when rounded, will produce 4.
In other places [2] I do the same simple addition and subtraction it produces 4.5, that when rounded will produce 5.
I know now that Pharo doesn't lie to me while other systems do, and all that Richard pointed to before.
The issue here is that I'm following some calculation formula that was defined in some of the "other" systems, and so when I follow such a formula I get these edgy cases where my system produces a different output.
In this case the formula is for golf handicap calculations, and it caused my system to give 4 instead of 5 to a player, resulting in giving the first place to a player other than the one deserved. It was no big deal (it's not The Masters), but these cases appear from time to time.
Is there any way to "configure" the floating point calculation to behave as the "other systems"?
What is the best way to anticipate these situations, am I the only one being bitten by these issues?
Thanks in advance for any hints about these problems.
Best regards,
[1] Dolphin Smalltalk, JS, Python, Ruby, Dart produces the same output as Pharo. [2] VisualWorks, VAST, Excel, VB and all calculators I tried
Esteban A. Maringolo
On Tue, Sep 8, 2020 at 12:45 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
On Tue, Sep 8, 2020 at 12:16 AM Richard O'Keefe <raoknz@gmail.com> wrote:
"7.1 roundTo: 0.1 should return 7.1" You're still not getting it.
I was until Konrad explained it.
Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. <snip> The systems that print 7.1 are LYING to you, and Pharo is not.
I'm not assuming a mistake from Pharo, I had a wrong expectation what to get if I round to that precision. I don't know whether other systems lie or simply fulfill user expectations, if you send the #roundTo: to a float, I did expect to get a number with the same precision. That is my expectation as a user. As in the other thread I expected two scaled decimals that are printed equal to also be compared as equal (which they don't).
Whether there is a good reason for those behaviors is beyond my current comprehension, but it certainly doesn't follow the "principle of least surprise".
In any case, the method proposed by Tomohiro solved my issues.
Regards,
Esteban A. Maringolo
participants (9)
-
ducasse -
Esteban Maringolo -
Konrad Hinsen -
Lorenzo -
Richard O'Keefe -
serge.stinckwich＠gmail.com -
Steffen Märcker -
Sven Van Caekenberghe -
Tomohiro Oda