Scaled decimals comparisons
Hi, I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar. So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1 But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1 It fails because the comparison returns false. I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula. I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this. Thanks! Esteban A. Maringolo
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work. (((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1 So how do you compare two ScaledDecimals that _should_ be the same? Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
Interesting. Your expression evaluates to true in VA Smalltalk. Am 01.09.20 um 06:17 schrieb Esteban Maringolo:
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work.
(((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1
So how do you compare two ScaledDecimals that _should_ be the same?
Regards!
Esteban A. Maringolo
On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
It evaluates to false in VW too. I guess it is a matter of internal representation, since ScaledDecimals/FixedPoints are a Fraction after all. What is weird, and I'd say wrong (at least from the UX perspective) is that in Pharo (and Dolphin): ((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) -> 17.900000000000002 While in VW and VAST they round the float properly: ((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) -> 17.9 Regards! Esteban A. Maringolo On Tue, Sep 1, 2020 at 1:24 AM jtuchel@objektfabrik.de <jtuchel@objektfabrik.de> wrote:
Interesting. Your expression evaluates to true in VA Smalltalk.
Am 01.09.20 um 06:17 schrieb Esteban Maringolo:
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work.
(((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1
So how do you compare two ScaledDecimals that _should_ be the same?
Regards!
Esteban A. Maringolo
On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
-- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1
Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help: roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor Translating slightly for Pharo: roundedByScale | scaleFactor | scaleFactor := 10 ** scale. ^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor You can then do: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1 "true" HTH. John
On 1 Sep 2020, at 05:17, Esteban Maringolo <emaringolo@gmail.com> wrote:
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work.
(((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1
So how do you compare two ScaledDecimals that _should_ be the same?
Regards!
Esteban A. Maringolo
On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
Hi John, I was doing a similar rounding in the test assertion, I think your method is better and broadly usable (and maybe convenient for an #equals: implementation). Thanks! Esteban A. Maringolo On Tue, Sep 1, 2020 at 4:33 AM John Aspinall <john.aspinall@gmail.com> wrote:
Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help:
roundedByScale
| scaleFactor |
scaleFactor := 10 ** scale.
^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor
Translating slightly for Pharo:
roundedByScale
| scaleFactor |
scaleFactor := 10 ** scale.
^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor
You can then do:
(82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1 "true"
HTH.
John
On 1 Sep 2020, at 05:17, Esteban Maringolo <emaringolo@gmail.com> wrote:
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work.
(((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1
So how do you compare two ScaledDecimals that _should_ be the same?
Regards!
Esteban A. Maringolo
On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
Hi john you can propose for inclusion in pharo. - Add a nice comment (with executablecomments would be a plus). - Possibly some tests - Send a PR.
On 1 Sep 2020, at 09:32, John Aspinall <john.aspinall@gmail.com> wrote:
Dolphin shows the same behaviour. I added the following method to ScaledDecimal to help:
roundedByScale
| scaleFactor |
scaleFactor := 10 ** scale.
^(ScaledDecimal newFromNumber: (fraction * scaleFactor) rounded scale: scale) / scaleFactor
Translating slightly for Pharo:
roundedByScale
| scaleFactor |
scaleFactor := 10 ** scale.
^(ScaledDecimal newFromNumber: (self * scaleFactor) rounded scale: scale) / scaleFactor
You can then do:
(82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) roundedByScale = 9.5s1 "true"
HTH.
John
On 1 Sep 2020, at 05:17, Esteban Maringolo <emaringolo@gmail.com> wrote:
As a follow up to this, adding a roundTo: before converting into a ScaledDecimal does not work.
(((91 - (2 * 35.9) - (0 / 2) * (113/121)) roundTo: 0.1) asScaledDecimal: 1) = 17.9s1
So how do you compare two ScaledDecimals that _should_ be the same?
Regards!
Esteban A. Maringolo
On Tue, Sep 1, 2020 at 1:07 AM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
The behaviour of ScaledDecimal in Squeak and Pharo is - different from what the ANSI Smalltalk standard says - different from what other Smalltalks do (not that they agree) - completely different from decimal arithmetic in COBOL, PL/I, or SQL - seriously confusing. What you EXPECT is an exact rational number that is equal to an integer divided by a power of 10, so that the printed representation faithfully represents the value with no error. What you GET in Pharo is a rational number that PRINTS with a given number of decimal places but that is all. Thus we expect that x := 10/3 asScaledDecimal: 2. x asString ==> '3.33s2' y := x+x+x. y asString ==> '9.99s2' But no, y asString => '10.00s2'. Here's what the comment in the VW class FixedPoint says: "There are two possible ways to express FixedPoint numbers. One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations. For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000. For this reason, we have adopted an alternative representation, which is a slight variation on Fraction. The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding. The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used." The last sentence in the first paragraph is one I cannot agree with. If I want a calculation where (1/7 * 7) gives me back 1, then I use exact Fractions. If I am using ScaledDecimal, it is because I *want* fixed point decimal numbers, with the properties appropriate to fixed point decimal numbers. Wanting something that I could use to talk to databases like MariaDB I found that I had to write my own FixedPoint class, only to find that VW called ScaledDecimal FixedPoint. *Sigh*. Unless and until the current ScaledDecimal is ripped out and buried in an unmarked grave at a crossroad with a stake through its heart, you may wish to add VW-compatibility methods roundedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) rounded / t asScaledDecimal: scale truncatedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) truncated / t asScaledDecimal: scale and then use aScaledDecimal roundedToScale = anotherOne roundedToScale Note that the ANSI standard, which was agreed to by a whole bunch of Smalltalk experts, says "A <scaledDecimal> converted to a <Fraction> will be a fraction having the same numeric value but having an integer numerator and a denominator which is ten raised to the power of the <scaledDecimal>âs scale factor." Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if the result has a denominator of 3, SOMEONE stuffed up. On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
Richard we are really interested in improving Pharo on that aspect. Now Iâm really not good with such domain. Can you send us some code and tests? S.
On 2 Sep 2020, at 12:31, Richard O'Keefe <raoknz@gmail.com> wrote:
The behaviour of ScaledDecimal in Squeak and Pharo is - different from what the ANSI Smalltalk standard says - different from what other Smalltalks do (not that they agree) - completely different from decimal arithmetic in COBOL, PL/I, or SQL - seriously confusing.
What you EXPECT is an exact rational number that is equal to an integer divided by a power of 10, so that the printed representation faithfully represents the value with no error. What you GET in Pharo is a rational number that PRINTS with a given number of decimal places but that is all.
Thus we expect that x := 10/3 asScaledDecimal: 2. x asString ==> '3.33s2' y := x+x+x. y asString ==> '9.99s2' But no, y asString => '10.00s2'.
Here's what the comment in the VW class FixedPoint says: "There are two possible ways to express FixedPoint numbers. One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations. For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000.
For this reason, we have adopted an alternative representation, which is a slight variation on Fraction. The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding. The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used."
The last sentence in the first paragraph is one I cannot agree with. If I want a calculation where (1/7 * 7) gives me back 1, then I use exact Fractions. If I am using ScaledDecimal, it is because I *want* fixed point decimal numbers, with the properties appropriate to fixed point decimal numbers. Wanting something that I could use to talk to databases like MariaDB I found that I had to write my own FixedPoint class, only to find that VW called ScaledDecimal FixedPoint. *Sigh*.
Unless and until the current ScaledDecimal is ripped out and buried in an unmarked grave at a crossroad with a stake through its heart, you may wish to add VW-compatibility methods
roundedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) rounded / t asScaledDecimal: scale
truncatedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) truncated / t asScaledDecimal: scale
and then use
aScaledDecimal roundedToScale = anotherOne roundedToScale
Note that the ANSI standard, which was agreed to by a whole bunch of Smalltalk experts, says "A <scaledDecimal> converted to a <Fraction> will be a fraction having the same numeric value but having an integer numerator and a denominator which is ten raised to the power of the <scaledDecimal>âs scale factor." Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if the result has a denominator of 3, SOMEONE stuffed up.
On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <emaringolo@gmail.com <mailto:emaringolo@gmail.com>> wrote: Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
Hi Richard, +1 to what was asked by Stef. How hard is it to bring your knowledge, and existing implementation, into a "proper" implementation of ScaledDecimal (or FixedPoint) in Pharo? Regards! Esteban A. Maringolo On Wed, Sep 2, 2020 at 12:12 PM Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Richard
we are really interested in improving Pharo on that aspect. Now Iâm really not good with such domain. Can you send us some code and tests?
S.
On 2 Sep 2020, at 12:31, Richard O'Keefe <raoknz@gmail.com> wrote:
The behaviour of ScaledDecimal in Squeak and Pharo is - different from what the ANSI Smalltalk standard says - different from what other Smalltalks do (not that they agree) - completely different from decimal arithmetic in COBOL, PL/I, or SQL - seriously confusing.
What you EXPECT is an exact rational number that is equal to an integer divided by a power of 10, so that the printed representation faithfully represents the value with no error. What you GET in Pharo is a rational number that PRINTS with a given number of decimal places but that is all.
Thus we expect that x := 10/3 asScaledDecimal: 2. x asString ==> '3.33s2' y := x+x+x. y asString ==> '9.99s2' But no, y asString => '10.00s2'.
Here's what the comment in the VW class FixedPoint says: "There are two possible ways to express FixedPoint numbers. One is as a scaled Integer, but the problem here is that you can lose precision during intermediate calculations. For example, a property that seems useful is that the calculation (1.000 / 7 * 7) should give you back the number 1.000.
For this reason, we have adopted an alternative representation, which is a slight variation on Fraction. The number is expressed as the ratio of a numerator to a denominator, with the addition of a precision that is used for printing and for rounding. The number is not usually rounded off to its scale, but if an intermediate result must be rounded to its scale before being used, the messages #roundedToScale and #truncatedToScale may be used."
The last sentence in the first paragraph is one I cannot agree with. If I want a calculation where (1/7 * 7) gives me back 1, then I use exact Fractions. If I am using ScaledDecimal, it is because I *want* fixed point decimal numbers, with the properties appropriate to fixed point decimal numbers. Wanting something that I could use to talk to databases like MariaDB I found that I had to write my own FixedPoint class, only to find that VW called ScaledDecimal FixedPoint. *Sigh*.
Unless and until the current ScaledDecimal is ripped out and buried in an unmarked grave at a crossroad with a stake through its heart, you may wish to add VW-compatibility methods
roundedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) rounded / t asScaledDecimal: scale
truncatedToScale |t| t := 10 raisedToInteger: scale. ^(numerator * t / denominator) truncated / t asScaledDecimal: scale
and then use
aScaledDecimal roundedToScale = anotherOne roundedToScale
Note that the ANSI standard, which was agreed to by a whole bunch of Smalltalk experts, says "A <scaledDecimal> converted to a <Fraction> will be a fraction having the same numeric value but having an integer numerator and a denominator which is ten raised to the power of the <scaledDecimal>âs scale factor." Try "(10.00s2 / 3) asFraction" in your Smalltalk, and if the result has a denominator of 3, SOMEONE stuffed up.
On Tue, 1 Sep 2020 at 16:08, Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
I was doing some basic calculations based on a formula, and I wanted to convert the result to a scaled decimal in order to avoid having these "loose" decimals in 10th position or something similar.
So I did the following: 82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1 -> 9.5s1
But When I do this in a test: (82 - (2 * 35.9) - (0 / 2) * (113/121) asScaledDecimal: 1) = 9.5s1
It fails because the comparison returns false.
I guess this is the proper behavior, but I'd expected that the conversion from a Float to a scaled decimal would have eliminated the extra precision from the formula.
I can do a roundTo: 0.1 before converting, but I'd like to know what would be the proper way of dealing with this.
Thanks!
Esteban A. Maringolo
-------------------------------------------- Stéphane Ducasse http://stephane.ducasse.free.fr / http://www.pharo.org 03 59 35 87 52 Assistant: Aurore Dalle FAX 03 59 57 78 50 TEL 03 59 35 86 16 S. Ducasse - Inria 40, avenue Halley, Parc Scientifique de la Haute Borne, Bât.A, Park Plaza Villeneuve d'Ascq 59650 France
participants (5)
-
Esteban Maringolo -
John Aspinall -
jtuchel＠objektfabrik.de -
Richard O'Keefe -
Stéphane Ducasse