pharo-users@lists.pharo.org

Any question about pharo is welcome

View all threads

Rounding in Floats

KH
Konrad Hinsen
Wed, Jun 16, 2021 2:20 PM

On 16/06/2021 15:52, Sven Van Caekenberghe wrote:

I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.

Handheld calculators use decimal floats, not binary floats. That doesn't
remove rounding issues, but it makes conversion to and from print
representations loss-free.

Konrad

On 16/06/2021 15:52, Sven Van Caekenberghe wrote: > I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free. Konrad
EM
Esteban Maringolo
Wed, Jun 16, 2021 6:09 PM

On Wed, Jun 16, 2021 at 10:49 AM Richard O'Keefe raoknz@gmail.com wrote:

The problem is that while it is possible to read the ANSI Smalltalk standard as requiring ScaledDecimals to be fixed point numbers (which are a kind of exact rational number of the form x * 10**y, x and y integers), in Squeak and Pharo ScaledDecimal numbers are NOT fixed-point numbers.  They are unlimited rational numbers with the scale being used for printing purposes only.  This creates enormous confusion and I really don't see any point in Squeak/Pharo ScaledDecimal existing at all.

I only used decimals in other platforms, and only for accounting
purposes, but to me if two scaled decimals print the same but answer
false to #= then something is wrong conceptually. I can identify the
issue, but the reasoning behind how it works, or even more, how to fix
it, is above my level.

There is point in having fixed-point decimal numbers as they are a perfect match for SQL data bases and several programming languages.

That's the only use I gave it.
So if we divided 10.0s1 / 3 ended up with 3.3s1 each, and we had to
calculate the rounding errors (0.1s1 in this case) to make all
balance.

I've now located a PDF of the USGA Rules of Handicapping.  Wow.  128 pages.

That's just USGA, there is a "World System" but each "region" or local
federation does a variation of the "world system". So Belgium does one
thing, Australia another one, Argentina another one, England decided
to only take one part...  standards, they call it :-)

I Implemented a couple of them.

And with all that they couldn't explain the calculations precisely.

Try to ask one federation how they calculate the PCC, and they won't
tell you, it's like the coca-cola formula.

Rounding
arrives in more than one place.  For example you might have to average the
best 1-8 of the last (as many as are available up to 20) scores and then round
to one decimal, but it is not stated how ties are to be broken (which can arise
for the best 2, 4, 6, or 8).

In this computation, the tie doesn't matter, because you start with an
already rounded number (called a "differential"), so it doesn't
matter if you choose 11.4 from the 5th or 11.4 from the 19th. The
rounding happens at the very end and determines a new rounded (to one
decimal) number that's going to be used in other formulas (like the
one that re-kicked this thread).

The ties in competitions are solved differently. In real tournaments
by playoff, or in amateur tournaments either by playoff or by a hole
by hole comparison. If that even happens[*], there is the option to
toss a coin to untie it.

[*] In +400 tournaments managed by my software (involving ~15000
scorecards), that never happened.

So to wrap up and bring it back to the topic, a proper ScaledDecimal
implementation would help me storing the numbers as such guaranteeing
no floating point funny thing happens, but since there is no such
thing, I'll rather build the scaled decimals manually (e.g.
ScaledDecimal fromFraction: 67/10) or not use scaled decimals and try
to use the Fractions directly wherever possible.

It's always good to learn new things.

On Wed, Jun 16, 2021 at 10:49 AM Richard O'Keefe <raoknz@gmail.com> wrote: > The problem is that while it is possible to read the ANSI Smalltalk standard as requiring ScaledDecimals to be fixed point numbers (which are a kind of exact rational number of the form x * 10**y, x and y integers), in Squeak and Pharo ScaledDecimal numbers are NOT fixed-point numbers. They are unlimited rational numbers with the scale being used for printing purposes only. This creates enormous confusion and I really don't see any point in Squeak/Pharo ScaledDecimal existing at all. I only used decimals in other platforms, and only for accounting purposes, but to me if two scaled decimals print the same but answer false to #= then something is wrong conceptually. I can identify the issue, but the reasoning behind how it works, or even more, how to fix it, is above my level. > There *is* point in having fixed-point decimal numbers as they are a perfect match for SQL data bases and several programming languages. That's the only use I gave it. So if we divided 10.0s1 / 3 ended up with 3.3s1 each, and we had to calculate the rounding errors (0.1s1 in this case) to make all balance. > I've now located a PDF of the USGA Rules of Handicapping. Wow. 128 pages. That's just USGA, there is a "World System" but each "region" or local federation does a variation of the "world system". So Belgium does one thing, Australia another one, Argentina another one, England decided to only take one part... standards, they call it :-) I Implemented a couple of them. > And with all that they couldn't explain the calculations precisely. Try to ask one federation how they calculate the PCC, and they won't tell you, it's like the coca-cola formula. > Rounding > arrives in more than one place. For example you might have to average the > best 1-8 of the last (as many as are available up to 20) scores and then round > to one decimal, but it is not stated how ties are to be broken (which can arise > for the best 2, 4, 6, or 8). In this computation, the tie doesn't matter, because you start with an _already rounded_ number (called a "differential"), so it doesn't matter if you choose 11.4 from the 5th or 11.4 from the 19th. The rounding happens at the very end and determines a new rounded (to one decimal) number that's going to be used in other formulas (like the one that re-kicked this thread). The ties in competitions are solved differently. In real tournaments by playoff, or in amateur tournaments either by playoff or by a hole by hole comparison. If that even happens[*], there is the option to toss a coin to untie it. [*] In +400 tournaments managed by my software (involving ~15000 scorecards), that never happened. So to wrap up and bring it back to the topic, a proper ScaledDecimal implementation would help me storing the numbers as such guaranteeing no floating point funny thing happens, but since there is no such thing, I'll rather build the scaled decimals manually (e.g. ScaledDecimal fromFraction: 67/10) or not use scaled decimals and try to use the Fractions directly wherever possible. It's always good to learn new things.
EM
Esteban Maringolo
Wed, Jun 16, 2021 6:23 PM

On Wed, Jun 16, 2021 at 10:52 AM Sven Van Caekenberghe sven@stfx.eu wrote:

I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.

I think that going for 1/10 fractions/precision is not going to help: you got a division by 113 in your formula [https://en.wikipedia.org/wiki/Handicap_(golf)], this will give smaller fractions.

And the reasons behind why they chose 113 are still unknown, because
that number is used to get a
"difficulty coefficient", so if the course is rated a 125, then
125/113 will give you roughly a ~1.1 coefficient.
They could have used 100.

The problem is not the calculation (modern 64-bit floats as in Pharo are plenty accurate), it is how you handle results. You should just round it correctly and be done with it.

Note that
a roundTo: 0.00000000000001. (1e-14) still gives 4.5
it is only one step further that you hit the limit and get the ugly but correct result.

I'm not doing that, and I only store the original number (which has a
fixed decimal) and the "rounded" (to no decimals) number, all
intermediate numbers are re-calculated (maybe cached) but not stored
elsewhere. What I might be doing wrong is to store the original index
as float instead of a Decimal number.

When computing with money, you would be inclined to put everything in cents (because you cannot divide them further). But once you start computing percentage discounts or taxes, you again get problems. At each of those steps you must make sure that no cents are lost.

And that includes adding a "rounding" item for any difference in all
invoices, orders, etc.

Regards!

Esteban A. Maringolo

On Wed, Jun 16, 2021 at 10:52 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. > > I think that going for 1/10 fractions/precision is not going to help: you got a division by 113 in your formula [https://en.wikipedia.org/wiki/Handicap_(golf)], this will give smaller fractions. And the reasons behind why they chose 113 are still unknown, because that number is used to get a "difficulty coefficient", so if the course is rated a 125, then 125/113 will give you roughly a ~1.1 coefficient. They could have used 100. > The problem is not the calculation (modern 64-bit floats as in Pharo are plenty accurate), it is how you handle results. You should just round it correctly and be done with it. > Note that > a roundTo: 0.00000000000001. (1e-14) still gives 4.5 > it is only one step further that you hit the limit and get the ugly but correct result. I'm not doing that, and I only store the original number (which has a fixed decimal) and the "rounded" (to no decimals) number, all intermediate numbers are re-calculated (maybe cached) but not stored elsewhere. What I might be doing wrong is to store the original index as float instead of a Decimal number. > When computing with money, you would be inclined to put everything in cents (because you cannot divide them further). But once you start computing percentage discounts or taxes, you again get problems. At each of those steps you must make sure that no cents are lost. And that includes adding a "rounding" item for any difference in all invoices, orders, etc. Regards! Esteban A. Maringolo
EM
Esteban Maringolo
Wed, Jun 16, 2021 6:28 PM

I didn't know there were decimal floats.

Are they used elsewhere?

Esteban A. Maringolo

On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen
konrad.hinsen@fastmail.net wrote:

On 16/06/2021 15:52, Sven Van Caekenberghe wrote:

I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.

Handheld calculators use decimal floats, not binary floats. That doesn't
remove rounding issues, but it makes conversion to and from print
representations loss-free.

Konrad

I didn't know there were decimal floats. Are they used elsewhere? Esteban A. Maringolo On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen <konrad.hinsen@fastmail.net> wrote: > > On 16/06/2021 15:52, Sven Van Caekenberghe wrote: > > I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. > > Handheld calculators use decimal floats, not binary floats. That doesn't > remove rounding issues, but it makes conversion to and from print > representations loss-free. > > > Konrad
SV
Sven Van Caekenberghe
Wed, Jun 16, 2021 6:45 PM

On 16 Jun 2021, at 16:20, Konrad Hinsen konrad.hinsen@fastmail.net wrote:

On 16/06/2021 15:52, Sven Van Caekenberghe wrote:

I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.

Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free.

Konrad

mmm, this is interesting.

It would be possible (and maybe it has already been done) to implement/add such a decimal floating point number to Pharo. It would require reimplementing all operations from scratch (esp. sin/cos/tan log/exp and so on), it would be slow, but interesting.

> On 16 Jun 2021, at 16:20, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote: > > On 16/06/2021 15:52, Sven Van Caekenberghe wrote: >> I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. > > Handheld calculators use decimal floats, not binary floats. That doesn't remove rounding issues, but it makes conversion to and from print representations loss-free. > > > Konrad > mmm, this is interesting. It would be possible (and maybe it has already been done) to implement/add such a decimal floating point number to Pharo. It would require reimplementing all operations from scratch (esp. sin/cos/tan log/exp and so on), it would be slow, but interesting.
KH
Konrad Hinsen
Wed, Jun 16, 2021 7:06 PM

Sven Van Caekenberghe sven@stfx.eu writes:

It would be possible (and maybe it has already been done) to
implement/add such a decimal floating point number to Pharo. It would
require reimplementing all operations from scratch (esp. sin/cos/tan
log/exp and so on), it would be slow, but interesting.

That would be an interesting experiment indeed. Either use one of the
IEEE754 decimal float formats
(https://en.wikipedia.org/wiki/Decimal_floating_point), or Douglas
Crockford's DEC64 proposal (https://www.crockford.com/dec64.html, which
strangely enough does not refer to the older IEEE standard). There are
implementations of both for various other languages that can serve as
inspiration.

Konrad.

Sven Van Caekenberghe <sven@stfx.eu> writes: > It would be possible (and maybe it has already been done) to > implement/add such a decimal floating point number to Pharo. It would > require reimplementing all operations from scratch (esp. sin/cos/tan > log/exp and so on), it would be slow, but interesting. That would be an interesting experiment indeed. Either use one of the IEEE754 decimal float formats (https://en.wikipedia.org/wiki/Decimal_floating_point), or Douglas Crockford's DEC64 proposal (https://www.crockford.com/dec64.html, which strangely enough does not refer to the older IEEE standard). There are implementations of both for various other languages that can serve as inspiration. Konrad.
D
ducasse
Wed, Jun 16, 2021 8:44 PM

Richard

As I said it thousands of times,
- first we are not good in math
- second Pharo is what we have and we never said that it is the best system ever
we just enhanced daily
- third, I never saw a piece of code from you.
- if you want to see Pharo improve (which I doubt about) just send us code and tests

I’m sad that you systematically ignore any of my emails on the topic.
You could have an impact but you prefer to hide yourselves in your tower. Good luck with it.

S.

Richard As I said it thousands of times, - first we are not good in math - second Pharo is what we have and we never said that it is the best system ever we just enhanced daily - third, I never saw a piece of code from you. - if you want to see Pharo improve (which I doubt about) just send us code and tests I’m sad that you systematically ignore any of my emails on the topic. You could have an impact but you prefer to hide yourselves in your tower. Good luck with it. S.
RO
Richard O'Keefe
Thu, Jun 17, 2021 12:41 AM

2Decimal floats are (a) part of the IEEE 758-2008 standard for
floating-point arithmetic,
(b) implemented as part of the instruction set in IBM z/Series and
POWER computers,
(c) allowed for in the current C standard, (d) partially supported by GCC
https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Decimal-Float.html
and (e) available for most machines via the "reference implementation" in C.
The current COBOL standard basically defines its arithmetic in terms of this.
Java of course has BigDecimal.  While decimal floats became part of IEEE 754 in
2008, they were previously covered by IEEE 854.

Reverting to ScaledDecimal, let us consider the 10.0s1 / 3 example.
There is a huge problem here with no really satisfactory answer.
In my library, it currently answers 3.3s1, exactly equal to 33/10.
In Squeak/Pharo, it answers "I am really 10/3 but print me with one decimal",
so you see 3.3a1 but it behaves like 10.3.
In VisualAge Smalltalk you get  3.333333333333333333333333333333
If it were not for the fact that the ANSI Smalltalk standard appears to
require that <scaled decimal> / <integer> yield a <scaled decimal>,
I would prefer to answer 10/3.  Decimals just plain are not closed under
division (although they are closed under addition, subtraction, and
multiplication).  When it comes to scaled decimals, the ANSI Smalltalk
standard is frankly a mess.  Amongst other things, it defers the
semantics of operations on them to the Language-Independent
Arithmetic standard, which explicitly denies having anything to say about them!

I understand why my library does what it does.  I even tried to
publish a paper about it.
I understand why VisualAge Smalltalk does what it does.  (I've read
IBM/360 PrincOps.)
I do NOT understand why Squeak and Pharo do what they do, and if anyone knows
the rationale for their ScaledDecmal please tell me.

On Thu, 17 Jun 2021 at 06:29, Esteban Maringolo emaringolo@gmail.com wrote:

I didn't know there were decimal floats.

Are they used elsewhere?

Esteban A. Maringolo

On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen
konrad.hinsen@fastmail.net wrote:

On 16/06/2021 15:52, Sven Van Caekenberghe wrote:

I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not.

Handheld calculators use decimal floats, not binary floats. That doesn't
remove rounding issues, but it makes conversion to and from print
representations loss-free.

Konrad

2Decimal floats are (a) part of the IEEE 758-2008 standard for floating-point arithmetic, (b) implemented as part of the instruction set in IBM z/Series and POWER computers, (c) allowed for in the current C standard, (d) partially supported by GCC https://gcc.gnu.org/onlinedocs/gcc-8.1.0/gcc/Decimal-Float.html and (e) available for most machines via the "reference implementation" in C. The current COBOL standard basically defines its arithmetic in terms of this. Java of course has BigDecimal. While decimal floats became part of IEEE 754 in 2008, they were previously covered by IEEE 854. Reverting to ScaledDecimal, let us consider the 10.0s1 / 3 example. There is a huge problem here with no really satisfactory answer. In my library, it currently answers 3.3s1, exactly equal to 33/10. In Squeak/Pharo, it answers "I am really 10/3 but print me with one decimal", so you see 3.3a1 but it *behaves* like 10.3. In VisualAge Smalltalk you get 3.333333333333333333333333333333 If it were not for the fact that the ANSI Smalltalk standard appears to require that <scaled decimal> / <integer> yield a <scaled decimal>, I would prefer to answer 10/3. Decimals just plain are not closed under division (although they are closed under addition, subtraction, and multiplication). When it comes to scaled decimals, the ANSI Smalltalk standard is frankly a mess. Amongst other things, it defers the semantics of operations on them to the Language-Independent Arithmetic standard, which explicitly denies having anything to say about them! I understand why my library does what it does. I even tried to publish a paper about it. I understand why VisualAge Smalltalk does what it does. (I've read IBM/360 PrincOps.) I do NOT understand why Squeak and Pharo do what they do, and if anyone knows the rationale for their ScaledDecmal please tell me. On Thu, 17 Jun 2021 at 06:29, Esteban Maringolo <emaringolo@gmail.com> wrote: > > I didn't know there were decimal floats. > > Are they used elsewhere? > > Esteban A. Maringolo > > On Wed, Jun 16, 2021 at 11:20 AM Konrad Hinsen > <konrad.hinsen@fastmail.net> wrote: > > > > On 16/06/2021 15:52, Sven Van Caekenberghe wrote: > > > I am also a bit intrigued by this. Like you said: several other programming languages (I tried a couple of Common Lisp and Scheme implementations) do the same as Pharo, but handheld calculators, normal and scientific, do not. > > > > Handheld calculators use decimal floats, not binary floats. That doesn't > > remove rounding issues, but it makes conversion to and from print > > representations loss-free. > > > > > > Konrad
RO
Richard O'Keefe
Thu, Jun 17, 2021 12:52 AM

"first we are not good in math"
First, I am not criticising YOU.  Pharo's weird ScaledDecimal is
Squeak's weird ScaledDecimal
and you have not changed it.  Fine.
"second Pharo is what we have and we never said that it is the best
system ever"
True, but irrelevant.
"third I never saw a piece of code from you"
That is not true.  Why, in this thread alone I contributed
#roundOut.  I have provided several
pieces of code in this mailing list over the years. You can say that
my code is no good, or
that you don't want it, or that you think there is a better way to
do the (admittedly small)
things that I have provided, but if you never saw them you must have
shut your eyes.
In addition, a snapshot of my system is available on the web for
anyone to play with.
I shall post here when my GitHub repository is populated; I just
have to clear a few more
things out of the local directory that I don't have the right to distribute.
(For example, benchmarking my CSV library against NeoCSV meant having to have
NeoCSV, but it's not mine to distribute.)

On Thu, 17 Jun 2021 at 08:45, ducasse stepharo@netcourrier.com wrote:

Richard

As I said it thousands of times,
- first we are not good in math
- second Pharo is what we have and we never said that it is the best system ever
we just enhanced daily
- third, I never saw a piece of code from you.
- if you want to see Pharo improve (which I doubt about) just send us code and tests

I’m sad that you systematically ignore any of my emails on the topic.
You could have an impact but you prefer to hide yourselves in your tower. Good luck with it.

S.

"first we are not good in math" First, I am not criticising YOU. Pharo's weird ScaledDecimal is Squeak's weird ScaledDecimal and you have not changed it. Fine. "second Pharo is what we have and we never said that it is the best system ever" True, but irrelevant. "third I never saw a piece of code from you" That is not true. Why, in this thread alone I contributed #roundOut. I have provided several pieces of code in this mailing list over the years. You can say that my code is no good, or that you don't want it, or that you think there is a better way to do the (admittedly small) things that I have provided, but if you never saw them you must have shut your eyes. In addition, a snapshot of my system is available on the web for anyone to play with. I shall post here when my GitHub repository is populated; I just have to clear a few more things out of the local directory that I don't have the right to distribute. (For example, benchmarking my CSV library against NeoCSV meant having to have NeoCSV, but it's not mine to distribute.) On Thu, 17 Jun 2021 at 08:45, ducasse <stepharo@netcourrier.com> wrote: > > Richard > > As I said it thousands of times, > - first we are not good in math > - second Pharo is what we have and we never said that it is the best system ever > we just enhanced daily > - third, I never saw a piece of code from you. > - if you want to see Pharo improve (which I doubt about) just send us code and tests > > I’m sad that you systematically ignore any of my emails on the topic. > You could have an impact but you prefer to hide yourselves in your tower. Good luck with it. > > S.
L
Lorenzo
Thu, Jun 17, 2021 6:33 AM

You are right!

Lorenzo

-----Messaggio originale-----
Da: ducasse [mailto:stepharo@netcourrier.com]
Inviato: mercoledì 16 giugno 2021 22:44
A: Any question about pharo is welcome pharo-users@lists.pharo.org
Oggetto: [Pharo-users] Re: Rounding in Floats

Richard

As I said it thousands of times,
- first we are not good in math
- second Pharo is what we have and we never said that it is the best system ever
we just enhanced daily
- third, I never saw a piece of code from you.
- if you want to see Pharo improve (which I doubt about) just send us code and tests

I’m sad that you systematically ignore any of my emails on the topic.
You could have an impact but you prefer to hide yourselves in your tower. Good luck with it.

S.

You are right! Lorenzo -----Messaggio originale----- Da: ducasse [mailto:stepharo@netcourrier.com] Inviato: mercoledì 16 giugno 2021 22:44 A: Any question about pharo is welcome <pharo-users@lists.pharo.org> Oggetto: [Pharo-users] Re: Rounding in Floats Richard As I said it thousands of times, - first we are not good in math - second Pharo is what we have and we never said that it is the best system ever we just enhanced daily - third, I never saw a piece of code from you. - if you want to see Pharo improve (which I doubt about) just send us code and tests I’m sad that you systematically ignore any of my emails on the topic. You could have an impact but you prefer to hide yourselves in your tower. Good luck with it. S.