pharo-users@lists.pharo.org

Any question about pharo is welcome

View all threads

Rounding in Floats

EM
Esteban Maringolo
Sun, Sep 6, 2020 2:06 PM

Hi,

Continuing with my issues with decimals, I'm having one issue that is
not clear to me why it happens.

If I do:
(9.1 + (-2.0)) roundTo: 0.1.
"7.1000000000000005"

I expect to get a single decimal Float (rounded with whatever
precision, but a single decimal).

Even if I do something like this:
7.1 roundTo: 0.1

It gives the wrong result.

In VW and VAST it provides the right result.
(9.1 + (-2.0)) roundTo: 0.1 "7.1"

In Dolphin it also returns the wrong result, it seems to use the same
algorithm to round it.

Is this a bug?

Esteban A. Maringolo

Hi, Continuing with my issues with decimals, I'm having one issue that is not clear to me why it happens. If I do: (9.1 + (-2.0)) roundTo: 0.1. "7.1000000000000005" I expect to get a single decimal Float (rounded with whatever precision, but a single decimal). Even if I do something like this: 7.1 roundTo: 0.1 It gives the wrong result. In VW and VAST it provides the right result. (9.1 + (-2.0)) roundTo: 0.1 "7.1" In Dolphin it also returns the wrong result, it seems to use the same algorithm to round it. Is this a bug? Esteban A. Maringolo
SV
Sven Van Caekenberghe
Sun, Sep 6, 2020 2:56 PM

On 6 Sep 2020, at 16:06, Esteban Maringolo emaringolo@gmail.com wrote:

Hi,

Continuing with my issues with decimals, I'm having one issue that is
not clear to me why it happens.

If I do:
(9.1 + (-2.0)) roundTo: 0.1.
"7.1000000000000005"

I expect to get a single decimal Float (rounded with whatever
precision, but a single decimal).

Even if I do something like this:
7.1 roundTo: 0.1

It gives the wrong result.

In VW and VAST it provides the right result.
(9.1 + (-2.0)) roundTo: 0.1 "7.1"

In Dolphin it also returns the wrong result, it seems to use the same
algorithm to round it.

Is this a bug?

Maybe.

But I would not approach the problem of rounding like that.
You probably want to control how numbers are printed.
I would keep the numbers themselves at maximum internal precision and only do something when printing them.

1 / 3 asFloat printShowingDecimalPlaces: 1.

Since you like Seaside, the following is even much more powerful (has many options):

GRNumberPrinter new precision: 1; print: 1/3 asFloat.

Check it out.

Esteban A. Maringolo

> On 6 Sep 2020, at 16:06, Esteban Maringolo <emaringolo@gmail.com> wrote: > > Hi, > > Continuing with my issues with decimals, I'm having one issue that is > not clear to me why it happens. > > If I do: > (9.1 + (-2.0)) roundTo: 0.1. > "7.1000000000000005" > > I expect to get a single decimal Float (rounded with whatever > precision, but a single decimal). > > Even if I do something like this: > 7.1 roundTo: 0.1 > > It gives the wrong result. > > In VW and VAST it provides the right result. > (9.1 + (-2.0)) roundTo: 0.1 "7.1" > > In Dolphin it also returns the wrong result, it seems to use the same > algorithm to round it. > > Is this a bug? Maybe. But I would not approach the problem of rounding like that. You probably want to control how numbers are printed. I would keep the numbers themselves at maximum internal precision and only do something when printing them. 1 / 3 asFloat printShowingDecimalPlaces: 1. Since you like Seaside, the following is even much more powerful (has *many* options): GRNumberPrinter new precision: 1; print: 1/3 asFloat. Check it out. > > Esteban A. Maringolo >
EM
Esteban Maringolo
Sun, Sep 6, 2020 8:21 PM

Hi Sven,

On Sun, Sep 6, 2020 at 11:57 AM Sven Van Caekenberghe sven@stfx.eu wrote:

On 6 Sep 2020, at 16:06, Esteban Maringolo emaringolo@gmail.com wrote:

(9.1 + (-2.0)) roundTo: 0.1.
"7.1000000000000005"

Is this a bug?

Maybe.

But I would not approach the problem of rounding like that.
You probably want to control how numbers are printed.

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.
And then it fails because of the difference above when it is
"semantically" correct.

I would keep the numbers themselves at maximum internal precision and only do something when printing them.

I do. I'm implementing the full WHS specification [1], and it mentions
that most calculations should preserve the maximum internal precision
and even introduces some weird situation where you have an "index"
that is rounded and another one that is called the same way but must
not be rounded because is going to used to compute some other value.

For most calculations it specifies it must round to 1 decimal or even
to no decimal, based on a round up if the decimal is greater or equal
than zero (e.g. 1.5 becomes 2), except for negative numbers, in which
case it rounds "down" (-1.5 becomes 1).

1 / 3 asFloat printShowingDecimalPlaces: 1.

Since you like Seaside, the following is even much more powerful (has many options):
GRNumberPrinter new precision: 1; print: 1/3 asFloat.

I'm already using the GRNumberPrinter and also the Date printer to
print these values.

Regards!

[1] https://en.wikipedia.org/wiki/Handicap_(golf)#World_Handicap_System

Hi Sven, On Sun, Sep 6, 2020 at 11:57 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > On 6 Sep 2020, at 16:06, Esteban Maringolo <emaringolo@gmail.com> wrote: > > (9.1 + (-2.0)) roundTo: 0.1. > > "7.1000000000000005" > > Is this a bug? > > Maybe. > > But I would not approach the problem of rounding like that. > You probably want to control how numbers are printed. It is not for printing but for testing. I want to assert that a certain calculation gives the expected result. And then it fails because of the difference above when it is "semantically" correct. > I would keep the numbers themselves at maximum internal precision and only do something when printing them. I do. I'm implementing the full WHS specification [1], and it mentions that most calculations should preserve the maximum internal precision and even introduces some weird situation where you have an "index" that is rounded and another one that is called the same way but must not be rounded because is going to used to compute some other value. For most calculations it specifies it must round to 1 decimal or even to no decimal, based on a round up if the decimal is greater or equal than zero (e.g. 1.5 becomes 2), except for negative numbers, in which case it rounds "down" (-1.5 becomes 1). > 1 / 3 asFloat printShowingDecimalPlaces: 1. > > Since you like Seaside, the following is even much more powerful (has *many* options): > GRNumberPrinter new precision: 1; print: 1/3 asFloat. I'm already using the GRNumberPrinter and also the Date printer to print these values. Regards! [1] https://en.wikipedia.org/wiki/Handicap_(golf)#World_Handicap_System
KH
Konrad Hinsen
Mon, Sep 7, 2020 8:50 AM

Hi Esteban,

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.
And then it fails because of the difference above when it is
"semantically" correct.

If you want reliable precision tests with IEEE floats, you should use
rounding to powers of 2, not 10. Round to 1/8 or 1/16, but not to 1/10.

The fundamental issue is that 1/10 does not have an exact finite
representation in base 2. No matter how I/O libraries handle the
conversion between base 2 (in the binary representation of IEEE floats)
and base 10 (in the text representation), there will always be bad
surprises because the expectations we have from working in base 10
cannot all be met with an internal representation in base 2.

So if your goal is not fundamentally related to I/O, forget about I/O
and design your code to work the binary level, in particular for
testing.

However, if your problem specification explicitly requires precision
guarantees in base 10 (which seems to be your case, from a quick glance
at the Wikipedia page you cite), then it's best to avoid binary floats
completely. Rationals (fractions) are one option, scaled integers
another one.

Konrad.

Hi Esteban, > It is not for printing but for testing. I want to assert that a > certain calculation gives the expected result. > And then it fails because of the difference above when it is > "semantically" correct. If you want reliable precision tests with IEEE floats, you should use rounding to powers of 2, not 10. Round to 1/8 or 1/16, but not to 1/10. The fundamental issue is that 1/10 does not have an exact finite representation in base 2. No matter how I/O libraries handle the conversion between base 2 (in the binary representation of IEEE floats) and base 10 (in the text representation), there will always be bad surprises because the expectations we have from working in base 10 cannot all be met with an internal representation in base 2. So if your goal is not fundamentally related to I/O, forget about I/O and design your code to work the binary level, in particular for testing. However, if your problem specification explicitly requires precision guarantees in base 10 (which seems to be your case, from a quick glance at the Wikipedia page you cite), then it's best to avoid binary floats completely. Rationals (fractions) are one option, scaled integers another one. Konrad.
SV
Sven Van Caekenberghe
Mon, Sep 7, 2020 12:10 PM

On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.

(9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001.

Floats should always be compared using an epsilon (precision) value in tests, not using equality.

> On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote: > > It is not for printing but for testing. I want to assert that a > certain calculation gives the expected result. Then you should use #assert:closeTo: and friends. (9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001. Floats should always be compared using an epsilon (precision) value in tests, not using equality.
SS
serge.stinckwich@gmail.com
Mon, Sep 7, 2020 12:23 PM

BTW why closeTo: is initialized by default with a value of 0.0001 ?
https://github.com/pharo-project/pharo/issues/3067

Sent from my iPhone

On 7 Sep 2020, at 20:10, Sven Van Caekenberghe sven@stfx.eu wrote:



On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.

(9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001.

Floats should always be compared using an epsilon (precision) value in tests, not using equality.

BTW why closeTo: is initialized by default with a value of 0.0001 ? https://github.com/pharo-project/pharo/issues/3067 Sent from my iPhone > On 7 Sep 2020, at 20:10, Sven Van Caekenberghe <sven@stfx.eu> wrote: > >  > >> On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote: >> >> It is not for printing but for testing. I want to assert that a >> certain calculation gives the expected result. > > Then you should use #assert:closeTo: and friends. > > (9.1 + (-2.0)) closeTo: 7.1 precision: 0.00001. > > Floats should always be compared using an epsilon (precision) value in tests, not using equality. > >
EM
Esteban Maringolo
Mon, Sep 7, 2020 12:27 PM

Hi Sven,

On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe sven@stfx.eu wrote:

On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.
(9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.

I remembered about it, and implemented my own
#assertDifferential:equals: which I first did convert to
ScaledDecimals and then compares with #roundedByScale.

Floats should always be compared using an epsilon (precision) value in tests, not using equality.

Sure, but 7.1 roundTo: 0.1 should return 7.1.  Shouldn't it?

Regards!

Esteban A. Maringolo

Hi Sven, On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote: > > > > It is not for printing but for testing. I want to assert that a > > certain calculation gives the expected result. > > Then you should use #assert:closeTo: and friends. > (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1. I remembered about it, and implemented my own #assertDifferential:equals: which I first did convert to ScaledDecimals and then compares with #roundedByScale. > Floats should always be compared using an epsilon (precision) value in tests, not using equality. Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it? Regards! Esteban A. Maringolo
TO
Tomohiro Oda
Mon, Sep 7, 2020 12:50 PM

Esteban,

You can instead use 7.1 round: 1 to get 7.1.
It's not the problem of the rounding algorithm.
It's because IEEE float can't express the exact value of 0.1.

tomo

2020年9月7日(月) 21:28 Esteban Maringolo emaringolo@gmail.com:

Hi Sven,

On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe sven@stfx.eu wrote:

On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.
(9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.

I remembered about it, and implemented my own
#assertDifferential:equals: which I first did convert to
ScaledDecimals and then compares with #roundedByScale.

Floats should always be compared using an epsilon (precision) value in tests, not using equality.

Sure, but 7.1 roundTo: 0.1 should return 7.1.  Shouldn't it?

Regards!

Esteban A. Maringolo

Esteban, You can instead use `7.1 round: 1` to get 7.1. It's not the problem of the rounding algorithm. It's because IEEE float can't express the exact value of 0.1. --- tomo 2020年9月7日(月) 21:28 Esteban Maringolo <emaringolo@gmail.com>: > > Hi Sven, > > On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > > > On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote: > > > > > > It is not for printing but for testing. I want to assert that a > > > certain calculation gives the expected result. > > > > Then you should use #assert:closeTo: and friends. > > (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1. > > I remembered about it, and implemented my own > #assertDifferential:equals: which I first did convert to > ScaledDecimals and then compares with #roundedByScale. > > > Floats should always be compared using an epsilon (precision) value in tests, not using equality. > > Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it? > > Regards! > > Esteban A. Maringolo >
EM
Esteban Maringolo
Mon, Sep 7, 2020 12:54 PM

Hi Tomohiro,

Thanks, that certainly is the simplest solution that does exactly what I need.

Maybe roundTo: should be considered harmful when using Floats.
(and it is very unlikely anybody is going to round to something other
than a power of 10)

Best regards!

Esteban A. Maringolo

On Mon, Sep 7, 2020 at 9:51 AM Tomohiro Oda tomohiro.tomo.oda@gmail.com wrote:

Esteban,

You can instead use 7.1 round: 1 to get 7.1.
It's not the problem of the rounding algorithm.
It's because IEEE float can't express the exact value of 0.1.

tomo

2020年9月7日(月) 21:28 Esteban Maringolo emaringolo@gmail.com:

Hi Sven,

On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe sven@stfx.eu wrote:

On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.
(9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.

I remembered about it, and implemented my own
#assertDifferential:equals: which I first did convert to
ScaledDecimals and then compares with #roundedByScale.

Floats should always be compared using an epsilon (precision) value in tests, not using equality.

Sure, but 7.1 roundTo: 0.1 should return 7.1.  Shouldn't it?

Regards!

Esteban A. Maringolo

Hi Tomohiro, Thanks, that certainly is the simplest solution that does exactly what I need. Maybe roundTo: should be considered harmful when using Floats. (and it is very unlikely anybody is going to round to something other than a power of 10) Best regards! Esteban A. Maringolo On Mon, Sep 7, 2020 at 9:51 AM Tomohiro Oda <tomohiro.tomo.oda@gmail.com> wrote: > > Esteban, > > You can instead use `7.1 round: 1` to get 7.1. > It's not the problem of the rounding algorithm. > It's because IEEE float can't express the exact value of 0.1. > --- > tomo > > 2020年9月7日(月) 21:28 Esteban Maringolo <emaringolo@gmail.com>: > > > > Hi Sven, > > > > On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > > > > > On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> wrote: > > > > > > > > It is not for printing but for testing. I want to assert that a > > > > certain calculation gives the expected result. > > > > > > Then you should use #assert:closeTo: and friends. > > > (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1. > > > > I remembered about it, and implemented my own > > #assertDifferential:equals: which I first did convert to > > ScaledDecimals and then compares with #roundedByScale. > > > > > Floats should always be compared using an epsilon (precision) value in tests, not using equality. > > > > Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it? > > > > Regards! > > > > Esteban A. Maringolo > > >
RO
Richard O'Keefe
Tue, Sep 8, 2020 3:15 AM

"7.1 roundTo: 0.1 should return 7.1"
You're still not getting it.
7.1 IS NOT 71/10.
0.1 IS NOT 1/10.
Binary floating point CANNOT represent either of those numbers.
You seem to be assuming that Pharo is making some mistake.
It isn't.  All it is doing is refusing to lie to you.

#include <math.h>
#include <stdio.h>

int main(void) {
printf("%.18f\n", 7.1);
printf("%.18f\n", round(7.1 / 0.1) * 0.1);
return 0;
}

% a.out
7.099999999999999645
7.100000000000000533

Does this help?  "7.1" is a bit LESS than 7.1.
"7.1 roundTo: 0.1" is a bit MORE than 7.1.
This is the best that the computer's hardware can do.
The systems that print 7.1 are LYING to you,
and Pharo is not.

On Tue, 8 Sep 2020 at 00:28, Esteban Maringolo emaringolo@gmail.com wrote:

Hi Sven,

On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe sven@stfx.eu wrote:

On 6 Sep 2020, at 22:21, Esteban Maringolo emaringolo@gmail.com

wrote:

It is not for printing but for testing. I want to assert that a
certain calculation gives the expected result.

Then you should use #assert:closeTo: and friends.
(9.1 + (-2.0)) closeTo: 7.1 precision: 0.1.

I remembered about it, and implemented my own
#assertDifferential:equals: which I first did convert to
ScaledDecimals and then compares with #roundedByScale.

Floats should always be compared using an epsilon (precision) value in

tests, not using equality.

Sure, but 7.1 roundTo: 0.1 should return 7.1.  Shouldn't it?

Regards!

Esteban A. Maringolo

"7.1 roundTo: 0.1 should return 7.1" You're still not getting it. 7.1 IS NOT 71/10. 0.1 IS NOT 1/10. Binary floating point CANNOT represent either of those numbers. You seem to be assuming that Pharo is making some mistake. It isn't. All it is doing is refusing to lie to you. #include <math.h> #include <stdio.h> int main(void) { printf("%.18f\n", 7.1); printf("%.18f\n", round(7.1 / 0.1) * 0.1); return 0; } % a.out 7.099999999999999645 7.100000000000000533 Does this help? "7.1" is a bit LESS than 7.1. "7.1 roundTo: 0.1" is a bit MORE than 7.1. This is the best that the computer's hardware can do. The systems that print 7.1 are LYING to you, and Pharo is not. On Tue, 8 Sep 2020 at 00:28, Esteban Maringolo <emaringolo@gmail.com> wrote: > Hi Sven, > > On Mon, Sep 7, 2020 at 9:10 AM Sven Van Caekenberghe <sven@stfx.eu> wrote: > > > > On 6 Sep 2020, at 22:21, Esteban Maringolo <emaringolo@gmail.com> > wrote: > > > > > > It is not for printing but for testing. I want to assert that a > > > certain calculation gives the expected result. > > > > Then you should use #assert:closeTo: and friends. > > (9.1 + (-2.0)) closeTo: 7.1 precision: 0.1. > > I remembered about it, and implemented my own > #assertDifferential:equals: which I first did convert to > ScaledDecimals and then compares with #roundedByScale. > > > Floats should always be compared using an epsilon (precision) value in > tests, not using equality. > > Sure, but 7.1 roundTo: 0.1 should return 7.1. Shouldn't it? > > Regards! > > Esteban A. Maringolo > >