Hi, using formatted number output recently I got the following output: 1.0 printPaddedWith: $0 to: 2.1. "'01.0'" 1.0 printPaddedWith: $0 to: 2.2. "'01.00'" 1.0 printPaddedWith: $0 to: 2.3. "'01.00'" <-- wrong 1.0 printPaddedWith: $0 to: 2.4. "'01.000'" <-- wrong 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'" running Float>>printPaddedWith:to: in the debugger showed me that the reason of this odd behaviour seems to be produced by Float>>truncated and/or Float>>fractionPart. I can provoke errors of that kind even easier: (2.3 fractionPart * 10) truncated. => "2" (instead of 3) In Primitive 51 (primitiveTruncated for Float>>truncated) I read frac = modf(rcvr, &trunc); so the problem looks like an issue with rounding errors in Math-Library's modf function. Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10) truncated." in C++ using modf: // g++ -lm truncate.cpp && ./a.out #include<iostream> #include<cmath> using namespace std; int main(){ double f1, f2, f3; modf(10 * 0.3, &f1); cout << f1 << endl; modf(10 * modf(2.3, &f2), &f3); cout << f3 << endl; } // output: // 3 (ok) // 2 (error - should be 3) Question: what can be done about this? As a matter of fact, several important methods spread over a number of classes (Float, Number, Morph ... being among them) are senders of Float>>truncated or Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke unexpected behaviour in surprising places. -- View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Thanks thanks for your report. I do not have the answer but this is important to get aware of that. Stef On Feb 27, 2011, at 10:08 PM, t olschewski wrote:
Hi, using formatted number output recently I got the following output:
1.0 printPaddedWith: $0 to: 2.1. "'01.0'" 1.0 printPaddedWith: $0 to: 2.2. "'01.00'" 1.0 printPaddedWith: $0 to: 2.3. "'01.00'" <-- wrong 1.0 printPaddedWith: $0 to: 2.4. "'01.000'" <-- wrong 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
running Float>>printPaddedWith:to: in the debugger showed me that the reason of this odd behaviour seems to be produced by Float>>truncated and/or Float>>fractionPart. I can provoke errors of that kind even easier:
(2.3 fractionPart * 10) truncated. => "2" (instead of 3)
In Primitive 51 (primitiveTruncated for Float>>truncated) I read
frac = modf(rcvr, &trunc);
so the problem looks like an issue with rounding errors in Math-Library's modf function.
Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10) truncated." in C++ using modf:
// g++ -lm truncate.cpp && ./a.out #include<iostream> #include<cmath> using namespace std; int main(){ double f1, f2, f3; modf(10 * 0.3, &f1); cout << f1 << endl; modf(10 * modf(2.3, &f2), &f3); cout << f3 << endl; } // output: // 3 (ok) // 2 (error - should be 3)
Question: what can be done about this?
As a matter of fact, several important methods spread over a number of classes (Float, Number, Morph ... being among them) are senders of Float>>truncated or Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke unexpected behaviour in surprising places.
-- View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Good catch! It looks to me like the root cause is that the argument 2.3 is being treated as a floating point value, whereas in this context it is really intended as a format specifier (i.e. a string, such as you might use in a sprintf() function). I wrote a test to document this bug, and added the fix to Squeak trunk. The test and the modified #printPaddedWith:to: method are in the attached change set. Dave
Hi, using formatted number output recently I got the following output:
1.0 printPaddedWith: $0 to: 2.1. "'01.0'" 1.0 printPaddedWith: $0 to: 2.2. "'01.00'" 1.0 printPaddedWith: $0 to: 2.3. "'01.00'" <-- wrong 1.0 printPaddedWith: $0 to: 2.4. "'01.000'" <-- wrong 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
running Float>>printPaddedWith:to: in the debugger showed me that the reason of this odd behaviour seems to be produced by Float>>truncated and/or Float>>fractionPart. I can provoke errors of that kind even easier:
(2.3 fractionPart * 10) truncated. => "2" (instead of 3)
In Primitive 51 (primitiveTruncated for Float>>truncated) I read
frac = modf(rcvr, &trunc);
so the problem looks like an issue with rounding errors in Math-Library's modf function.
Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10) truncated." in C++ using modf:
// g++ -lm truncate.cpp && ./a.out #include<iostream> #include<cmath> using namespace std; int main(){ double f1, f2, f3; modf(10 * 0.3, &f1); cout << f1 << endl; modf(10 * modf(2.3, &f2), &f3); cout << f3 << endl; } // output: // 3 (ok) // 2 (error - should be 3)
Question: what can be done about this?
As a matter of fact, several important methods spread over a number of classes (Float, Number, Morph ... being among them) are senders of Float>>truncated or Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke unexpected behaviour in surprising places.
-- View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Thanks david. Stef On Feb 28, 2011, at 4:09 AM, David T. Lewis wrote:
Good catch!
It looks to me like the root cause is that the argument 2.3 is being treated as a floating point value, whereas in this context it is really intended as a format specifier (i.e. a string, such as you might use in a sprintf() function).
I wrote a test to document this bug, and added the fix to Squeak trunk. The test and the modified #printPaddedWith:to: method are in the attached change set.
Dave
Hi, using formatted number output recently I got the following output:
1.0 printPaddedWith: $0 to: 2.1. "'01.0'" 1.0 printPaddedWith: $0 to: 2.2. "'01.00'" 1.0 printPaddedWith: $0 to: 2.3. "'01.00'" <-- wrong 1.0 printPaddedWith: $0 to: 2.4. "'01.000'" <-- wrong 1.0 printPaddedWith: $0 to: 2.5. "'01.00000'"
running Float>>printPaddedWith:to: in the debugger showed me that the reason of this odd behaviour seems to be produced by Float>>truncated and/or Float>>fractionPart. I can provoke errors of that kind even easier:
(2.3 fractionPart * 10) truncated. => "2" (instead of 3)
In Primitive 51 (primitiveTruncated for Float>>truncated) I read
frac = modf(rcvr, &trunc);
so the problem looks like an issue with rounding errors in Math-Library's modf function.
Indeed, I can reproduce the odd result of "(2.3 fractionPart * 10) truncated." in C++ using modf:
// g++ -lm truncate.cpp && ./a.out #include<iostream> #include<cmath> using namespace std; int main(){ double f1, f2, f3; modf(10 * 0.3, &f1); cout << f1 << endl; modf(10 * modf(2.3, &f2), &f3); cout << f3 << endl; } // output: // 3 (ok) // 2 (error - should be 3)
Question: what can be done about this?
As a matter of fact, several important methods spread over a number of classes (Float, Number, Morph ... being among them) are senders of Float>>truncated or Float>>fractionPart, so rounding errors in primitive 51 or 52 could provoke unexpected behaviour in surprising places.
-- View this message in context: http://forum.world.st/rounding-errors-tp3327130p3327130.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. <Float-printPaddedWith-dtl.1.cs>
thanks a lot. -- View this message in context: http://forum.world.st/rounding-errors-tp3327130p3329484.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
participants (3)
-
David T. Lewis -
Stéphane Ducasse -
t olschewski