Estebanâs list of blocking issues for release Pharo 6.0 (updated to April 03)
Hello, This are the things I believe still needs to be solved to be able to release Pharo 6.0: VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P) - make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it. Image: ===== - iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :) List is a lot smaller now :) If everything goes well, we can target to release next week. Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :) cheers! Esteban
2017-04-03 15:00 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P) - make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it.
Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
I will definitely help but I need more information :-) -- Pavel
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
2017-04-03 15:00 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P) - make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it.
Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
I have found one memory leak related to subscription type mixing: https://pharo.fogbugz.com/f/cases/19911 Then I had here a really huge image with 1.2 GB with all known memory leaks solved. It was leaking because of Undeclared, after cleaning of them (Undeclared removeUnreferencedKeys) it shrinked to 59 MB. In this particular case it was caused by code like this: initialize super initialize. noBindingBlock := [ objectSpace nilObject ] because noBindingBlock was undeclared, it created a reference from globals (Undeclared) to a particular context with all mess it referenced. Not sure if we should make some action like clean Undeclared before every snapshot. But the case like this is quite obscure and the information that people should beware it should be enough. Cheers, -- Pavel
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
2017-04-04 10:15 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:
initialize super initialize. noBindingBlock := [ objectSpace nilObject ]
because noBindingBlock was undeclared, it created a reference from globals (Undeclared) to a particular context with all mess it referenced.
I think system should at least skip assignment to undeclared variable. So it should never store any value. But better would be raise error on assignment. So user will immediately see where is a problem.
2017-04-04 10:31 GMT+02:00 Denis Kudriashov <dionisiydk@gmail.com>:
2017-04-04 10:15 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:
initialize super initialize. noBindingBlock := [ objectSpace nilObject ]
because noBindingBlock was undeclared, it created a reference from globals (Undeclared) to a particular context with all mess it referenced.
I think system should at least skip assignment to undeclared variable. So it should never store any value. But better would be raise error on assignment. So user will immediately see where is a problem.
The undeclared mechanism is regularly used e.g. during loading of packages. So if we will do something like this, rather not for Pharo 6 :-) -- Pavel
2017-04-04 10:40 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:
The undeclared mechanism is regularly used e.g. during loading of packages. So if we will do something like this, rather not for Pharo 6 :-)
Yes, But I am talking about runtime behaviour. It of course could fails kind of post load actions/initialization. But they will anyway fail (maybe silently) if expected variable is undeclared
Hi Pavel, On Tue, Apr 4, 2017 at 1:15 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2017-04-03 15:00 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P) - make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it.
Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
I have found one memory leak related to subscription type mixing: https://pharo.fogbugz.com/f/cases/19911
Then I had here a really huge image with 1.2 GB with all known memory leaks solved. It was leaking because of Undeclared, after cleaning of them (Undeclared removeUnreferencedKeys) it shrinked to 59 MB.
In this particular case it was caused by code like this:
initialize super initialize. noBindingBlock := [ objectSpace nilObject ]
because noBindingBlock was undeclared, it created a reference from globals (Undeclared) to a particular context with all mess it referenced.
Not sure if we should make some action like clean Undeclared before every snapshot. But the case like this is quite obscure and the information that people should beware it should be enough.
Cleaning Undeclared arbitrarily is a very bad idea. Undeclared is where we can find and eliminate Undeclared variables. Just eliminating them from Undeclared does not solve the problem. The variables are still referenced in coded. But when one removes them from Undeclared they become harder to find. Instead the right solution is a) to have tests in the standard test suite that check that Undeclared is empty. If Undeclared is not empty the test should fail. b) in any packaging or release process that produces a clean image there should be a check for Undeclared being empty, and the build should fail if it is not empty. c) Smalltalk programmers should understand how undeclared variables work in Smalltalk: Variables are declared either in some global pool such as Smalltalk, TextConstants or a class pool (the dictionary holding class variables), or as instance variables of a class, or as arguments and temporary variables of a method. These are the only variables in the language. When the compiler sees a variable name it searches for it, first amongst arguments and temporaries, second amongst instance variables, third amongst class variables, fourth amongst the class's imported pools, fifth in globals (Smalltalk) and finally in Undeclared. If the variable is not found in any of these places then a binding is created in Undeclared and that is used to implement the variable. So undeclared variables are compiled as globals in Undeclared. Whenever a variable is declared in a global pool (Smalltalk, a shared pool or a class pool) the compiler checks for a binding in Undeclared and uses that if it exists. Adding a class declares the class's binding in Smalltalk. So uses of variables before they are declared get fixed up on declaration without having to recompile the entire system. This is how the system manages circularities amongst classes. A reference to a class that has yet to be installed will be compiled as a reference to a binding in Undeclared. When the class is installed it is set as the value of the Undeclared binding, and the binding is moved from Undeclared to Smalltalk. When an instance variable is defined in a class that changes the shape of the class's instances and so the class's methods are recompiled and that will bind any undeclared references to the instance variable correctly. So an entry in Undeclared is a) an indication that somewhere there is a missing declaration b) a place where one can look for references to the undeclared binding (references in the dictionary inspector, and I hope the Glamour inspector, that opens when one inspects Undeclared) c) a simple mechanism for allowing circular references amongst classes So under no circumstances remove bindings from Undeclared unless there are no references from code to those bindings. There is a method that does this: Undeclared removeUnusedBindings HTH
Cheers, -- Pavel
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
-- _,,,^..^,,,_ best, Eliot
On Wed, Apr 5, 2017 at 2:16 AM, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Pavel,
On Tue, Apr 4, 2017 at 1:15 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2017-04-03 15:00 GMT+02:00 Esteban Lorenzano <estebanlm@gmail.com>:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P) - make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it.
Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
I have found one memory leak related to subscription type mixing: https://pharo.fogbugz.com/f/cases/19911
Then I had here a really huge image with 1.2 GB with all known memory leaks solved. It was leaking because of Undeclared, after cleaning of them (Undeclared removeUnreferencedKeys) it shrinked to 59 MB.
In this particular case it was caused by code like this:
initialize super initialize. noBindingBlock := [ objectSpace nilObject ]
because noBindingBlock was undeclared, it created a reference from globals (Undeclared) to a particular context with all mess it referenced.
Not sure if we should make some action like clean Undeclared before every snapshot. But the case like this is quite obscure and the information that people should beware it should be enough.
Cleaning Undeclared arbitrarily is a very bad idea. Undeclared is where we can find and eliminate Undeclared variables. Just eliminating them from Undeclared does not solve the problem. The variables are still referenced in coded. But when one removes them from Undeclared they become harder to find.
Instead the right solution is a) to have tests in the standard test suite that check that Undeclared is empty. If Undeclared is not empty the test should fail. b) in any packaging or release process that produces a clean image there should be a check for Undeclared being empty, and the build should fail if it is not empty. c) Smalltalk programmers should understand how undeclared variables work in Smalltalk:
Variables are declared either in some global pool such as Smalltalk, TextConstants or a class pool (the dictionary holding class variables), or as instance variables of a class, or as arguments and temporary variables of a method. These are the only variables in the language. When the compiler sees a variable name it searches for it, first amongst arguments and temporaries, second amongst instance variables, third amongst class variables, fourth amongst the class's imported pools, fifth in globals (Smalltalk) and finally in Undeclared. If the variable is not found in any of these places then a binding is created in Undeclared and that is used to implement the variable. So undeclared variables are compiled as globals in Undeclared.
Whenever a variable is declared in a global pool (Smalltalk, a shared pool or a class pool) the compiler checks for a binding in Undeclared and uses that if it exists. Adding a class declares the class's binding in Smalltalk. So uses of variables before they are declared get fixed up on declaration without having to recompile the entire system. This is how the system manages circularities amongst classes. A reference to a class that has yet to be installed will be compiled as a reference to a binding in Undeclared. When the class is installed it is set as the value of the Undeclared binding, and the binding is moved from Undeclared to Smalltalk.
When an instance variable is defined in a class that changes the shape of the class's instances and so the class's methods are recompiled and that will bind any undeclared references to the instance variable correctly.
So an entry in Undeclared is a) an indication that somewhere there is a missing declaration b) a place where one can look for references to the undeclared binding (references in the dictionary inspector, and I hope the Glamour inspector, that opens when one inspects Undeclared) c) a simple mechanism for allowing circular references amongst classes
So under no circumstances remove bindings from Undeclared unless there are no references from code to those bindings. There is a method that does this: Undeclared removeUnusedBindings
HTH
Very informative. Thanks for that insight. Maybe should capture it for Deep Into Pharo (but I'm off to bed right now) cheers -ben
Cheers, -- Pavel
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
-- _,,,^..^,,,_ best, Eliot
Hi,
On Apr 3, 2017, at 9:00 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P)
Does this mean that on Mac the 32bit VM should not crash when working with Athens and Roassal? What about the 64bit? Cheers, Doru
- make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it. Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
-- www.tudorgirba.com www.feenk.com "To lead is not to demand things, it is to make them happen."
On 4 Apr 2017, at 12:29, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Apr 3, 2017, at 9:00 AM, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P)
Does this mean that on Mac the 32bit VM should not crash when working with Athens and Roassal?
What about the 64bit?
there is no win64 bits pharo version Esteban
Cheers, Doru
- make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it. Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
-- www.tudorgirba.com <http://www.tudorgirba.com/> www.feenk.com <http://www.feenk.com/>
"To lead is not to demand things, it is to make them happen."
On 4 Apr 2017, at 12:31, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 4 Apr 2017, at 12:29, Tudor Girba <tudor@tudorgirba.com <mailto:tudor@tudorgirba.com>> wrote:
Hi,
On Apr 3, 2017, at 9:00 AM, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
Hello,
This are the things I believe still needs to be solved to be able to release Pharo 6.0:
VM: === - win32, cairo fonts problem (I submitted a new VM to test, I think this is fixed :P)
Does this mean that on Mac the 32bit VM should not crash when working with Athens and Roassal?
What about the 64bit?
there is no win64 bits pharo version missread :) yes it shouldnât. Athens 64 should work⦠still not tested intensively, TBH⦠but the tiger demo was working :) Roassal is responsibility of âroassalersâ :P (but they may find athens problems, which Iâm eager to hear).
Esteban
Esteban
Cheers, Doru
- make all tests pass (there are still some failing tests when compiling a new VM)/. - libgit2: Still not there, Jan and Ben were trying and sent some suggestions but I still didnât had the time to test it. Image: =====
- iceberg still needs to go inside the image, Iâm running to release version 0.4 (maybe this week?) and then we can integrate. - I'm still convinced there is a leak. I need to check this in detail. And of course some help on this⦠Pavel?⦠would be cool :)
List is a lot smaller now :) If everything goes well, we can target to release next week.
Next: - Test the new process and tape some videos explaining how we will contribute with a bootstrapped process using github :)
cheers! Esteban
-- www.tudorgirba.com <http://www.tudorgirba.com/> www.feenk.com <http://www.feenk.com/>
"To lead is not to demand things, it is to make them happen."
participants (6)
-
Ben Coman -
Denis Kudriashov -
Eliot Miranda -
Esteban Lorenzano -
Pavel Krivanek -
Tudor Girba