Problem cleaning up image
Hello, Isn't cleanup the headless image working right now? I know it has some problems, but I tried to: curl get.pharo.org | bash ./pharo Pharo.image clean --production It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled. Regards, Vitor
Hello, when I remember correctly when I tested cleaning last time, the problem was in the method ImageCleaner>>cleanUpForRelease (called form cleanUpForProduction). Try to comment "self cleanUpMethods." in ImageCleaner>>cleanUpForRelease - "cleanUpMethods" method never finish. Have you tried "minimal image" or "kernel image" (instead of cleaning full image)? pf
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
Hello, Yes, this is a bug that we really need to fix. (sadly no information to add other thant that).
On 09 Sep 2016, at 15:16, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org <http://get.pharo.org/> | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
Ok may be this is just an endless loop :) Le 12/9/16 à 10:22, Marcus Denker a écrit :
Hello,
Yes, this is a bug that we really need to fix. (sadly no information to add other thant that).
On 09 Sep 2016, at 15:16, Vitor Medina Cruz <vitormcruz@gmail.com <mailto:vitormcruz@gmail.com>> wrote:
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org <http://get.pharo.org/> | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
Ok, I did some extra tests and I think I figured what is happening: the code I load on the image subject to clean has a startUp method that starts a Teapot server on port 8080. I tried various scenarios: 1- Load a fresh image from get.pharo.org and execute clean --> Ok 2- Load a fresh image from get.pharo.org only with the dependencies from my project (ConfigurationOfTwitterChallenge), one at a time, and execute clean --> Ok 3- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute clean --> Fail 4- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute eval "TwitterChallengeMain stopDefaultExecution. ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true." --> *Ok *(Assuming ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true. does the same as clean --production) It appears I cannot clean an Image with the server started. Question: is that an error or I really should not try to clean an image with a server on? If the former I open an issue on FogBuz and I will try on a not-headless vm to see if some error happen. Regards, Vitor On Mon, Sep 12, 2016 at 6:59 AM, stepharo <stepharo@free.fr> wrote:
Ok may be this is just an endless loop :)
Le 12/9/16 à 10:22, Marcus Denker a écrit :
Hello,
Yes, this is a bug that we really need to fix. (sadly no information to add other thant that).
On 09 Sep 2016, at 15:16, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
Hi, I think it is not really realistic to expect this to work *after* loading code of others. Why not -> do image clean -> *then* load the packages you need? Marcus
On 13 Sep 2016, at 03:19, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Ok, I did some extra tests and I think I figured what is happening: the code I load on the image subject to clean has a startUp method that starts a Teapot server on port 8080. I tried various scenarios:
1- Load a fresh image from get.pharo.org <http://get.pharo.org/> and execute clean --> Ok 2- Load a fresh image from get.pharo.org <http://get.pharo.org/> only with the dependencies from my project (ConfigurationOfTwitterChallenge), one at a time, and execute clean --> Ok 3- Load a fresh image from get.pharo.org <http://get.pharo.org/>, apply ConfigurationOfTwitterChallenge and execute clean --> Fail 4- Load a fresh image from get.pharo.org <http://get.pharo.org/>, apply ConfigurationOfTwitterChallenge and execute eval "TwitterChallengeMain stopDefaultExecution. ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true." --> Ok (Assuming ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true. does the same as clean --production)
It appears I cannot clean an Image with the server started. Question: is that an error or I really should not try to clean an image with a server on? If the former I open an issue on FogBuz and I will try on a not-headless vm to see if some error happen.
Regards, Vitor
On Mon, Sep 12, 2016 at 6:59 AM, stepharo <stepharo@free.fr <mailto:stepharo@free.fr>> wrote: Ok may be this is just an endless loop :)
Le 12/9/16 à 10:22, Marcus Denker a écrit :
Hello,
Yes, this is a bug that we really need to fix. (sadly no information to add other thant that).
On 09 Sep 2016, at 15:16, Vitor Medina Cruz <vitormcruz@gmail.com <mailto:vitormcruz@gmail.com>> wrote:
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org <http://get.pharo.org/> | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
Humm, why not? When I load the configuration it loads several other packages, wouldn't that need a cleanup too? When I first see that I could cleanup an image my first thought was to put it in a production state with my code, then I would cleanup to leave it as lean as possible. Isn't that correct? If I should not clean after load my packages why provide the functionality? Why not provide clean images to be used in production so that people could download and load it's ready to production code? Regards, Vitor On Tue, Sep 13, 2016 at 4:00 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
Hi,
I think it is not really realistic to expect this to work *after* loading code of others.
Why not
-> do image clean -> *then* load the packages you need?
Marcus
On 13 Sep 2016, at 03:19, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Ok, I did some extra tests and I think I figured what is happening: the code I load on the image subject to clean has a startUp method that starts a Teapot server on port 8080. I tried various scenarios:
1- Load a fresh image from get.pharo.org and execute clean --> Ok 2- Load a fresh image from get.pharo.org only with the dependencies from my project (ConfigurationOfTwitterChallenge), one at a time, and execute clean --> Ok 3- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute clean --> Fail 4- Load a fresh image from get.pharo.org, apply ConfigurationOfTwitterChallenge and execute eval "TwitterChallengeMain stopDefaultExecution. ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true." --> *Ok *(Assuming ImageCleaner cleanUpForProduction. Smalltalk snapshot: true andQuit: true. does the same as clean --production)
It appears I cannot clean an Image with the server started. Question: is that an error or I really should not try to clean an image with a server on? If the former I open an issue on FogBuz and I will try on a not-headless vm to see if some error happen.
Regards, Vitor
On Mon, Sep 12, 2016 at 6:59 AM, stepharo <stepharo@free.fr> wrote:
Ok may be this is just an endless loop :)
Le 12/9/16 à 10:22, Marcus Denker a écrit :
Hello,
Yes, this is a bug that we really need to fix. (sadly no information to add other thant that).
On 09 Sep 2016, at 15:16, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Hello,
Isn't cleanup the headless image working right now? I know it has some problems, but I tried to:
curl get.pharo.org | bash ./pharo Pharo.image clean --production
It runs for hours and don't complete. Last time I leave it for about 5 hours before I cancelled.
Regards, Vitor
On 13 Sep 2016, at 13:15, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Humm, why not? When I load the configuration it loads several other packages, wouldn't that need a cleanup too? When I first see that I could cleanup an image my first thought was to put it in a production state with my code, then I would cleanup to leave it as lean as possible. Isn't that correct?
If I should not clean after load my packages why provide the functionality? Why not provide clean images to be used in production so that people could download and load it's ready to production code?
No, the #cleanUpForProduction just unloads tests, examples and things like that⦠it only touches loaded code in the sense that it unloads all packages named -Tests and -Examples. But if the external package happens to be structured in a way that this is bad, it would brake. I never expected this to be run *after* more packages are loaded, just *before*. What it does right now it this (you can see that it could be simpler if our packages are better structured): cleanUpForProduction "self new cleanUpForProduction" "delete logo" PolymorphSystemSettings showDesktopLogo: false. self packagesForCleanUpInProduction do: [ :each | (MCPackage named: each) unload ]. self helpPackages do: [ :each | (MCPackage named: each) unload ]. self testPackages do: [ :each | (MCPackage named: each) unload ]. self examplePackages do: [ :each | (MCPackage named: each) unload ]. (MCPackage named: 'ScriptLoader40') unload. (MCPackage named: 'MonticelloMocks') unload. (MCPackage named: 'ToolsTest') unload. (MCPackage named: 'Announcements-Tests-Core') unload. (MCPackage named: 'AST-Tests-Core') unload. (MCPackage named: 'AST-Interpreter-Test') unload. (MCPackage named: 'Ring-Tests-Containers') unload. (MCPackage named: 'Ring-Tests-Kernel') unload. (MCPackage named: 'Ring-Tests-Monticello') unload. (MCPackage named: 'Regex-Tests-Core') unload. (MCPackage named: 'Refactoring-Tests-Changes') unload. (MCPackage named: 'Refactoring-Tests-Core') unload. (MCPackage named: 'Refactoring-Tests-Critics') unload. (MCPackage named: 'Refactoring-Tests-Environment') unload. (MCPackage named: 'FileSystem-Tests-Core') unload. (MCPackage named: 'FileSystem-Tests-Disk') unload. (MCPackage named: 'FileSystem-Tests-Memory') unload. (MCPackage named: 'GT-Tests-Inspector') unload. (MCPackage named: 'Glamour-Tests-Core') unload. (MCPackage named: 'Glamour-Tests-Morphic') unload. (MCPackage named: 'Glamour-Tests-Resources') unload. (MCPackage named: 'Glamour-Tests-Rubric') unload. (MCPackage named: 'Versionner-Core-Model') unload. (MCPackage named: 'Versionner-Core-DependenciesModel') unload. (MCPackage named: 'Versionner-Core-Announcements') unload. (MCPackage named: 'Versionner-Core-Commands') unload. (MCPackage named: 'Versionner-Spec-Browser') unload. (MCPackage named: 'Versionner-Tests-Core-Commands') unload. (MCPackage named: 'Versionner-Tests-Core-DependenciesModel') unload. (MCPackage named: 'Versionner-Tests-Core-Model') unload. (MCPackage named: 'Versionner-Tests-Resources') unload. self cleanUpForRelease. Marcus
I understand. I thought it did more stuff, like remove necessary development tools or executing processes, I don't know, but considering its implementation it should affect only the image size, right? Thanks, Vitor On Tue, Sep 13, 2016 at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 13 Sep 2016, at 13:15, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
Humm, why not? When I load the configuration it loads several other packages, wouldn't that need a cleanup too? When I first see that I could cleanup an image my first thought was to put it in a production state with my code, then I would cleanup to leave it as lean as possible. Isn't that correct?
If I should not clean after load my packages why provide the functionality? Why not provide clean images to be used in production so that people could download and load it's ready to production code?
No, the #cleanUpForProduction just unloads tests, examples and things like that⦠it only touches loaded code in the sense that it unloads all packages named -Tests and -Examples.
But if the external package happens to be structured in a way that this is bad, it would brake.
I never expected this to be run *after* more packages are loaded, just *before*.
What it does right now it this (you can see that it could be simpler if our packages are better structured):
cleanUpForProduction "self new cleanUpForProduction" "delete logo" PolymorphSystemSettings showDesktopLogo: false.
self packagesForCleanUpInProduction do: [ :each | (MCPackage named: each) unload ]. self helpPackages do: [ :each | (MCPackage named: each) unload ]. self testPackages do: [ :each | (MCPackage named: each) unload ]. self examplePackages do: [ :each | (MCPackage named: each) unload ]. (MCPackage named: 'ScriptLoader40') unload.
(MCPackage named: 'MonticelloMocks') unload. (MCPackage named: 'ToolsTest') unload. (MCPackage named: 'Announcements-Tests-Core') unload. (MCPackage named: 'AST-Tests-Core') unload. (MCPackage named: 'AST-Interpreter-Test') unload. (MCPackage named: 'Ring-Tests-Containers') unload. (MCPackage named: 'Ring-Tests-Kernel') unload. (MCPackage named: 'Ring-Tests-Monticello') unload. (MCPackage named: 'Regex-Tests-Core') unload. (MCPackage named: 'Refactoring-Tests-Changes') unload. (MCPackage named: 'Refactoring-Tests-Core') unload. (MCPackage named: 'Refactoring-Tests-Critics') unload. (MCPackage named: 'Refactoring-Tests-Environment') unload. (MCPackage named: 'FileSystem-Tests-Core') unload. (MCPackage named: 'FileSystem-Tests-Disk') unload. (MCPackage named: 'FileSystem-Tests-Memory') unload.
(MCPackage named: 'GT-Tests-Inspector') unload. (MCPackage named: 'Glamour-Tests-Core') unload. (MCPackage named: 'Glamour-Tests-Morphic') unload. (MCPackage named: 'Glamour-Tests-Resources') unload. (MCPackage named: 'Glamour-Tests-Rubric') unload.
(MCPackage named: 'Versionner-Core-Model') unload. (MCPackage named: 'Versionner-Core-DependenciesModel') unload. (MCPackage named: 'Versionner-Core-Announcements') unload. (MCPackage named: 'Versionner-Core-Commands') unload. (MCPackage named: 'Versionner-Spec-Browser') unload. (MCPackage named: 'Versionner-Tests-Core-Commands') unload. (MCPackage named: 'Versionner-Tests-Core-DependenciesModel') unload. (MCPackage named: 'Versionner-Tests-Core-Model') unload. (MCPackage named: 'Versionner-Tests-Resources') unload.
self cleanUpForRelease.
Marcus
On 13 Sep 2016, at 18:54, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
I understand. I thought it did more stuff, like remove necessary development tools or executing processes,
It does call #cleanUpForRelease, but this we call at *every* update on the CI, too. This calls the #cleanUp method of all classes and does some more (rehashing all dictionaries, restarting all? methods holding on to blocksâ¦).
I don't know, but considering its implementation it should affect only the image size, right?
Yes. Of course right now Cog is not compacting the somewhat empty allocated memory regions, so e.g. the 47 MB image should be around 15 MB empty space and the cleanup will not shrink it. But when this is fixed, you should see a change. A saved image is too a large extend methods and literals they point to (especially for things like Tutorials). In the past the normal image was ca. 25MB, while the shrieked one after cleanup for deployment maybe 15? I do not remember. But that was already a bit of âmoored lawâ time ago ;-) Marcus
Thanks for the answers! On Tue, Sep 13, 2016 at 2:46 PM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 13 Sep 2016, at 18:54, Vitor Medina Cruz <vitormcruz@gmail.com> wrote:
I understand. I thought it did more stuff, like remove necessary development tools or executing processes,
It does call #cleanUpForRelease, but this we call at *every* update on the CI, too. This calls the #cleanUp method of all classes and does some more (rehashing all dictionaries, restarting all? methods holding on to blocksâ¦).
I don't know, but considering its implementation it should affect only the image size, right?
Yes. Of course right now Cog is not compacting the somewhat empty allocated memory regions, so e.g. the 47 MB image should be around 15 MB empty space and the cleanup will not shrink it.
But when this is fixed, you should see a change. A saved image is too a large extend methods and literals they point to (especially for things like Tutorials). In the past the normal image was ca. 25MB, while the shrieked one after cleanup for deployment maybe 15? I do not remember.
But that was already a bit of âmoored lawâ time ago ;-)
Marcus
participants (4)
-
Marcus Denker -
Petr Fischer -
stepharo -
Vitor Medina Cruz