[Pharo-project] Startup/shutdown
Hello all, I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list. No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped. Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient. I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm? Bill =========================================================== [Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009 * Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some) Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory... Nicolas 2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
I am not so sure it will not be a big deal. With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. It's something that just feels wrong in capital letters, and we are trying to fix things. It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
As a compromise, i suggest to introduce 2 different shutdown modes: - with cleanup - without cleanup and then reflect it in UI, or in system preferences. Case closed :) 2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
I am not so sure it will not be a big deal. Â With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. Â It's something that just feels wrong in capital letters, and we are trying to fix things.
It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. Â However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Â Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Â Are we content with having to close and reopen all of that just to save the image? Â I submit that this is a flawed design. Â Worst case, the vm knows when it loads an image and could tell the image that it happened. Â From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. Â External resources are released on a shutdown trigger, but not image save. Â This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. Â It would trigger events for starup and shutdown, help with logging, etc. Â Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
  * Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? Â IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. Â What I do not get is why the vm does not tell it. Â If it did, I _think_ we could clean all of this up. Â Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
Sig, I must disagree, because there is more to what I am saying than simply to skip cleanup (I am saying to do it in two parts, each at the correct time and only at the correct time). Further, the current behavior has nothing to do with shutdown - it is done pre and post snapshot. Dolphin has it right; we should borrow the idea. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, August 27, 2009 8:26 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown As a compromise, i suggest to introduce 2 different shutdown modes: - with cleanup - without cleanup and then reflect it in UI, or in system preferences. Case closed :) 2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
I am not so sure it will not be a big deal. Â With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. Â It's something that just feels wrong in capital letters, and we are trying to fix things.
It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. Â However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Â Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Â Are we content with having to close and reopen all of that just to save the image? Â I submit that this is a flawed design. Â Worst case, the vm knows when it loads an image and could tell the image that it happened. Â From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. Â External resources are released on a shutdown trigger, but not image save. Â This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. Â It would trigger events for starup and shutdown, help with logging, etc. Â Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
  * Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? Â IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. Â What I do not get is why the vm does not tell it. Â If it did, I _think_ we could clean all of this up. Â Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
El jue, 27-08-2009 a las 21:55 -0400, Schwab,Wilhelm K escribió:
Sig,
I must disagree, because there is more to what I am saying than simply to skip cleanup (I am saying to do it in two parts, each at the correct time and only at the correct time). Further, the current behavior has nothing to do with shutdown - it is done pre and post snapshot. Dolphin has it right; we should borrow the idea.
Bill
Well, I suppose that there is a compelling reason to have it. Just one more scenario that I think that will be common, not that there won't be others that really need a solution like you describe: In a big web app, for example, you'll deploy a lot of PharoCore images (or better yet, PharoKernel as the one Pavel announced today) that manage each of them a lot of sessions of the users. The important data to save each time is outside the image, maybe on magma, a rdbs or in the file system, for files uploaded to the webapp. In case an error show up, I should have to modify all the images and save them all. Better is to patch the PharoCore template image and to restart them all to get a consistent fix on all the images at almost the same time. If I were to do it by hand, and saving each image, this will mean maybe hours to finish. In a case like this, it is more practical to restart the server with a minimal programed and announced maintenance window. But, as Stef likes to say: code, code, code! I have never used Dolphin. You have more experience with it and maybe you could port the code or implement a brand new based on that ideas and contribute it to Pharo. I will be glad to test it, of course. Cheers
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, August 27, 2009 8:26 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
As a compromise, i suggest to introduce 2 different shutdown modes: - with cleanup - without cleanup
and then reflect it in UI, or in system preferences. Case closed :)
2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
I am not so sure it will not be a big deal. With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. It's something that just feels wrong in capital letters, and we are trying to fix things.
It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Miguel Cobá http://miguel.leugim.com.mx
Miguel, Good point. I think the most compelling reasons are correctness and native windows. The latter will be most flexibly handled (at least in part) by FFI/Alien, and the current situation will bring that to a halt. I will very likely build some minimal sesion managers. With some cleverness, it might be possible to do with information we have available now - no promises though :) Any clever ideas? I am torn between thinking that lazy initialization should be enough and thinking that the vm will have to tell the image it has just been started. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 9:19 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown El jue, 27-08-2009 a las 21:55 -0400, Schwab,Wilhelm K escribió:
Sig,
I must disagree, because there is more to what I am saying than simply to skip cleanup (I am saying to do it in two parts, each at the correct time and only at the correct time). Further, the current behavior has nothing to do with shutdown - it is done pre and post snapshot. Dolphin has it right; we should borrow the idea.
Bill
Well, I suppose that there is a compelling reason to have it. Just one more scenario that I think that will be common, not that there won't be others that really need a solution like you describe: In a big web app, for example, you'll deploy a lot of PharoCore images (or better yet, PharoKernel as the one Pavel announced today) that manage each of them a lot of sessions of the users. The important data to save each time is outside the image, maybe on magma, a rdbs or in the file system, for files uploaded to the webapp. In case an error show up, I should have to modify all the images and save them all. Better is to patch the PharoCore template image and to restart them all to get a consistent fix on all the images at almost the same time. If I were to do it by hand, and saving each image, this will mean maybe hours to finish. In a case like this, it is more practical to restart the server with a minimal programed and announced maintenance window. But, as Stef likes to say: code, code, code! I have never used Dolphin. You have more experience with it and maybe you could port the code or implement a brand new based on that ideas and contribute it to Pharo. I will be glad to test it, of course. Cheers
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, August 27, 2009 8:26 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
As a compromise, i suggest to introduce 2 different shutdown modes: - with cleanup - without cleanup
and then reflect it in UI, or in system preferences. Case closed :)
2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
I am not so sure it will not be a big deal. With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. It's something that just feels wrong in capital letters, and we are trying to fix things.
It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Miguel Cobá http://miguel.leugim.com.mx _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
Sig,
I must disagree, because there is more to what I am saying than simply to skip cleanup (I am saying to do it in two parts, each at the correct time and only at the correct time). Â Further, the current behavior has nothing to do with shutdown - it is done pre and post snapshot. Â Dolphin has it right; we should borrow the idea.
Oh, yes, i mistaken. I by shutdown i meant snapshot. Its just from current Squeak POV shutdown is always performed before snapshot.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Igor Stasenko Sent: Thursday, August 27, 2009 8:26 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
As a compromise, i suggest to introduce 2 different shutdown modes:  - with cleanup  - without cleanup
and then reflect it in UI, or in system preferences. Case closed :)
2009/8/28 Schwab,Wilhelm K <bschwab@anest.ufl.edu>:
I am not so sure it will not be a big deal. Â With an open system, there is no reason not to use development tools to make changes on production machines, and saving the image would be part of that process; every occurance would clobber all sockets, all web users, all database connections, etc. Â It's something that just feels wrong in capital letters, and we are trying to fix things.
It definitely smells bad from a scientific computation perspective where external connections and saving the image are very common.
Bill
-----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martinez Sent: Thursday, August 27, 2009 7:57 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown
El jue, 27-08-2009 a las 20:23 -0400, Schwab,Wilhelm K escribió:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. Â However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Â Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Â Are we content with having to close and reopen all of that just to save the image? Â I submit that this is a flawed design. Â Worst case, the vm knows when it loads an image and could tell the image that it happened. Â From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. Â External resources are released on a shutdown trigger, but not image save. Â This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. Â It would trigger events for starup and shutdown, help with logging, etc. Â Can it be donw without hacking the vm?
Just a question, what is the big deal with this, as you most often save on a development environment and not in a production environment. And as the number of saves on a production environment are seldom, it isn't a problem I think. In the dev enviroment you are pointing to dev services and you have few open conections/whatever so that is not a big deal either.
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
  * Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance???   * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? Â IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. Â What I do not get is why the vm does not tell it. Â If it did, I _think_ we could clean all of this up. Â Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Miguel Cobá http://miguel.leugim.com.mx
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Best regards, Igor Stasenko AKA sig.
shutDown: already provides a boolean indicating whether a quit is about to occur or not. I suppose any code is welcome not to cleanUp properly if it's not a quitting situation.. What feels wrong to me would be to save "garbage" (what will be unusable sockets, etc.) into the saved image.
occurance would clobber all sockets, all web users, all database connections,
Which is why first-class connections / sockets should be used, which don't depend on their underlying physical socket to be always connected. This is what Magma does. A socket can get disconnected at any time for a variety of reasons, not just an image save/shutdown. I think the software should, transparently to the user, reestablish connections lazily, making saving the image a non-issue. - Chris
Chris, I respectfully disagree. I know what you mean about "saving garbage," but after giving it a lot of thought over years of faithful service from Dolphin, I realized that it is a perfectly fine design because it allows objects and external resources to plow merrily past an image save. The bogus data is quickly discarded on startup because it is only then that it can be recognized as garbage. As for first-class connections, that sounds fine as far as it goes. Who sets policy about timeouts and expected activity levels? I submit that one size does not fit all; Microsoft tried it (DCOM), and it was a _total_ disaster. Provide all the slick stuff you want, but we should also grant our peers (developers) access to working sockets, and having them cut off on snapshots just because "Squeak does it that way" is not the way to meet Pharo's mission. Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Chris Muller Sent: Friday, August 28, 2009 10:21 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown shutDown: already provides a boolean indicating whether a quit is about to occur or not. I suppose any code is welcome not to cleanUp properly if it's not a quitting situation.. What feels wrong to me would be to save "garbage" (what will be unusable sockets, etc.) into the saved image.
occurance would clobber all sockets, all web users, all database connections,
Which is why first-class connections / sockets should be used, which don't depend on their underlying physical socket to be always connected. This is what Magma does. A socket can get disconnected at any time for a variety of reasons, not just an image save/shutdown. I think the software should, transparently to the user, reestablish connections lazily, making saving the image a non-issue. - Chris _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Aug 28, 2009, at 2:23 AM, Schwab,Wilhelm K wrote:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
this is clear that dolphin people always value code quality over hyper innovation. If dolphin would work on mac and (if it was going open source except of been killed) I would use it daily.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stef, I am not quite willing to speak of Dolphin in the past tense, as there is a collaboration between Object Arts and Lesser. My interest is not so much running on macs as running off of Windows. Depending on various forces, I might turn into a mac user, but for now Linux looks like the escape plan. Oddly enough, I was trying to figure out if I could reasonably organize a fork of Squeak (and very much doubted that I could make it happen) when I learned of Saphire. Thanks for getting all of this going! Bill -----Original Message----- From: pharo-project-bounces@lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse Sent: Friday, August 28, 2009 2:29 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Startup/shutdown On Aug 28, 2009, at 2:23 AM, Schwab,Wilhelm K wrote:
Hello all,
I am doing some more porting and hitting some trouble spots - no real surprise that they exist. However, I am struck by a strong belief that Dolphin has a lot of things right, and startup and shutdown is very high on that list.
this is clear that dolphin people always value code quality over hyper innovation. If dolphin would work on mac and (if it was going open source except of been killed) I would use it daily.
No offense to Nicolas, who has been quite helpful, let's consider a somewhat extrmeme example. Suppose a server is using an ORB to do SSL protected communications with various servers and clients, some Seaside work, and has a few database connections open. Are we content with having to close and reopen all of that just to save the image? I submit that this is a flawed design. Worst case, the vm knows when it loads an image and could tell the image that it happened. From there, a lot of extra work could be dumped.
Nicolas is quite right that step (2) below must be done carefully, but it has been handled nicely in Dolphin by registering potential offenders in the startup triggers and doing the cleanup at that time. External resources are released on a shutdown trigger, but not image save. This will strike again with native windows, as the current approach would (stop me if this is a fallacy) have everything released and reopened on image save, which would be very inefficient.
I have code that expects a class called SessionManager, and I am seriously thinking of creating it. It would trigger events for starup and shutdown, help with logging, etc. Can it be donw without hacking the vm?
Bill
===========================================================
[Pharo-project] Save and quit vs. Save - a clue to performance??? Nicolas Cellier nicolas.cellier.aka.nice at gmail.com Thu Jul 16 22:48:34 CEST 2009
* Previous message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Next message: [Pharo-project] Save and quit vs. Save - a clue to performance??? * Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Current policy is to cleanup before saving, and reconnect after saving as if a fresh startup occured As you seem to suggest, a possible alternative without using fork would be to: 1) distinguish whether this is a newly restarted image (primSnapshot returning true or false...) 2) perform all the clean-ups at startup if newly restarted image 3) rely on the process exit() to perform required resource clean-up in case we quit (or let a self aboutToQuit do some)
Step 2) would deserve great care, any unprotected access to an uninitialized pointer of the old image would crash or trash memory...
Nicolas
2009/7/16 Schwab,Wilhelm K <bschwab at anest.ufl.edu>:
Nicolas,
One thing I forgot to mention: thread safety??? IIRC, Dolphin triggers startup events, and all external resources clear pointers then (though they will release on shutdown, not image saving) to force a lazy connection.
I see why, absent help from the vm, the image does not know when it starts and stops. What I do not get is why the vm does not tell it. If it did, I _think_ we could clean all of this up. Sometimes the truth hurts though, so please find any faults in my logic (or lack thereof).
Bill
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (5)
-
Chris Muller -
Igor Stasenko -
Miguel Enrique Cobá Martinez -
Schwab,Wilhelm K -
Stéphane Ducasse