[Pharo-project] Class initialisation after loading
Hi, For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded. Apparently #initialize is only sent to *newly* loaded classes that did not exist before ? It does not get sent to existing classes being reloaded/overwritten ? Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ? What would be the best practice here ? The most portable one ? I want to be able to make sure #initialize is sent when I want it. Thx, Sven PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn. -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff. Norbert Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Morning, Norbert, On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ? Sven
Norbert
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Am 14.01.2013 um 07:54 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Morning, Norbert,
On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ?
I think that the primary purpose is to initialize newly created space. In case of shape change this doesn't happen. So taking the example you initialize e.g. an instvar with a collection. Do you want to reinitialize it even if you corrected a typo in the comment of a method? If I wanted to have control over those things I would on class initialization register a class of mine to the local change notification system in the image. But this isn't easy to do in a cross platform manner. Maybe there are better ways. I'm sure someone will speak up then. Norbert
Sven
Norbert
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
On 14 Jan 2013, at 09:09, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 07:54 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Morning, Norbert,
On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ?
I think that the primary purpose is to initialize newly created space. In case of shape change this doesn't happen. So taking the example you initialize e.g. an instvar with a collection. Do you want to reinitialize it even if you corrected a typo in the comment of a method? If I wanted to have control over those things I would on class initialization register a class of mine to the local change notification system in the image. But this isn't easy to do in a cross platform manner. Maybe there are better ways. I'm sure someone will speak up then.
Let's make it more concrete. Zinc-Resource-Meta-Core is a package that among other thins contains ZnMimeType. ZnMimeType class>>initialize sets up some constants (lists of known mime types, file extension to mime type mappings, you get the idea). When I add new constants in a new source version, I want them to be added when doing an automated CI build, obviously. Right now, this does *not* happen, probably because the class and #initialize method already exist in the image where it is being load in. Indeed, you would not always want costly initializations to run, but there should at least be a hook to do it, no ?
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Am 14.01.2013 um 09:24 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 14 Jan 2013, at 09:09, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 07:54 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Morning, Norbert,
On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ?
I think that the primary purpose is to initialize newly created space. In case of shape change this doesn't happen. So taking the example you initialize e.g. an instvar with a collection. Do you want to reinitialize it even if you corrected a typo in the comment of a method? If I wanted to have control over those things I would on class initialization register a class of mine to the local change notification system in the image. But this isn't easy to do in a cross platform manner. Maybe there are better ways. I'm sure someone will speak up then.
Let's make it more concrete. Zinc-Resource-Meta-Core is a package that among other thins contains ZnMimeType. ZnMimeType class>>initialize sets up some constants (lists of known mime types, file extension to mime type mappings, you get the idea). When I add new constants in a new source version, I want them to be added when doing an automated CI build, obviously. Right now, this does *not* happen, probably because the class and #initialize method already exist in the image where it is being load in.
I understand. The problem is introduced by having a cache of those mime types. That is often a problem because the cache is designed as infinite valid but there is a use case that needs a little less infinite :) I think it depends how often that "adding" takes place. And it seems to me it does not occur very often. Apart from that cache invalidation is one of the more unpleasant things we face. If you want your cache to update you need to check something. But that is what you want to avoid for performance reasons. So I would see a postLoadIt for these rare jobs is a good match. Norbert
Indeed, you would not always want costly initializations to run, but there should at least be a hook to do it, no ?
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
On 14 Jan 2013, at 09:38, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 09:24 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 14 Jan 2013, at 09:09, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 07:54 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Morning, Norbert,
On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ?
I think that the primary purpose is to initialize newly created space. In case of shape change this doesn't happen. So taking the example you initialize e.g. an instvar with a collection. Do you want to reinitialize it even if you corrected a typo in the comment of a method? If I wanted to have control over those things I would on class initialization register a class of mine to the local change notification system in the image. But this isn't easy to do in a cross platform manner. Maybe there are better ways. I'm sure someone will speak up then.
Let's make it more concrete. Zinc-Resource-Meta-Core is a package that among other thins contains ZnMimeType. ZnMimeType class>>initialize sets up some constants (lists of known mime types, file extension to mime type mappings, you get the idea). When I add new constants in a new source version, I want them to be added when doing an automated CI build, obviously. Right now, this does *not* happen, probably because the class and #initialize method already exist in the image where it is being load in.
I understand. The problem is introduced by having a cache of those mime types. That is often a problem because the cache is designed as infinite valid but there is a use case that needs a little less infinite :) I think it depends how often that "adding" takes place. And it seems to me it does not occur very often. Apart from that cache invalidation is one of the more unpleasant things we face. If you want your cache to update you need to check something. But that is what you want to avoid for performance reasons. So I would see a postLoadIt for these rare jobs is a good match.
Thanks for the discussion, Norbert, I still feel that in most cases, when loading code, or when building new images, I want the class initializers of loaded classes to run to make sure everything is in a known, up to date state. Loading code or building images does not happen a lot and it feels strange to load something but not have it initialized. We'll see if somebody else has anything to say about this.
Indeed, you would not always want costly initializations to run, but there should at least be a hook to do it, no ?
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
On 14 Jan 2013, at 11:05, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 14 Jan 2013, at 09:38, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 09:24 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 14 Jan 2013, at 09:09, Norbert Hartl <norbert@hartl.name> wrote:
Am 14.01.2013 um 07:54 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Morning, Norbert,
On 14 Jan 2013, at 07:44, Norbert Hartl <norbert@hartl.name> wrote:
As for every other objects initialize is called when the object is created. Upgrading code might change the shape of the object but then there is no initialize call. Do you want to have initialize called every tiny code change? Using *cello you can use postLoadDoits to tweak stuff.
You mean that, in your opinion, making sure that the #initialize is called in every scenario (new/existing class) can only be done with an explicit action ?
I think that the primary purpose is to initialize newly created space. In case of shape change this doesn't happen. So taking the example you initialize e.g. an instvar with a collection. Do you want to reinitialize it even if you corrected a typo in the comment of a method? If I wanted to have control over those things I would on class initialization register a class of mine to the local change notification system in the image. But this isn't easy to do in a cross platform manner. Maybe there are better ways. I'm sure someone will speak up then.
Let's make it more concrete. Zinc-Resource-Meta-Core is a package that among other thins contains ZnMimeType. ZnMimeType class>>initialize sets up some constants (lists of known mime types, file extension to mime type mappings, you get the idea). When I add new constants in a new source version, I want them to be added when doing an automated CI build, obviously. Right now, this does *not* happen, probably because the class and #initialize method already exist in the image where it is being load in.
I understand. The problem is introduced by having a cache of those mime types. That is often a problem because the cache is designed as infinite valid but there is a use case that needs a little less infinite :) I think it depends how often that "adding" takes place. And it seems to me it does not occur very often. Apart from that cache invalidation is one of the more unpleasant things we face. If you want your cache to update you need to check something. But that is what you want to avoid for performance reasons. So I would see a postLoadIt for these rare jobs is a good match.
Thanks for the discussion, Norbert, I still feel that in most cases, when loading code, or when building new images, I want the class initializers of loaded classes to run to make sure everything is in a known, up to date state. Loading code or building images does not happen a lot and it feels strange to load something but not have it initialized.
We'll see if somebody else has anything to say about this.
In a private email conversation, Lukas Renggli helped me out. This is what I learned, in my words: It is indeed so that after a Monticello load, a class initializer is only run for totally new classes or when the source code string of the class #initialize method coming in is different from the one already present. So adding/changing a comment or whitespace is all that is needed to force it to run. See, apparently, MCMethodDefinition>>#postloadOver: Now, this Zinc build based on Pharo 1.3 is all green again: http://jenkins.lukas-renggli.ch/job/Zinc/ I learned something important that was bugging me for some time. Sven
Indeed, you would not always want costly initializations to run, but there should at least be a hook to do it, no ?
Am 14.01.2013 um 07:20 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Hi,
For a long time I have been having troubles with #initialize (not) being sent to classes that just got loaded.
Apparently #initialize is only sent to *newly* loaded classes that did not exist before ?
It does not get sent to existing classes being reloaded/overwritten ?
Does the fact that the #initialize class method itself gets overwritten (actually changes) have anything to do with it ?
What would be the best practice here ? The most portable one ?
I want to be able to make sure #initialize is sent when I want it.
Thx,
Sven
PS: I am looking for a solution that would work for Monticello package loading, via Gofer or Metacello as well as regular #fileIn.
-- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
After some thought i decided to contribute my 2cents. First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important. What we missing, i think is a package pre/post load/update action(s) which tied directly to the package, so developer can decide by himself what needs to be (re)initialized and in what order without need of having some automated tool to decide it (often wrongly). I would really like that one day we could have packages as a first-class entities in a system, so we could express things in simple way: Package named: #Foo. Foo>> onLoad "perform an action when the package is first loaded into image" MyClass initialize. Foo>> onUpdate: oldVersion "perform an action when different version of package is loaded into image" MyClass reinitialize. of course, being able to control those actions is just a tip of the iceberg. P.S. i know that MC supports package pre/post load scripts (but strangely i cannot find where). so, it is really opaque to all of the tools, since it is outside of the source codebase of the package. P.P.S. Metacello also supports pre/post actions, so i think right now a best place where you can put (re)initialization code is into metacello configuration, and initialize things explicitly, rather than relying on implicit logic of tools. -- Best regards, Igor Stasenko.
Hi Igor, On 17 Jan 2013, at 12:25, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
What we missing, i think is a package pre/post load/update action(s) which tied directly to the package, so developer can decide by himself what needs to be (re)initialized and in what order without need of having some automated tool to decide it (often wrongly).
I would really like that one day we could have packages as a first-class entities in a system, so we could express things in simple way:
Package named: #Foo.
Foo>> onLoad "perform an action when the package is first loaded into image" MyClass initialize.
Foo>> onUpdate: oldVersion "perform an action when different version of package is loaded into image" MyClass reinitialize.
of course, being able to control those actions is just a tip of the iceberg.
Your analysis is correct. We, library/framwork developers, have to work with what is available, across multiple versions of Pharo, so a solution based on the current situation is preferable. Of course, we can and should improve what we can.
P.S. i know that MC supports package pre/post load scripts (but strangely i cannot find where). so, it is really opaque to all of the tools, since it is outside of the source codebase of the package.
What about MCScriptDefinition and subclasses ? Not that I have any idea how to use this mechanism ;-)
P.P.S. Metacello also supports pre/post actions, so i think right now a best place where you can put (re)initialization code is into metacello configuration, and initialize things explicitly, rather than relying on implicit logic of tools.
-- Best regards, Igor Stasenko.
Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ]. One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g. initialize Sentinel ifNil: [Sentinel := Object new] This avoids breaking existing instances that would end up referring to an older value for Sentinel. Much the same goes for any class state that would end up in an instance. One can use become: here too. What we missing, i think is a package pre/post load/update action(s)
which tied directly to the package, so developer can decide by himself what needs to be (re)initialized and in what order without need of having some automated tool to decide it (often wrongly).
The question is which is better, hooks to allow control of initialization on package load (be it Monticello or Fuel) or simple rules for (frequent) reinialization? The former allows one to write naive initialization but at the cost of mastering some complex mechanisms in package loading. The latter forces one to write careful initialization code but has the benefit of simple package loading rules. Further, writing idempotent class initialization code is only a problem with complex class initialization code, and presumably a programmer who is writing such complex code is up to the task of making it idempotent. In simple cases, writing idempotent code should be simple. Personally I'm starting to prefer the latter approach because the complexity ends up in common Smalltalk code (class initialization) and in one place instead of in special Smalltalk code (package loading hooks) in multiple places (all the various package systems). The tools always support common Smalltalk code better than these special hooks. For example, load scripts may be stored as strings in packages, and there-in senders and implementors won't find them. There are, I think, great advantages to having a system composed of freely reinitializable components. It is one of Gilad Bracha's design goals behind Newspeak to allow running programs to be updated on the fly as fixes are released. Gilad particularly likes the fact that this means one can discard legacy code, since all (if this vision is realized) Newspeak programs should be at the latest version. A Smalltalk system with this property would be far more reliable. Imagine a long-running server application that is simply updated as and when bug fixes are available. I think that forcing frequent class reinitialization and forcing users to write idempotent class initialization is consistent with Pharo's philosophical position on fixing broken things, and on good documentation and simplicity. I also think that it's a good idea. I would really like that one day we could have packages as a
first-class entities in a system, so we could express things in simple way:
Package named: #Foo.
Foo>> onLoad "perform an action when the package is first loaded into image" MyClass initialize.
Foo>> onUpdate: oldVersion "perform an action when different version of package is loaded into image" MyClass reinitialize.
of course, being able to control those actions is just a tip of the iceberg.
Yes, this is exactly the route I took with VisualWorks parcels. And it was necessary because the code we packaged was legacy and we didn't have the cycles to rewrite that code. But I think it's too complex. It adds another layer to class loading, with its own conventions, that may be specific to a particular package system. Whereas if the rule is "your class will be reinitialized whenever it is loaded in whatever form" it's simple, unambiguous and something one can cope with. KISS.
P.S. i know that MC supports package pre/post load scripts (but strangely i cannot find where). so, it is really opaque to all of the tools, since it is outside of the source codebase of the package.
P.P.S. Metacello also supports pre/post actions, so i think right now a best place where you can put (re)initialization code is into metacello configuration, and initialize things explicitly, rather than relying on implicit logic of tools.
-- Best regards, Igor Stasenko.
-- best, Eliot
I think that forcing frequent class reinitialization and forcing users to write idempotent class initialization is consistent with Pharo's philosophical position on fixing broken things, and on good documentation and simplicity. I also think that it's a good idea.
+1 :) This is also why I love so much bootstrap because it forces us to verify that initialization is working and done systematically. Stef
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well. But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan. frank
On Fri, Jan 18, 2013 at 8:47 AM, Frank Shearar <frank.shearar@gmail.com>wrote:
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com>
wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well.
Agreed, but the two can be decoupled. For example one can build the state and then assign it. Assignment is currently atomic so there are no multicore issues here (yet :) ). And (I'm sure you realize, for give the pedantry) the above example isn't lazy, it is eager. It simply refuses to create another Sentinel if one exists already.
But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan.
frank
-- best, Eliot
phhh, let me try again :) On Fri, Jan 18, 2013 at 8:47 AM, Frank Shearar <frank.shearar@gmail.com>wrote:
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com>
wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well.
Agreed, but the two can be decoupled. For example one can build the state and then assign it. Assignment is currently atomic so there are no multicore issues here (yet :) ). Further, assignments are not suspension points so a sequence of (just) assignments are guaranteed to occur without interruption. And (I'm sure you realize, for give the pedantry) the above example isn't lazy, it is eager. It simply refuses to create another Sentinel if one exists already.
But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan.
frank
-- best, Eliot
On 18 January 2013 21:49, Eliot Miranda <eliot.miranda@gmail.com> wrote:
phhh, let me try again :)
On Fri, Jan 18, 2013 at 8:47 AM, Frank Shearar <frank.shearar@gmail.com> wrote:
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well.
Agreed, but the two can be decoupled. For example one can build the state and then assign it. Assignment is currently atomic so there are no multicore issues here (yet :) ). Further, assignments are not suspension points so a sequence of (just) assignments are guaranteed to occur without interruption. And (I'm sure you realize, for give the pedantry) the above example isn't lazy, it is eager. It simply refuses to create another Sentinel if one exists already.
So because of the assignment guarantees we won't have two processes entering the ifNil: block? (I'll try extract one foot from my mouth and maybe replace it with another...) In this particular case things seem pretty innocuous: let's say two processes enter the #initialize. They both manage to enter the ifNil: (I'm aware this is compiled away to jumps, but the impact of this I don't know) through a time-of-check-time-of-use race, and you end up generating two objects. But one's simply lost. As long as references to Sentinel don't escape the class, no one need know anything at all. frank
But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan.
frank
-- best, Eliot
On Fri, Jan 18, 2013 at 2:38 PM, Frank Shearar <frank.shearar@gmail.com>wrote:
On 18 January 2013 21:49, Eliot Miranda <eliot.miranda@gmail.com> wrote:
phhh, let me try again :)
On Fri, Jan 18, 2013 at 8:47 AM, Frank Shearar <frank.shearar@gmail.com> wrote:
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com>
wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until
its
fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well.
Agreed, but the two can be decoupled. For example one can build the state and then assign it. Assignment is currently atomic so there are no multicore issues here (yet :) ). Further, assignments are not suspension points so a sequence of (just) assignments are guaranteed to occur without interruption. And (I'm sure you realize, for give the pedantry) the above example isn't lazy, it is eager. It simply refuses to create another Sentinel if one exists already.
So because of the assignment guarantees we won't have two processes entering the ifNil: block? (I'll try extract one foot from my mouth and maybe replace it with another...)
In this particular case things seem pretty innocuous: let's say two processes enter the #initialize. They both manage to enter the ifNil: (I'm aware this is compiled away to jumps, but the impact of this I don't know) through a time-of-check-time-of-use race, and you end up generating two objects. But one's simply lost. As long as references to Sentinel don't escape the class, no one need know anything at all.
right. but good point. i hadn't thought of this case. the scenarios i'm thinking of are those where one is loading/updating a monticello or fuel package or similar or redefining a class initialize method. in these scenarios there's only likely to be one thread of control running the initialize.
frank
But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan.
frank
-- best, Eliot
-- best, Eliot
On 19 January 2013 02:02, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Fri, Jan 18, 2013 at 2:38 PM, Frank Shearar <frank.shearar@gmail.com> wrote:
On 18 January 2013 21:49, Eliot Miranda <eliot.miranda@gmail.com> wrote:
phhh, let me try again :)
On Fri, Jan 18, 2013 at 8:47 AM, Frank Shearar <frank.shearar@gmail.com> wrote:
On 17 January 2013 17:45, Eliot Miranda <eliot.miranda@gmail.com> wrote:
On Thu, Jan 17, 2013 at 3:25 AM, Igor Stasenko <siguctua@gmail.com> wrote:
After some thought i decided to contribute my 2cents.
First, i think it is impossible to introduce a (re)initialization logic which would suit all different scenarios. Because it is really depends on what is stored in class variables and/or pools and if you add subclasses and then instances into the soup, you might end up with something which is really hard to safely reinitialize, because it may have some inconsistent state at different stages of initialization. It also, sometimes an order of initialization is important.
Yes, but on *can* write idempotent class initialization code. So that if the system does reinitialize on every load old code may break until its fixed, but new code will have the advantage of always being correctly initialized. Note that in the VMMaker class reinitializations of the core VM classes (the interpreter, garbage collector, jit, etc) are done *on each launch of the simulator*, and *each vm generation*, let alone on each package load :). [ I'm not recommending this :) ].
One thing I do is when creating singletons for sentinels etc I check whether they've been initialized. e.g.
initialize Sentinel ifNil: [Sentinel := Object new]
Knowing full well that I know next to nothing about concurrency in the image, and so realising that this might be a complete non-issue: lazy initialization and concurrency do not mix well.
Agreed, but the two can be decoupled. For example one can build the state and then assign it. Assignment is currently atomic so there are no multicore issues here (yet :) ). Further, assignments are not suspension points so a sequence of (just) assignments are guaranteed to occur without interruption. And (I'm sure you realize, for give the pedantry) the above example isn't lazy, it is eager. It simply refuses to create another Sentinel if one exists already.
So because of the assignment guarantees we won't have two processes entering the ifNil: block? (I'll try extract one foot from my mouth and maybe replace it with another...)
In this particular case things seem pretty innocuous: let's say two processes enter the #initialize. They both manage to enter the ifNil: (I'm aware this is compiled away to jumps, but the impact of this I don't know) through a time-of-check-time-of-use race, and you end up generating two objects. But one's simply lost. As long as references to Sentinel don't escape the class, no one need know anything at all.
right. but good point. i hadn't thought of this case. the scenarios i'm thinking of are those where one is loading/updating a monticello or fuel package or similar or redefining a class initialize method. in these scenarios there's only likely to be one thread of control running the initialize.
I've been bitten too many times not to be leery of that particular case :). I had a nasty bug with a lazy initialisation in a 3rd party library where a TCP server kept a thread pool. If the thread pool was nil it would make a new one (so the Delphi equivalent of f ifNil: [f := blah]). On application shutdown, a thing running in that threadpool could call back to the TCP server. In particular, the TCP server would ask all threads to terminate and then nil the pool... and one of the threads would call back and reinitialise a whole new threadpool. The last thing the application did was wait for all threads to finish, and knew this by decrementing a counter. Only that cause-a-new-threadpool thread would try decrement the counter of a brand new threadpool. The application would happily wait forever for a notification from a thread that was long dead! The solution was simplicity itself: strip out the lazy initialization, and you don't get a second threadpool accidentally made. frank
frank
But I do agree with your sentiment: idempotency is highly desirable, and somehow forcing non-idempotent class initialisation to fail quickly and noisily sounds like a good plan.
frank
-- best, Eliot
-- best, Eliot
participants (6)
-
Eliot Miranda -
Frank Shearar -
Igor Stasenko -
Norbert Hartl -
Stéphane Ducasse -
Sven Van Caekenberghe