August 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] [BUG] MethodContext allInstances size

by Igor Stasenko

2009/8/5 Marcus Denker <denker(a)acm.org>: > > On 05.08.2009, at 12:30, Igor Stasenko wrote: > >> 2009/8/5 Marcus Denker <denker(a)acm.org>: >>> >>> On 05.08.2009, at 12:00, StÃ©phane Ducasse wrote: >>> >>>> I did that in pharo 211 and it worked. >>>> >>> Yes, that was before closures. >>> >>>> Â Â Â MethodContext allInstances size 2309 >>>> >>> >>> with the blockClosures, creation of method contexts happes far more >>> frequently than it used to. >>> (as both closures and methods use the same context objects) >>> >>> It of course could be a sign that it happens actually too much. >>> >> Hmm, if that so, then we would expect to see an interpreter speed >> degradation. But its not. :) > > Yes, see the other mail. The reason is that the allocation of contexts > in case of blocks happens now at #value, whereas it used to be just done > at the definiton point (#blockCopy). > > So nothing to worry about. > Could it be rewritten to something like that: ( for all classes except UndefinedObject) allInstances | all inst next | all := OrderedCollection new. inst := self someInstance. inst ifNil: [ ^ #() ]. [inst == 0] whileFalse: [ next := inst nextObject. next class == self ifTrue: [ all add: next ]. next == all ifTrue: [ ^ all ]. inst := next]. ^ all here i added the stop rule ' next == all ifTrue: [ ^ all ].' , which should return from the loop once heap reached the last allocated object. The caller of #allInstances expects to see all instances of class existed before invocation of this method, and not interested in those ones, which was created as a side effect of running it. Right? >> I think it is correct behavior, that you can't get #allInstances of >> class when you creating them in a loop which serves to collect them >> all. >> The solution would be to collect #allInstances primitively (by a >> single primitive call), then there is no chance that anything happen >> with object memory during memory scan by primitive. > > > Yes, and the real solution of course is to change the VM to allocate > less > contexts and use the C-stack instead... > hmm, doesn't that breaks the smalltalk introspection capabilities, since then you can't see all the context objects which is used by interpreter for running your code? I want to point, that the way how VM allocating contexts is implementation details , and if it done correctly then you're still should be able to access all context objects. > Â Â Â Â Marcus > >>> >>> Â Â Â Â Marcus >>> >>> -- >>> Marcus Denker - http://marcusdenker.de >>> PLEIAD Lab - Computer Science Department (DCC) - University of Chile >>> >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by StÃ©phane Ducasse

yes. Will do that in the next integration phase. Stef On Aug 5, 2009, at 7:01 PM, Marcus Denker wrote: > > On 05.08.2009, at 12:42, Marcus Denker wrote: >>>> >>> Hmm, if that so, then we would expect to see an interpreter speed >>> degradation. But its not. :) >> >> Yes, see the other mail. The reason is that the allocation of >> contexts >> in case of blocks happens now at #value, whereas it used to be just >> done >> at the definiton point (#blockCopy). >> >> So nothing to worry about. > > > Eliot says: > > | So you can either > | - wait for the stack VM or > | - inline allInstancesDo: into allInstances or > | - implement allInstacesDo: specially in ContextPart or > | - implement a pair of primitives to answer allInstances and > allObject atomically. > | > | This latter approach allows much more flexibility in implementing > the garbage collector subsequently; for > | example segmenting the heap and adding and freeing segments as > required, which complicates the simple object > | ordering provided by the single heap but has much better memory > usage. > > > I vote for inling for now... this fixes the problem. > > allInstances > "Answer a collection of all current instances of the receiver." > > | all inst next | > all := OrderedCollection new. > inst := self someInstance. > [inst == nil] > whileFalse: [ > next := inst nextInstance. > inst == all ifFalse: [all add: inst]. > inst := next]. > ^ all asArray > > > Marcus > > > -- > Marcus Denker - http://marcusdenker.de > PLEIAD Lab - Computer Science Department (DCC) - University of Chile > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by StÃ©phane Ducasse

> Yes, see the other mail. The reason is that the allocation of contexts > in case of blocks happens now at #value, whereas it used to be just > done > at the definiton point (#blockCopy). > > So nothing to worry about. still it crashes my image when doing MethodContext allInstances may be a temporary solution would be to redefine allInstances on MethodContext >> I think it is correct behavior, that you can't get #allInstances of >> class when you creating them in a loop which serves to collect them >> all. >> The solution would be to collect #allInstances primitively (by a >> single primitive call), then there is no chance that anything happen >> with object memory during memory scan by primitive. > > > Yes, and the real solution of course is to change the VM to allocate > less > contexts and use the C-stack instead... > > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by StÃ©phane Ducasse

Soon I will be able to be a fly on your shoulder and learn much faster than via emails :) Stef >> > > Yes, but not "too much" in this sense. When creating new instances > while counting > them, the case "no problem" and "endless loop" are very close > together... a very > tiny increase, or even just a change in context-creation behavior > could trigger > the case that it creates too many contexts. > > This is one possbility: the change in creation pattern cause the > problem, nothing > to worry about. > > So let see. > > It used to be that block-context where created by sending blockCopy: > to thiscontext: > > 13 <89> pushThisContext: > 14 <75> pushConstant: 0 > 15 <C8> send: blockCopy: > 16 <A4 02> jumpTo: 20 > 18 <73> pushConstant: nil > 19 <7D> blockReturn > 20 <87> pop > 21 <78> returnSelf > > Now, the same code reads like this: > > 13 <8F 00 00 02> closureNumCopied: 0 numArgs: 0 bytes 17 to 18 > 17 <73> pushConstant: nil > 18 <7D> blockReturn > 19 <87> pop > 20 <78> returnSelf > > > and the closureNumCopied bytecode creates a BlockClosure object which > is *not* the > context. The context is later created when evaluating the Closure. > > So the time *when* an allocation of a new context may happen changed > fundamtally. > > If we now look at the code for getting allinstances: > > allInstances > "Answer a collection of all current instances of the receiver." > > | all | > all := OrderedCollection new. > self allInstancesDo: [:x | x == all ifFalse: [all add: x]]. > ^ all asArray > > (allInstancesDo: is carefully written to not use any real blocks, just > inlined ones). > > So the context for the block was created just once, before calling > #llInstancesDo: > Bytecode: > > 29 <40> pushLit: OrderedCollection > 30 <CC> send: new > 31 <68> popIntoTemp: 0 > 32 <70> self > 33 <89> pushThisContext: > 34 <76> pushConstant: 1 > 35 <C8> send: blockCopy: > 36 <A4 0B> jumpTo: 49 > .... and so on. > > Now with the Closures, it evaluates the closure in allInstancesDo:, > leading for the > closure VM to a potential allocation of a context in the execution of > allInstancesDo: > (and for each eval, itÂ´s a loop). So we now have potentiall > allocations of new instances > of methodcontext when iterating over the existing ones. --> race > condition. > > Marcus > > > > -- > Marcus Denker - http://marcusdenker.de > PLEIAD Lab - Computer Science Department (DCC) - University of Chile > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by Marcus Denker

On 05.08.2009, at 12:42, Marcus Denker wrote: >>> >> Hmm, if that so, then we would expect to see an interpreter speed >> degradation. But its not. :) > > Yes, see the other mail. The reason is that the allocation of contexts > in case of blocks happens now at #value, whereas it used to be just > done > at the definiton point (#blockCopy). > > So nothing to worry about. Eliot says: | So you can either | - wait for the stack VM or | - inline allInstancesDo: into allInstances or | - implement allInstacesDo: specially in ContextPart or | - implement a pair of primitives to answer allInstances and allObject atomically. | | This latter approach allows much more flexibility in implementing the garbage collector subsequently; for | example segmenting the heap and adding and freeing segments as required, which complicates the simple object | ordering provided by the single heap but has much better memory usage. I vote for inling for now... this fixes the problem. allInstances "Answer a collection of all current instances of the receiver." | all inst next | all := OrderedCollection new. inst := self someInstance. [inst == nil] whileFalse: [ next := inst nextInstance. inst == all ifFalse: [all add: inst]. inst := next]. ^ all asArray Marcus -- Marcus Denker - http://marcusdenker.de PLEIAD Lab - Computer Science Department (DCC) - University of Chile

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by Marcus Denker

On 05.08.2009, at 12:30, Igor Stasenko wrote: > 2009/8/5 Marcus Denker <denker(a)acm.org>: >> >> On 05.08.2009, at 12:00, StÃ©phane Ducasse wrote: >> >>> I did that in pharo 211 and it worked. >>> >> Yes, that was before closures. >> >>> MethodContext allInstances size 2309 >>> >> >> with the blockClosures, creation of method contexts happes far more >> frequently than it used to. >> (as both closures and methods use the same context objects) >> >> It of course could be a sign that it happens actually too much. >> > Hmm, if that so, then we would expect to see an interpreter speed > degradation. But its not. :) Yes, see the other mail. The reason is that the allocation of contexts in case of blocks happens now at #value, whereas it used to be just done at the definiton point (#blockCopy). So nothing to worry about. > I think it is correct behavior, that you can't get #allInstances of > class when you creating them in a loop which serves to collect them > all. > The solution would be to collect #allInstances primitively (by a > single primitive call), then there is no chance that anything happen > with object memory during memory scan by primitive. Yes, and the real solution of course is to change the VM to allocate less contexts and use the C-stack instead... Marcus >> >> Marcus >> >> -- >> Marcus Denker - http://marcusdenker.de >> PLEIAD Lab - Computer Science Department (DCC) - University of Chile >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Best regards, > Igor Stasenko AKA sig. > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by Marcus Denker

On 05.08.2009, at 12:08, StÃ©phane Ducasse wrote: >> I did that in pharo 211 and it worked. >>> >> Yes, that was before closures. >> >>> MethodContext allInstances size 2309 >>> >> >> with the blockClosures, creation of method contexts happes far more >> frequently than it used to. >> (as both closures and methods use the same context objects) >> >> It of course could be a sign that it happens actually too much. > > I stopped in the image that did not crash yet at a couple of > billions :) > Yes, but not "too much" in this sense. When creating new instances while counting them, the case "no problem" and "endless loop" are very close together... a very tiny increase, or even just a change in context-creation behavior could trigger the case that it creates too many contexts. This is one possbility: the change in creation pattern cause the problem, nothing to worry about. So let see. It used to be that block-context where created by sending blockCopy: to thiscontext: 13 <89> pushThisContext: 14 <75> pushConstant: 0 15 <C8> send: blockCopy: 16 <A4 02> jumpTo: 20 18 <73> pushConstant: nil 19 <7D> blockReturn 20 <87> pop 21 <78> returnSelf Now, the same code reads like this: 13 <8F 00 00 02> closureNumCopied: 0 numArgs: 0 bytes 17 to 18 17 <73> pushConstant: nil 18 <7D> blockReturn 19 <87> pop 20 <78> returnSelf and the closureNumCopied bytecode creates a BlockClosure object which is *not* the context. The context is later created when evaluating the Closure. So the time *when* an allocation of a new context may happen changed fundamtally. If we now look at the code for getting allinstances: allInstances "Answer a collection of all current instances of the receiver." | all | all := OrderedCollection new. self allInstancesDo: [:x | x == all ifFalse: [all add: x]]. ^ all asArray (allInstancesDo: is carefully written to not use any real blocks, just inlined ones). So the context for the block was created just once, before calling #llInstancesDo: Bytecode: 29 <40> pushLit: OrderedCollection 30 <CC> send: new 31 <68> popIntoTemp: 0 32 <70> self 33 <89> pushThisContext: 34 <76> pushConstant: 1 35 <C8> send: blockCopy: 36 <A4 0B> jumpTo: 49 .... and so on. Now with the Closures, it evaluates the closure in allInstancesDo:, leading for the closure VM to a potential allocation of a context in the execution of allInstancesDo: (and for each eval, itÂ´s a loop). So we now have potentiall allocations of new instances of methodcontext when iterating over the existing ones. --> race condition. Marcus -- Marcus Denker - http://marcusdenker.de PLEIAD Lab - Computer Science Department (DCC) - University of Chile

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by Igor Stasenko

2009/8/5 Marcus Denker <denker(a)acm.org>: > > On 05.08.2009, at 12:00, StÃ©phane Ducasse wrote: > >> I did that in pharo 211 and it worked. >> > Yes, that was before closures. > >> Â Â Â MethodContext allInstances size 2309 >> > > with the blockClosures, creation of method contexts happes far more > frequently than it used to. > (as both closures and methods use the same context objects) > > It of course could be a sign that it happens actually too much. > Hmm, if that so, then we would expect to see an interpreter speed degradation. But its not. :) I think it is correct behavior, that you can't get #allInstances of class when you creating them in a loop which serves to collect them all. The solution would be to collect #allInstances primitively (by a single primitive call), then there is no chance that anything happen with object memory during memory scan by primitive. > > Â Â Â Â Marcus > > -- > Marcus Denker - http://marcusdenker.de > PLEIAD Lab - Computer Science Department (DCC) - University of Chile > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > -- Best regards, Igor Stasenko AKA sig.

Aug. 5, 2009

Re: [Pharo-project] [BUG] MethodContext allInstances size

by StÃ©phane Ducasse

> I did that in pharo 211 and it worked. >> > Yes, that was before closures. > >> MethodContext allInstances size 2309 >> > > with the blockClosures, creation of method contexts happes far more > frequently than it used to. > (as both closures and methods use the same context objects) > > It of course could be a sign that it happens actually too much. I stopped in the image that did not crash yet at a couple of billions :)

Aug. 5, 2009

Re: [Pharo-project] Wrong values of variables showed in execution context when doing self break?

by Mariano Martinez Peck

2009/8/5 Javier PimÃ¡s <elpochodelagente(a)gmail.com> > I tested in latest (pharo1.0beta-10401web09.07.5) and noticed that now > pharo uses the same debugger for breakpoints and for "debugits", which is > nice. The bug I mentioned seems to be in OTDebugger, which has been removed > from the image, so it could be said it's "fixed". > > Now, this debugger has ooother problem... if doing the same as before > (AClassToTest doSomeThingWith: 14@17) in the bottom right list appears " > stack top", and if I click it, an error pops up (AClassToTest DNU: #asText). > So what happened with #asText? Please. Open a bug ticket. I saw this one: http://code.google.com/p/pharo/issues/detail?id=724 seems similar but not the same. > > > Also more questions: with this debugger there isn't any step out button, > nor a return one, which I found useful often. Is this going to stay that > way? > It is a matter of the toolset. OBPackageBrowser and all that staff were removed from the image because they were very slow. We decided to remove them until they are fast enought. However, you can install them and set them as default is you want. Does someone know how can I install OBPackageBrowser and that ToolSet ? the one that was used before ? It would be nice to put that piece of code in here: http://code.google.com/p/pharo/wiki/CodeSnippets > > Another thing I noticed is that OBPackageBrowser has also been removed, and > the actual class browser doesn't have a search bar on top. How do you search > for a class if you don't know in what package is defined? I feel really lost > without the search bar... See stef answer or install OBPackageBrowser > > > Thanks, > Javier. > > 2009/8/4 Mariano Martinez Peck <marianopeck(a)gmail.com> > > >> >> On Tue, Aug 4, 2009 at 8:24 AM, Damien Cassou <damien.cassou(a)gmail.com>wrote: >> >>> 2009/8/4 Javier PimÃ¡s <elpochodelagente(a)gmail.com>: >>> > >>> > Here is an example to reproduce it. Maybe somebody has already opened a >>> > ticket for this problem. To try you can doit the following code and >>> look at >>> > the values shown of x and y in the bottom left list of debugger. >>> > >>> > AClassToTest doSomeThingWith: 14@17. >>> >>> I can't reproduce your bug. Please try again in a 10402 image or later >>> and open an issue tagged Milestone-1.0 if you can still reproduce it. >>> Also attach the code to the issue so that people can load easily. >> >> >> >> I also did it but didn't find something strange. Perhaps a screenshot may >> help. >> >> best, >> >> mariano >> >> >>> >>> >>> -- >>> Damien Cassou >>> http://damiencassou.seasidehosting.st >>> >>> "Lambdas are relegated to relative obscurity until Java makes them >>> popular by not having them." James Iry >>> >>> _______________________________________________ >>> Pharo-project mailing list >>> Pharo-project(a)lists.gforge.inria.fr >>> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >>> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > > > > -- > Javier PimÃ¡s > Ciudad de Buenos Aires > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Aug. 5, 2009