May 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] [squeak-dev] [BUG][FIX] CompiledMethodTrailer

by Mariano Martinez Peck

On Tue, May 10, 2011 at 1:24 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: > Okay, seems like i found the bug. > > > (CompiledMethodTrailer new sourcePointer: 0) testEncoding > > fails. > > And it happens only when you pass 0 as source pointer!! > Such an obscure case! > So...this means that after a day and a half of work of the whole community we finally fix it?? I am happy. Thanks Igor and the rest who helped as well :) Now...I am not sure if I understand. I saw your changes and you test. But my question is what happen with those incorrect CM that are arround. It means that the next build image, when it does the CompiledMethod cleanUp and destroy the sourcePointers, I will let correct CMs ? Does it mean that if I build a dev image now with such fix, I won't have these weird CMs bothering me? Ok....I am trying now.... Thanks a lot to everybody for the effort, Mariano > > @squeakers, for the context, check the discussion "Problem with > CompiledMethodTrailer" > in pharo-dev list. > > On 10 May 2011 12:47, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > > >> > >> > > >> > I must be wrong, because the dev image when it is build, it calls all > >> > cleanUp methods for classes, so CompiledMethod class >> cleanUp shuld > >> > have > >> > been called, but if I take the Pharo1.3 result, I can see the source > >> > code > >> > (not decompiled) > >> > > >> > >> But you can't see the source of those methods in browser, because > >> browser works only with installed methods > >> while cleanup touching only non-installed ones. > > > > > > grrrr sorry, I forgot about the #isInstalled ... I went directly to see > > #destroySourcePointer > > heheheh > > > >> > >> >> > >> >> When i do this, my image hangs. > >> > > >> > I have just tried and it crash!!! Actually, I evaluated > CompiledMethod > >> > cleanUp. And just after cmd+b in CompiledMethod (to browse it) -> > crash > >> > How it is possilbe it didn't crash while building the dev image? or > >> > the > >> > first time I browse a class? > >> > maybe this incorrect CM are created by the build process just after > the > >> > cleanUp ? > >> > > >> I don't know yet. > >> Maybe there is some test, which breaking them when it runs? Or tests > >> are not run on those images? > > > > I was thinking exactly the same. But no, this image has not run the > tests. > > There are other images specially to download from hudson that are those > > after running tests :( > > > > > > > > -- > > Mariano > > http://marianopeck.wordpress.com > > > > > > > > -- > Best regards, > Igor Stasenko AKA sig. > > > > -- Mariano http://marianopeck.wordpress.com

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

The fix in http://code.google.com/p/pharo/issues/detail?id=4186 now one thing left: we need to get rid of hanging methods with corrupted trailers. And we do it like this: (CompiledMethod allInstances select: [:m | m isInstalled not and: [ m trailer kind = #VarLengthSourcePointer ]] ) do: [:each | each at: each size put: 0 ] This will replace a corrupted trailer with empty one. -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] [BUG][FIX] CompiledMethodTrailer

by Igor Stasenko

And here the test , which covering this issue. -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Henrik Sperre Johansen

On 10.05.2011 12:44, Igor Stasenko wrote: > On 10 May 2011 11:27, Mariano Martinez Peck<marianopeck(a)gmail.com> wrote: >> >> On Tue, May 10, 2011 at 10:59 AM, Igor Stasenko<siguctua(a)gmail.com> wrote: >>> On 10 May 2011 09:48, Mariano Martinez Peck<marianopeck(a)gmail.com> wrote: >>>> >>>> On Tue, May 10, 2011 at 9:41 AM, Marcus Denker<marcus.denker(a)inria.fr> >>>> wrote: >>>>> On May 10, 2011, at 9:32 AM, Mariano Martinez Peck wrote: >>>>> >>>>>> what is even worst, is that even after removing those correct CM, and >>>>>> doing a GC etc...they still don't disappear. >>>>>> >>>>> There is >>>>> >>>>> ScriptLoader new cleanUpMethods >>>>> >>>>> This is called from #cleanUpForRelease and normally should make sure >>>>> there >>>>> are no old methods. >>>>> >>>> >>>> hehehehe if I do: ScriptLoader new cleanUpForRelease, then inspect >>>> ((CompiledMethod allInstances select: [:each | each trailer kind = >>>> #VarLengthSourcePointer] ) ) >>>> >>>> they are not GC'ed and if then I click on the first element in the >>>> inspector.... VM CRASH!!! with both, InterpreterVM and CogVM. >>>> >>>> :( >>>> >>> okay. it seems i found the offender. Its a CompiledMethod class>>cleanUp. >>> It changing a source pointer of all non-installed methods to 0. >> Good catch!!! :) Anyway, why CompiledMethod class>>cleanUp would like to >> destroy source pointers??? >> If I understood correctly (please correct me), all compiled methods will >> loose the pointer to sources and hence they will be decompiled after when >> they are ask their source! > Yes, if you wipe the sourcePointer, there is no way how you can get a > source code of method. I believe the "not getting the source code" was on purpose. It's done f.ex. as part of condenseChanges, as the source pointers would have been invalid anyways. The intent I believe is that you should get decompiled source if you inspect old(but still referenced) CompiledMethods, instead of a bogus display of some random part of the new changes file, and remove error handling in source lookup where pointer could potentially be beyond end of file. This of course worked much faster before trailers, when setSourcePointer: actually modified the instance inline, rather than become'ing a copy, nowadays I agree with Igor that it'd probably be more conceptually clean to the empty MethodTrailer in destroySourcePointer :) Just remember to make sure such instances decompile as expected to uphold why the zapping was done in the first place. If we're lucky, it won't crash the VM either ;) Cheers, Henry

May 10, 2011

[Pharo-project] [BUG][FIX] CompiledMethodTrailer

by Igor Stasenko

Okay, seems like i found the bug. (CompiledMethodTrailer new sourcePointer: 0) testEncoding fails. And it happens only when you pass 0 as source pointer!! Such an obscure case! @squeakers, for the context, check the discussion "Problem with CompiledMethodTrailer" in pharo-dev list. On 10 May 2011 12:47, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > >> >> > >> > I must be wrong, because the dev image when it is build, it calls all >> > cleanUp methods for classes, so CompiledMethod class >> cleanUp shuld >> > have >> > been called, but if I take the Pharo1.3 result, I can see the source >> > code >> > (not decompiled) >> > >> >> But you can't see the source of those methods in browser, because >> browser works only with installed methods >> while cleanup touching only non-installed ones. > > > grrrr sorry, I forgot about the #isInstalled ... I went directly to see > #destroySourcePointer > heheheh > >> >> >> >> >> When i do this, my image hangs. >> > >> > I have just tried and it crash!!!Â Actually, I evaluated CompiledMethod >> > cleanUp. And just after cmd+b in CompiledMethod (to browse it) -> crash >> > How it is possilbe it didn't crash while building the dev image?Â Â or >> > the >> > first time I browse a class? >> > maybe this incorrect CM are created by the build process just after the >> > cleanUp ? >> > >> I don't know yet. >> Maybe there is some test, which breaking them when it runs? Or tests >> are not run on those images? > > I was thinking exactly the same. But no, this image has not run the tests. > There are other images specially to download from hudson that are those > after running tests :( > > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] Time to rename copyWithTrailerBytes: ?

by Igor Stasenko

On 10 May 2011 12:35, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > If I understand correctly, before the trailers we used to use just an array > of bytes...now we use a CompiledMethodTrailer. So, the method > copyWithTrailerBytes:Â is misleading. > yes. it should be named copyWithTrailer: instead, but since at the time i introducing trailers there was only bytes, i had to preserve the name for compatibility (among other things). Btw, if you look at CompiledMethod, there's a lot of things which can be cleaned-up/refactored. For example all code which managing sourcePointer(s) and SourceFiles, not belongs to CompiledMethod. It should reside somewhere in SourceFileArray (or its subclass). > cheers > > -- > Mariano > http://marianopeck.wordpress.com > > -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Mariano Martinez Peck

> > > > > I must be wrong, because the dev image when it is build, it calls all > > cleanUp methods for classes, so CompiledMethod class >> cleanUp shuld > have > > been called, but if I take the Pharo1.3 result, I can see the source code > > (not decompiled) > > > > But you can't see the source of those methods in browser, because > browser works only with installed methods > while cleanup touching only non-installed ones. > grrrr sorry, I forgot about the #isInstalled ... I went directly to see #destroySourcePointer heheheh > > >> > >> When i do this, my image hangs. > > > > I have just tried and it crash!!! Actually, I evaluated CompiledMethod > > cleanUp. And just after cmd+b in CompiledMethod (to browse it) -> crash > > How it is possilbe it didn't crash while building the dev image? or the > > first time I browse a class? > > maybe this incorrect CM are created by the build process just after the > > cleanUp ? > > > I don't know yet. > Maybe there is some test, which breaking them when it runs? Or tests > are not run on those images? > I was thinking exactly the same. But no, this image has not run the tests. There are other images specially to download from hudson that are those after running tests :( -- Mariano http://marianopeck.wordpress.com

May 10, 2011

Re: [Pharo-project] Problem with CompiledMethodTrailer

by Igor Stasenko

On 10 May 2011 11:27, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > > On Tue, May 10, 2011 at 10:59 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> On 10 May 2011 09:48, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: >> > >> > >> > On Tue, May 10, 2011 at 9:41 AM, Marcus Denker <marcus.denker(a)inria.fr> >> > wrote: >> >> >> >> On May 10, 2011, at 9:32 AM, Mariano Martinez Peck wrote: >> >> >> >> > >> >> > what is even worst, is that even after removing those correct CM, and >> >> > doing a GC etc...they still don't disappear. >> >> > >> >> >> >> There is >> >> >> >> Â Â Â Â ScriptLoader new cleanUpMethods >> >> >> >> This is called from #cleanUpForRelease and normally should make sure >> >> there >> >> are no old methods. >> >> >> > >> > >> > hehehehe if I do: ScriptLoader new cleanUpForRelease, then inspect >> > ((CompiledMethod allInstances select:Â [:each | each trailer kind = >> > #VarLengthSourcePointer] ) ) >> > >> > they are not GC'ed and if then I click on the first element in the >> > inspector.... VM CRASH!!!Â with both, InterpreterVM and CogVM. >> > >> > :( >> > >> >> okay. it seems i found the offender. Its a CompiledMethod class>>cleanUp. >> It changing a source pointer of all non-installed methods to 0. > > Good catch!!!Â :)Â Â Anyway, why CompiledMethod class >>cleanUp would like to > destroy source pointers??? > If I understood correctly (please correct me), all compiled methods will > loose the pointer to sources and hence they will be decompiled after when > they are ask their source! Yes, if you wipe the sourcePointer, there is no way how you can get a source code of method. > > I must be wrong, because the dev image when it is build, it calls all > cleanUp methods for classes, so CompiledMethod class >> cleanUp shuld have > been called, but if I take the Pharo1.3 result, I can see the source code > (not decompiled) > But you can't see the source of those methods in browser, because browser works only with installed methods while cleanup touching only non-installed ones. >> >> When i do this, my image hangs. > > I have just tried and it crash!!!Â Actually, I evaluated CompiledMethod > cleanUp. And just after cmd+b in CompiledMethod (to browse it) -> crash > How it is possilbe it didn't crash while building the dev image?Â Â or the > first time I browse a class? > maybe this incorrect CM are created by the build process just after the > cleanUp ? > I don't know yet. Maybe there is some test, which breaking them when it runs? Or tests are not run on those images? > >> >> Now i thinking that it should actually use empty trailer for those >> methods i.e. CompiledMethodTrailer empty.. >> >> Still i found it strange why some methods has broken bytecode. > > Yes!! and why they are not GCed!! > >> >> My theory is that some nasty code somewhere copied a compiled methods >> using some unsafe technique, like: >> >> 1 to: oldMethod size do: [ :i | newMethod at: i put: (oldMethod at: i) ] > > I will search to see if I find something. > The thing is that it is reproducible. Each new dev image build from scratch > in a core, has these guys. > >> >> Because when you constructing compiled method normally (by compiling >> them) , there is no way how you can get broken compiled methods. (we >> would discover it much much earlier, > > yes, exactly. > >> >> and in fact CompiledMethodTrailers are working for more than a year in >> both pharo and squeak, and there was no any issues like this from >> their side). >> > > +1 > >> >> -- >> Best regards, >> Igor Stasenko AKA sig. >> > > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Best regards, Igor Stasenko AKA sig.

May 10, 2011

[Pharo-project] Time to rename copyWithTrailerBytes: ?

by Mariano Martinez Peck

If I understand correctly, before the trailers we used to use just an array of bytes...now we use a CompiledMethodTrailer. So, the method copyWithTrailerBytes: is misleading. cheers -- Mariano http://marianopeck.wordpress.com

May 10, 2011

Re: [Pharo-project] Issue in loading SqueakDBX Patch

by Mariano Martinez Peck

On Tue, May 10, 2011 at 11:40 AM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > Hi. I cannot understand why you can use the MonticelloBrowser but not > Gofer. Are you behind a proxy? if true, check in the menu -> settings -> > Network -> Use HTTP proxy and "Proxy authentification". > > The packages to load SqueakDBX IN ORDER are: > > From repo http://source.squeakfoundation.org/FFI > > load : FFI-Pools-eem.2 > load : FFI-Kernel-tbn.20 > load : FFI-Tests-jcg.4 > load : FFI-MacOS-spd.5 > load : FFI-Unix-tbn.3 > load : FFI-Win32-ar.3 > > From repo http://www.squeaksource.com/SqueakDBX > > load : SqueakDBX-Smalltalk-Dialect-tbn.6 > load : SqueakDBX-Dialect-Squeak-MarianoMartinezPeck.6 > load : OpenDBX-Core-GuillermoPolito.283 > > > I don't want to dissapoint you, but be aware that for SqueakDBX you need > the library OpenDBX plus the native database libraries. This is explained in > http://www.squeakdbx.org/Architecture%20and%20desing > > In addition, this should be of help: > > http://www.squeakdbx.org/Compiling%20and%20installing%20OpenDBX > http://www.squeakdbx.org/Compiling%20for%20different%20backends > > Finally, notice that when you have an error like the one of Gofer, you can > send by email PharoDebug.log which should have been created in the same > directory where your .image is. Such file contains all the stacktrace. > > Cheers > > Mariano > > > On Tue, May 10, 2011 at 8:27 AM, sourav roy <sourav_pharo(a)in.com> wrote: > >> >> Hi Peck, >> >> Instead of running the Gofer script, can I download the each package and >> load it in my Pharo. This I am asking because the office environment is >> secured because of which am not able to do load the required configuration >> packages via script. Please if it is possible for you the suggest me the >> names of the packages required to load SqueakDBX. >> >> P.S:Screen shot of the error given below. >> >> Thanks&Regards, >> >> [image: gogreen] >> >> *Sourav Roy|Misys Loan IQ Dev Team|HCL >> Technologies|SEZ-Bangalore|Mob:8861133201* >> >> *From:* Mariano Martinez Peck [mailto:marianopeck@gmail.com] >> *Sent:* Monday, May 09, 2011 6:06 PM >> *To:* Sourav Roy >> *Subject:* Re: Re: [Pharo-project] Issue in loading SqueakDBX Patch >> >> On Mon, May 9, 2011 at 1:30 PM, sourav roy <sourav_pharo(a)in.com> wrote: >> >> Hi, >> >> I loaded the following patches in pharo 1.2.1 which i downloaded from >> http://www.squeaksource.com/SqueakDBX. >> >> -ConfigurationOfSqueakDBX-MarianoMartinezPeck.8 >> -penDBX-Core-MarianoMartinezPeck.256 >> -queakDBX-Dialect-Squeak-MarianoMartinezPeck.3 >> -queakDBX-Smalltalk-Dialect-MarianoMartinezPeck.5 >> >> >> Loading that is not correct because: >> 1) the order is not correct (for example SqueakDBX-Smalltalk-Dialect >> should be first than SqueakDBX-Dialect-Squeak) >> 2) you are not loading SqueakDBX dependencies like FFI (and in fact the >> sintax error you have is because you are not loading FFI). >> >> To solves those issues, we have a Metacello configuration. Gofer has >> nothing to do with a secured environment or not: all you have to do is to >> load the package ConfigurationOfSqueakDBX. No matter how: you can use the >> following Gofer script: >> >> Gofer new >> squeaksource: 'MetacelloRepository'; >> package: 'ConfigurationOfSqueakDBX'; >> load. >> >> Or, open a Monticello Browser, -> add repository -> >> www.squeaksource.com/MetacelloRepository -> select >> ConfiguratonOfSqueakDBX -> load. >> >> Once you have download ConfigurationOfSqueakDBX all you have to do is: >> >> (ConfigurationOfSqueakDBX project version: #stable) load. >> >> I *really* recommend you to read the Metacello chapter of Pharo By >> Example, since all this is explained there: >> https://gforge.inria.fr/frs/download.php/28462/Metacello.pdf >> >> Cheers >> >> Mariano >> >> Please advice me if am wrong and suggest me the correct method for it. >> I am not able to load it through Gofer as am working in a secured >> environment. >> >> Thanks, >> >> Sourav >> >> >> ---------- Original message ---------- >> From:"Mariano Martinez Peck"< marianopeck(a)gmail.com > >> Date: 9 May 11 14:55:24 >> Subject: Re: [Pharo-project] Issue in loading SqueakDBX Patch >> To: Pharo-project(a)lists.gforge.inria.fr; >> >> Hi....that's weird. Could you tell me please how did you exactly install >> SqueakDBX? >> >> >> On Mon, May 9, 2011 at 11:19 AM, sourav roy <sourav_pharo(a)in.com> wrote: >> >> Hi All, >> >> I am getting some error while loading a SqueakDBX patch in Pharo 1.2.1. >> Please see the below screen shot attcahed and advice >> >> *Error! Filename not specified.* >> >> *Thanks,* >> >> *Sourav Roy| Bangalore|INDIA * >> >> ------------------------------ >> >> ::DISCLAIMER:: >> >> ----------------------------------------------------------------------------------------------------------------------- >> >> The contents of this e-mail and any attachment(s) are confidential and >> intended for the named recipient(s) only. >> It shall not attach any liability on the originator or HCL or its >> affiliates. Any views or opinions presented in >> this email are solely those of the author and may not necessarily reflect >> the opinions of HCL or its affiliates. >> Any form of reproduction, dissemination, copying, disclosure, >> modification, distribution and / or publication of >> this message without the prior written consent of the author of this >> e-mail is strictly prohibited. If you have >> received this email in error please delete it and notify the sender >> immediately. Before opening any mail and >> attachments please check them for viruses and defect. >> >> ---------------------------------------------------------- >> ------------------------------------------------------------- >> >> Dear *pharo-project !* Get Yourself a cool, short *(a)in.com* Email ID >> now!<http://secure.in.com/sso/commonreg.php?utm_source=invite&utm_medium=outgoing> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> Get Yourself a cool, short *(a)in.com* Email ID now!<http://mail.in.com/mails/new_reg.php?utm_source=invite&utm_medium=outgoing> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> Get Yourself a cool, short *(a)in.com* Email ID now!<http://mail.in.com/mails/new_reg.php?utm_source=invite&utm_medium=outgoing> >> > > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

May 10, 2011