March 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] osprocess in pharo 2.0

by Camillo Bruni

thanks, I changed it and it sort of works: https://ci.inria.fr/pharo-contribution/job/OSProcess/ though an impressive number of 81 tests fail, that is almost 50% of all the tests. Is that normal? On 2013-03-08, at 17:13, "Sean P. DeNigris" <sean(a)clipperadams.com> wrote: > On Mar 8, 2013, at 3:49 AM, Camillo Bruni-3 [via Smalltalk] <ml-node+s1294792n4675680h68(a)n4.nabble.com> wrote: >> I wonder why it fails on the build-server? > It fails because ThisOSProcess class>>shutDown: includes "OSProcess accessor breakDependents", but OSProcess accessor returns nil because none of the platform packages are loaded by the 'Tests' group of ConfigurationOfOSProcess. > > You can load 'Tests' and 'All OS' via "--groups='All OS',Tests" with the slice for "Issue 7662: [ENH]: ConfigurationCommandLineHandler: Accept multiple group names" > > Going forward, I'm not sure whether Tests should depend on the OS packages. You may want to load just the current OS, for example. I think probably a good error message and maybe a retry to find a suitable OS class if the accessor is nil... > > cc: David Lewis > > > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/osprocess-in-pharo-2-0-tp4674453p4675783.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

March 8, 2013

Re: [Pharo-project] Porting SmaCC from 1.4 to 2.0; doSilently: equivalent ?

by Goubier Thierry

Le 08/03/2013 16:58, Mariano Martinez Peck a Ã©crit : > > > On Fri, Mar 8, 2013 at 12:52 PM, Goubier Thierry <thierry.goubier(a)cea.fr > <mailto:thierry.goubier@cea.fr>> wrote: > > Hi, > > I'm trying to port SmaCC to 2.0 and I'm looking for the equivalent > to SystemChangeNotifier>>__doSilently:. Is it > SystemAnnouncer>>__suspendAllWhile:? > > (doSilently: is used in SmaCC tests). > > > There is also a nice #valueWithoutNotifications in BlockClosure. > > BTW, does it make sense to port SmaCC? > Which use case do you have in which you prefer SmaCC over say PetitParse? > I took a look few weeks ago to something done in SmaCC and it was full > of ugly unmaintainable class side tables..... If you are OK with grammars as a way to clearly specify a syntax, and you don't have too much context in it, a LALR parser as SmaCC is hard to beat (fairly short to write, and to maintain *IF* you use SmaCC-Dev). Moreover, SmaCC lexers being entirely compiled in Smalltalk (i.e. not a simulation of an automaton but direct compilation of the minimized DFA), it's hard to be any faster. And never, never touch the tables by hand. They are all generated ! The only thing you need to understand a SmaCC generated parser and scanner are the comments class-side. For the kind of parsing I do, I need SmaCC. I don't know yet if PetitParser can take the load. Thierry > Thanks, > > Thierry > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 > > > > > -- > Mariano > http://marianopeck.wordpress.com -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

March 8, 2013

Re: [Pharo-project] VM Crashes

by Mariano Martinez Peck

On Fri, Mar 8, 2013 at 11:44 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 08 Mar 2013, at 13:09, Camillo Bruni <camillobruni(a)gmail.com> wrote: > > > On 2013-03-08, at 11:53, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > >> On 08 Mar 2013, at 11:16, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> > >>> But I have said this before: the wall clock time of loading a lot of > code is actually close to unacceptable - I don't think it is the download > or the compilation, but more all the dynamic stuff that happens after that. > There should be a way to not do all those updates for each method and move > the updates to one big batch update after the load - if that is possible. > >> > >> To continue my rant (sorry ;-) about the problem with slow code loading. > >> > >> These are some benchmarks on the same machine: > >> > >> $ ./vm.sh experimental.image eval '[Smalltalk allClassesAndTraits do: > #compileAll] timeToRun' > >> 106532 > >> > >> $ ./stack/vm.sh experimental.image eval '[Smalltalk allClassesAndTraits > do: #compileAll] timeToRun' > >> 221708 > >> > >> So it takes like 3 minutes to recompile every method in the system. > >> > >> How in the hell can it take 40 minutes to load some code (with all > packages already present in the package-cache (but then again the > package-cache is only 3.5 Mb, which could be downloaded in seconds)) ? > > > > I with you on this one! :D I always wondered what Metacello and Compiler > do there! :D > > Well, open the Time Profiler and load your favourite Config. You'll have a > hard time finding the actual time slice of the compiler. As far as I could > see in my run it was less than 1% ! All the rest MC, Gofer, RPackage and > SystemAnnouncement stuff, even completion stuff. > > Indeed, we noticed this as well when benchmarking Tanker to replace source compilation in Monticello. We got to a similar conclusion. Quite a challenge to optimize if you ask me. > > - Batch notifications: do not do a notification per method/class added. Instead do one notification with all the methods/classes together. -- Mariano http://marianopeck.wordpress.com

March 8, 2013

Re: [Pharo-project] VM Crashes

by Esteban Lorenzano

yes, thanks to Eliot fixes, latest VMs are a lot more stable... but looks that the problem is still arising in certain weird situations :( On Mar 8, 2013, at 5:13 PM, Paul DeBruicker <pdebruic(a)gmail.com> wrote: > Does that happen when you use the VM from the top of the list here: > > http://pharo.gforge.inria.fr/ci/vm/pharo/linux/ > > specifically: > http://pharo.gforge.inria.fr/ci/vm/pharo/linux/pharo-linux-stable.zip > > ? > > > I haven't had the problem since switching to it when loading the config of > Seaside 31 on Ubuntu 12.04 > > > > > > > Sven Van Caekenberghe-2 wrote >> Thanks. >> >> But my 2nd and 3rd runs crashed, both at >> >> Smalltalk stack dump: >> 0xbf9c4c18 M MethodDictionary(Object)>becomeForward: 0x79299080: a(n) >> MethodDictionary >> 0xbf9c4c44 M MethodDictionary>grow 0x79299080: a(n) MethodDictionary >> 0xbf9c4c5c M MethodDictionary(HashedCollection)>fullCheck 0x79299080: a(n) >> MethodDictionary >> 0xbf9c4c78 M MethodDictionary>at:put: 0x79299080: a(n) MethodDictionary >> >> so it is still becomeForward: but while growing a MethodDictionary, which >> is a pretty basic operation I guess. >> >> On 08 Mar 2013, at 15:46, Esteban Lorenzano < > >> estebanlm@ > >> > wrote: >> >>> included :) >>> >>> On Mar 8, 2013, at 2:52 PM, Sven Van Caekenberghe < > >> sven@ > >> > wrote: >>> >>>> Hi Esteban, >>>> >>>> On 08 Mar 2013, at 12:44, Esteban Lorenzano < > >> estebanlm@ > >> > wrote: >>>> >>>>> Hi Sven, >>>>> >>>>> One of the problems we currently have is that all method installation >>>>> execute a becomeForward:, which is of course not terrible cleaver... I >>>>> tried using the squeak implementation of it and everything looks >>>>> working (and it has a small optimization that can help). >>>>> >>>>> Can you trying the load by installing it? >>>>> >>>>> CompiledMethod>>#setSourcePointer: srcPointer >>>>> "We can't change the trailer of existing method, since it could have a >>>>> completely different format. Therefore we need to generate a copy >>>>> with new trailer, containing a srcPointer, and then #become it." >>>>> | trailer copy start | >>>>> >>>>> trailer := srcPointer = 0 >>>>> ifTrue: [ >>>>> "catch the common case of setting the source pointer to 0 when >>>>> already 0" >>>>> self sourcePointer = 0 ifTrue: [ ^ self ]. >>>>> CompiledMethodTrailer empty ] >>>>> ifFalse: [ >>>>> CompiledMethodTrailer new sourcePointer: srcPointer ]. >>>>> copy := self copyWithTrailerBytes: trailer. >>>>> >>>>> "ar 3/31/2010: Be a bit more clever since #become: is slow. >>>>> If the old and the new trailer have the same size, just replace it." >>>>> (self trailer class == trailer class and:[ self size = copy size ]) >>>>> ifTrue: [ >>>>> start := self endPC + 1. >>>>> self replaceFrom: start to: self size with: copy startingAt: start ] >>>>> ifFalse: [ >>>>> self becomeForward: copy ]. >>>>> >>>>> ^self "will be copy if #become was needed" >>>>> >>>>> this can help in the speed and also can help on the vm crash problem >>>>> (even if it is a workaround)... >>>>> >>>>> Esteban >>>> >>>> I used the following version >>>> >>>> CompiledMethod>>#setSourcePointer: srcPointer >>>> "We can't change the trailer of existing method, since it could have >>>> completely different format. >>>> Therefore we need to generate a copy with new trailer, containing >>>> scrPointer, and then become it." >>>> >>>> | trailer copy | >>>> trailer := CompiledMethodTrailer new sourcePointer: srcPointer. >>>> copy := self copyWithTrailerBytes: trailer. >>>> "If possible do a replace in place as an optimization" >>>> (self trailer class == trailer class and: [ self size = copy size ]) >>>> ifTrue: [ >>>> | start | >>>> start := self endPC + 1. >>>> self replaceFrom: start to: self size with: copy startingAt: start ] >>>> ifFalse: [ self becomeForward: copy ]. >>>> ^ self >>>> >>>> And it worked fine. My build completed successfully (1 try only). >>>> But it took about as long as before (10 minutes using a Cog based VM). >>>> >>>> Thanks again for the suggestion, I think we should incorporate that >>>> change, no ? >>>> >>>> The class test is ugly and most probably not necessary in the current >>>> image, I would say. >>>> >>>> Sven >>>> >>>>> On Mar 8, 2013, at 11:53 AM, Sven Van Caekenberghe < > >> sven@ > >> > wrote: >>>>> >>>>>> >>>>>> On 08 Mar 2013, at 11:16, Sven Van Caekenberghe < > >> sven@ > >> > wrote: >>>>>> >>>>>>> But I have said this before: the wall clock time of loading a lot of >>>>>>> code is actually close to unacceptable - I don't think it is the >>>>>>> download or the compilation, but more all the dynamic stuff that >>>>>>> happens after that. There should be a way to not do all those updates >>>>>>> for each method and move the updates to one big batch update after >>>>>>> the load - if that is possible. >>>>>> >>>>>> To continue my rant (sorry ;-) about the problem with slow code >>>>>> loading. >>>>>> >>>>>> These are some benchmarks on the same machine: >>>>>> >>>>>> $ ./vm.sh experimental.image eval '[Smalltalk allClassesAndTraits do: >>>>>> #compileAll] timeToRun' >>>>>> 106532 >>>>>> >>>>>> $ ./stack/vm.sh experimental.image eval '[Smalltalk >>>>>> allClassesAndTraits do: #compileAll] timeToRun' >>>>>> 221708 >>>>>> >>>>>> So it takes like 3 minutes to recompile every method in the system. >>>>>> >>>>>> How in the hell can it take 40 minutes to load some code (with all >>>>>> packages already present in the package-cache (but then again the >>>>>> package-cache is only 3.5 Mb, which could be downloaded in seconds)) ? >>>>>> >>>>>> Sven >>>>>> >>>>>> -- >>>>>> Sven Van Caekenberghe >>>>>> http://stfx.eu >>>>>> Smalltalk is the Red Pill >>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>> >>> > > > > > > -- > View this message in context: http://forum.world.st/VM-Crashes-tp4675526p4675784.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

March 8, 2013

[Pharo-project] VM Crashes

by Camillo Bruni

To join in again, my regression tests still seem to fail loading. So it looks like a similar effect as what sven experiences! https://ci.inria.fr/pharo/job/Pharo-2.0-Regression-Tests/VM=PharoVM,label=m…

March 8, 2013

Re: [Pharo-project] VM Crashes

by Paul DeBruicker

Does that happen when you use the VM from the top of the list here: http://pharo.gforge.inria.fr/ci/vm/pharo/linux/ specifically: http://pharo.gforge.inria.fr/ci/vm/pharo/linux/pharo-linux-stable.zip ? I haven't had the problem since switching to it when loading the config of Seaside 31 on Ubuntu 12.04 Sven Van Caekenberghe-2 wrote > Thanks. > > But my 2nd and 3rd runs crashed, both at > > Smalltalk stack dump: > 0xbf9c4c18 M MethodDictionary(Object)>becomeForward: 0x79299080: a(n) > MethodDictionary > 0xbf9c4c44 M MethodDictionary>grow 0x79299080: a(n) MethodDictionary > 0xbf9c4c5c M MethodDictionary(HashedCollection)>fullCheck 0x79299080: a(n) > MethodDictionary > 0xbf9c4c78 M MethodDictionary>at:put: 0x79299080: a(n) MethodDictionary > > so it is still becomeForward: but while growing a MethodDictionary, which > is a pretty basic operation I guess. > > On 08 Mar 2013, at 15:46, Esteban Lorenzano < > estebanlm@ > > wrote: > >> included :) >> >> On Mar 8, 2013, at 2:52 PM, Sven Van Caekenberghe < > sven@ > > wrote: >> >>> Hi Esteban, >>> >>> On 08 Mar 2013, at 12:44, Esteban Lorenzano < > estebanlm@ > > wrote: >>> >>>> Hi Sven, >>>> >>>> One of the problems we currently have is that all method installation >>>> execute a becomeForward:, which is of course not terrible cleaver... I >>>> tried using the squeak implementation of it and everything looks >>>> working (and it has a small optimization that can help). >>>> >>>> Can you trying the load by installing it? >>>> >>>> CompiledMethod>>#setSourcePointer: srcPointer >>>> "We can't change the trailer of existing method, since it could have a >>>> completely different format. Therefore we need to generate a copy >>>> with new trailer, containing a srcPointer, and then #become it." >>>> | trailer copy start | >>>> >>>> trailer := srcPointer = 0 >>>> ifTrue: [ >>>> "catch the common case of setting the source pointer to 0 when >>>> already 0" >>>> self sourcePointer = 0 ifTrue: [ ^ self ]. >>>> CompiledMethodTrailer empty ] >>>> ifFalse: [ >>>> CompiledMethodTrailer new sourcePointer: srcPointer ]. >>>> copy := self copyWithTrailerBytes: trailer. >>>> >>>> "ar 3/31/2010: Be a bit more clever since #become: is slow. >>>> If the old and the new trailer have the same size, just replace it." >>>> (self trailer class == trailer class and:[ self size = copy size ]) >>>> ifTrue: [ >>>> start := self endPC + 1. >>>> self replaceFrom: start to: self size with: copy startingAt: start ] >>>> ifFalse: [ >>>> self becomeForward: copy ]. >>>> >>>> ^self "will be copy if #become was needed" >>>> >>>> this can help in the speed and also can help on the vm crash problem >>>> (even if it is a workaround)... >>>> >>>> Esteban >>> >>> I used the following version >>> >>> CompiledMethod>>#setSourcePointer: srcPointer >>> "We can't change the trailer of existing method, since it could have >>> completely different format. >>> Therefore we need to generate a copy with new trailer, containing >>> scrPointer, and then become it." >>> >>> | trailer copy | >>> trailer := CompiledMethodTrailer new sourcePointer: srcPointer. >>> copy := self copyWithTrailerBytes: trailer. >>> "If possible do a replace in place as an optimization" >>> (self trailer class == trailer class and: [ self size = copy size ]) >>> ifTrue: [ >>> | start | >>> start := self endPC + 1. >>> self replaceFrom: start to: self size with: copy startingAt: start ] >>> ifFalse: [ self becomeForward: copy ]. >>> ^ self >>> >>> And it worked fine. My build completed successfully (1 try only). >>> But it took about as long as before (10 minutes using a Cog based VM). >>> >>> Thanks again for the suggestion, I think we should incorporate that >>> change, no ? >>> >>> The class test is ugly and most probably not necessary in the current >>> image, I would say. >>> >>> Sven >>> >>>> On Mar 8, 2013, at 11:53 AM, Sven Van Caekenberghe < > sven@ > > wrote: >>>> >>>>> >>>>> On 08 Mar 2013, at 11:16, Sven Van Caekenberghe < > sven@ > > wrote: >>>>> >>>>>> But I have said this before: the wall clock time of loading a lot of >>>>>> code is actually close to unacceptable - I don't think it is the >>>>>> download or the compilation, but more all the dynamic stuff that >>>>>> happens after that. There should be a way to not do all those updates >>>>>> for each method and move the updates to one big batch update after >>>>>> the load - if that is possible. >>>>> >>>>> To continue my rant (sorry ;-) about the problem with slow code >>>>> loading. >>>>> >>>>> These are some benchmarks on the same machine: >>>>> >>>>> $ ./vm.sh experimental.image eval '[Smalltalk allClassesAndTraits do: >>>>> #compileAll] timeToRun' >>>>> 106532 >>>>> >>>>> $ ./stack/vm.sh experimental.image eval '[Smalltalk >>>>> allClassesAndTraits do: #compileAll] timeToRun' >>>>> 221708 >>>>> >>>>> So it takes like 3 minutes to recompile every method in the system. >>>>> >>>>> How in the hell can it take 40 minutes to load some code (with all >>>>> packages already present in the package-cache (but then again the >>>>> package-cache is only 3.5 Mb, which could be downloaded in seconds)) ? >>>>> >>>>> Sven >>>>> >>>>> -- >>>>> Sven Van Caekenberghe >>>>> http://stfx.eu >>>>> Smalltalk is the Red Pill >>>>> >>>>> >>>> >>>> >>> >>> >> >> -- View this message in context: http://forum.world.st/VM-Crashes-tp4675526p4675784.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

March 8, 2013

Re: [Pharo-project] osprocess in pharo 2.0

by Sean P. DeNigris

On Mar 8, 2013, at 3:49 AM, Camillo Bruni-3 [via Smalltalk] <ml-node+s1294792n4675680h68(a)n4.nabble.com> wrote: > I wonder why it fails on the build-server? It fails because ThisOSProcess class>>shutDown: includes "OSProcess accessor breakDependents", but OSProcess accessor returns nil because none of the platform packages are loaded by the 'Tests' group of ConfigurationOfOSProcess. You can load 'Tests' and 'All OS' via "--groups='All OS',Tests" with the slice for "Issue 7662: [ENH]: ConfigurationCommandLineHandler: Accept multiple group names" Going forward, I'm not sure whether Tests should depend on the OS packages. You may want to load just the current OS, for example. I think probably a good error message and maybe a retry to find a suitable OS class if the accessor is nil... cc: David Lewis ----- Cheers, Sean -- View this message in context: http://forum.world.st/osprocess-in-pharo-2-0-tp4674453p4675783.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

March 8, 2013

Re: [Pharo-project] VM Crashes

by Paul DeBruicker

Camillo Bruni-3 wrote > If it's too old, what do you do? you go and fetch a newer VM ;). > So I would say the implied actions are ok, especially for newcomers, > but there should be more details indeed. Since Zinc is now included could the warning instead be a choice dialog that lets the user download either an appropriate VM for the image or an altogether more-up-to-date image+vm combo rather than just leaving the user to figure out how and where to find an appropriate VM? -- View this message in context: http://forum.world.st/VM-Crashes-tp4675526p4675781.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

March 8, 2013

Re: [Pharo-project] Porting SmaCC from 1.4 to 2.0; doSilently: equivalent ?

by Goubier Thierry

Le 08/03/2013 16:55, Esteban Lorenzano a Ã©crit : > yep :) Thanks, Thierry > On Mar 8, 2013, at 4:52 PM, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: > >> Hi, >> >> I'm trying to port SmaCC to 2.0 and I'm looking for the equivalent to SystemChangeNotifier>>doSilently:. Is it SystemAnnouncer>>suspendAllWhile:? >> >> (doSilently: is used in SmaCC tests). >> >> Thanks, >> >> Thierry >> -- >> Thierry Goubier >> CEA list >> Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s >> 91191 Gif sur Yvette Cedex >> France >> Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 >> > > > > -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

March 8, 2013

Re: [Pharo-project] Porting SmaCC from 1.4 to 2.0; doSilently: equivalent ?

by Mariano Martinez Peck

On Fri, Mar 8, 2013 at 12:52 PM, Goubier Thierry <thierry.goubier(a)cea.fr>wrote: > Hi, > > I'm trying to port SmaCC to 2.0 and I'm looking for the equivalent to > SystemChangeNotifier>>**doSilently:. Is it SystemAnnouncer>>** > suspendAllWhile:? > > (doSilently: is used in SmaCC tests). > > There is also a nice #valueWithoutNotifications in BlockClosure. BTW, does it make sense to port SmaCC? Which use case do you have in which you prefer SmaCC over say PetitParse? I took a look few weeks ago to something done in SmaCC and it was full of ugly unmaintainable class side tables..... > Thanks, > > Thierry > -- > Thierry Goubier > CEA list > Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s > 91191 Gif sur Yvette Cedex > France > Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95 > > -- Mariano http://marianopeck.wordpress.com

March 8, 2013