January 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] [ANN] ClaseUseDiscovery tool

by StÃ©phane Ducasse

thanks mariano On Jan 11, 2010, at 9:59 AM, Mariano Martinez Peck wrote: > Hi folks: As part of my PhD I started to work with ImageSegment. This week I clean up all the stuff related to class discovery from there. I then wanted to put that extracted code in a separate class. I didn't like that code at all, so at the end I finished rewriting completely this stuff. > > The cool thing is that: > > - I remove all that stuff from ImageSegment (which has nothing to do with that) > - I remove that stuff from classes like Behavior and ClassDescription > - I put all these as an external package that can be perfectly loaded in core > - I rewrote that in style that I like much more. At least for me, it is much more easy to understand and to use now > - I wrote test cases > - I wrote good class comment. > > The repository is: http://www.squeaksource.com/ClaseUseDiscovery > and the license is MIT. > > It is a little project, but maybe someone find it useful. This project only works with Pharo images newer than 1.1 version 11142 > > I paste here the class comment of CUDClassesUseDiscoverer: > > Cheers > > Mariano > > > > > > A CUDClassesUseDiscoverer is a little class which can be used to discovering used and unused classes during a period of time. If you want to build minimal images, it is useful also to detect for example, the amount of classes that are used just to do a simple message invokating. It helps also when some classes are being used and you don't know why. > > Little examples of how to use it: > > | aDiscoverer | > > aDiscoverer := CUDClassesUseDiscoverer startDiscovery. > ......DO SOMETHING HERE...... > aDiscoverer stopDiscovery. > > After that, you can: > > aDiscoverer activeClassesDuringDiscovery. > aDiscoverer inactiveClassesDuringDiscovery. > aDiscoverer activeClassesByCategoryDuringDiscovery. > aDiscoverer swapOutInactiveClassesDuringDiscovery. > > For more details, see the tests in class CUDClassesUseDiscovererTest. > > Explanation: > > The Virtual Machine pays attention not to crash totally when you put a nil in a Method Category. In addition, there is a possibility to force a MethodDictionary fault (message induceMDFault) with the message. This method copies the methodDict in the organization slot. Actually, it creates an Array with the methodDict and the organization and set it in the organization slot. Finally, it sets a nil to the methodDict instance variable. After this is done, when that class is used and the methodDict is asked (it is very important to acces the methodDict using the accessor) it checks whether it is nil or not. If it is nil, it will search for that methodDict in the Array of the organizaction it will then copy it to the methodDict instance variable and will log in Smalltalk at: #MDFaultDict. Sometimes you will see that you have classes that were used but you do not know why or who used it. To solve this question, we can have that log. But in order to have that trace, you have to use the message startDiscoveryWithTrace instead of startDiscovery. The log will be then put in that dictionary and you can inspect it after the discovery. > > Taking that explanation in mind, you can follow these steps: > > 1) Send the message CUDClassesUseDiscoverer startDiscovery or startDiscoveryWithTrace which forces a MethodDictionary fault in all classes. Yes! It sends the message induceMDFault to all classes except classes like Array, Object, Class, Message and MethodDictionary. After this, all classes but those ones, will have the methodDict with nil. > 2) Do a few things, use the system for a while. Maybe even save and resume the image. > > 3) Stop the discovery. This will stop everything and restore all remaining MethodDictionary faults. It will also process which classes were active and which ones were not. > > 4) Now, it is easy. All classes that still have a nil in the mehodDict is because they were not used, and those who have something different from nil is because they were. With the message activeClassesDuringDiscovery or inactiveClassesDuringDiscovery you can and return the active or unactive classes respectively. There is also the message swapOutInactiveClassesDuringDiscovery which makes up segments by grouping unused classes by system category and write them to disk. There are some categories that are excluded because they have problems, like System and Kernel. > > > If you started the discovery with the message startDiscoveryWithTrace, after doing all those steps you can then inspect Smalltalk at: #MDFaultDict and you will see that in the dictionary you will have an element per class. That element is a ByteString that contains all the stack trace. This shows you all the messages and why that class was used. > > This approach of swapping out inactive classes have two big problems: > > - The granularity is too high: the class. As soon as only one method is invoked, the whole class is recovered. > - It is very common to recover all classes doing a simple action. For example, searching for implementors or senders of a certain method. Because of this, when you are developing in your Smalltalk image is very likely that all classes are recovered in less than 5 minutes of work. It is not useful. Browsing or editing code seems not to be a problem, thus. Maybe this approach has more sense in production images rather than development. > > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Jan. 11, 2010

Re: [Pharo-project] Some notes we took during our experience

by StÃ©phane Ducasse

Yes > > Would you rather WordArrayForSegment to be in the package 'System-Object Storage' where it is ImageSegment right now than being in Collections-Arrayed ? > > > > I will try to look for this. I have several things in my ImageSegment to-do list :) > > - FilePackage -> Tools > - TextDiffBuilder -> in its own package. > > Stef > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Jan. 11, 2010

Re: [Pharo-project] [update 1.1] #11151

by StÃ©phane Ducasse

Excellent! Thanks a lot. On Jan 11, 2010, at 9:49 AM, Alain Plantec wrote: > yep! :) > Now, the preferences to settings migration is over for the core. > It means that the core can leave without Preferences (but not pharo-dev > so far). > The old preferences browser can still be opened > (<world>/Tools/More.../PreferencesBrowser). > The problem we had with updates 11149->11150 shows us that the core can > also leave without settings :) > Now, I fix comments and write documentation so that people can define > settings for pharo-dev. > > Alain > > Marcus Denker a Ã©crit : >> 11150 >> ----- >> >> Issue 1593 : Packages: strange MC packages in need of cleanup (again) >> >> MC lost lots of classes... so this loads all affected packages (the settings ones). >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> >> > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Jan. 11, 2010

Re: [Pharo-project] [BetaTesting] [ANN] Pharo1.0-10505-rc1dev10.01.1

by Stan Shepherd

Mariano Martinez Peck wrote: > > On Sun, Jan 10, 2010 at 9:02 PM, Stan Shepherd > <stan.shepherd414(a)gmail.com>wrote: > >> The only concern is that it took some hours > > > hours ? seriously ? > > >> to download and install Seaside >> (albeit on a slow spare PC). And with occassional network glitches, I had >> to >> restart a method a few times from a debugger. > Yes, it was 3 hr 29m on the old machine, but I don't know what percent it was halted due to network glitches. It would be really cool if it made say 3 retries when its HTTP gets failed. I ran overnight on main machine (Ubuntu 9.04, 3 GHz dual core 1GB ram, nominal 7Gb internet but other downloads running) with message tally - 71 minutes. ...Stan -- View this message in context: http://n2.nabble.com/BetaTesting-ANN-Pharo1-0-10505-rc1dev10-01-1-tp4278312… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Jan. 11, 2010

Re: [Pharo-project] [ANN] ClaseUseDiscovery tool

by Fernando olivero

Good Work Mariano! It's always a pleasure when Smalltalker's remove complexity from core classes. And wish you good luck with your phd work. Fernando On Jan 11, 2010, at 9:59 AM, Mariano Martinez Peck wrote: > Hi folks: As part of my PhD I started to work with ImageSegment. This week I clean up all the stuff related to class discovery from there. I then wanted to put that extracted code in a separate class. I didn't like that code at all, so at the end I finished rewriting completely this stuff. > > The cool thing is that: > > - I remove all that stuff from ImageSegment (which has nothing to do with that) > - I remove that stuff from classes like Behavior and ClassDescription > - I put all these as an external package that can be perfectly loaded in core > - I rewrote that in style that I like much more. At least for me, it is much more easy to understand and to use now > - I wrote test cases > - I wrote good class comment. > > The repository is: http://www.squeaksource.com/ClaseUseDiscovery > and the license is MIT. > > It is a little project, but maybe someone find it useful. This project only works with Pharo images newer than 1.1 version 11142 > > I paste here the class comment of CUDClassesUseDiscoverer: > > Cheers > > Mariano > > > > > > A CUDClassesUseDiscoverer is a little class which can be used to discovering used and unused classes during a period of time. If you want to build minimal images, it is useful also to detect for example, the amount of classes that are used just to do a simple message invokating. It helps also when some classes are being used and you don't know why. > > Little examples of how to use it: > > | aDiscoverer | > > aDiscoverer := CUDClassesUseDiscoverer startDiscovery. > ......DO SOMETHING HERE...... > aDiscoverer stopDiscovery. > > After that, you can: > > aDiscoverer activeClassesDuringDiscovery. > aDiscoverer inactiveClassesDuringDiscovery. > aDiscoverer activeClassesByCategoryDuringDiscovery. > aDiscoverer swapOutInactiveClassesDuringDiscovery. > > For more details, see the tests in class CUDClassesUseDiscovererTest. > > Explanation: > > The Virtual Machine pays attention not to crash totally when you put a nil in a Method Category. In addition, there is a possibility to force a MethodDictionary fault (message induceMDFault) with the message. This method copies the methodDict in the organization slot. Actually, it creates an Array with the methodDict and the organization and set it in the organization slot. Finally, it sets a nil to the methodDict instance variable. After this is done, when that class is used and the methodDict is asked (it is very important to acces the methodDict using the accessor) it checks whether it is nil or not. If it is nil, it will search for that methodDict in the Array of the organizaction it will then copy it to the methodDict instance variable and will log in Smalltalk at: #MDFaultDict. Sometimes you will see that you have classes that were used but you do not know why or who used it. To solve this question, we can have that log. But in order to have that trace, you have to use the message startDiscoveryWithTrace instead of startDiscovery. The log will be then put in that dictionary and you can inspect it after the discovery. > > Taking that explanation in mind, you can follow these steps: > > 1) Send the message CUDClassesUseDiscoverer startDiscovery or startDiscoveryWithTrace which forces a MethodDictionary fault in all classes. Yes! It sends the message induceMDFault to all classes except classes like Array, Object, Class, Message and MethodDictionary. After this, all classes but those ones, will have the methodDict with nil. > 2) Do a few things, use the system for a while. Maybe even save and resume the image. > > 3) Stop the discovery. This will stop everything and restore all remaining MethodDictionary faults. It will also process which classes were active and which ones were not. > > 4) Now, it is easy. All classes that still have a nil in the mehodDict is because they were not used, and those who have something different from nil is because they were. With the message activeClassesDuringDiscovery or inactiveClassesDuringDiscovery you can and return the active or unactive classes respectively. There is also the message swapOutInactiveClassesDuringDiscovery which makes up segments by grouping unused classes by system category and write them to disk. There are some categories that are excluded because they have problems, like System and Kernel. > > > If you started the discovery with the message startDiscoveryWithTrace, after doing all those steps you can then inspect Smalltalk at: #MDFaultDict and you will see that in the dictionary you will have an element per class. That element is a ByteString that contains all the stack trace. This shows you all the messages and why that class was used. > > This approach of swapping out inactive classes have two big problems: > > - The granularity is too high: the class. As soon as only one method is invoked, the whole class is recovered. > - It is very common to recover all classes doing a simple action. For example, searching for implementors or senders of a certain method. Because of this, when you are developing in your Smalltalk image is very likely that all classes are recovered in less than 5 minutes of work. It is not useful. Browsing or editing code seems not to be a problem, thus. Maybe this approach has more sense in production images rather than development. > > > > > > <ATT00001..txt>

Jan. 11, 2010

[Pharo-project] Alien & (using Gofer and Flair)

by John M McIntosh

Whoever is looking after the Alien Gofer & Flair loader needs to revisit the logic since Flair fights and loses against the current Gofer. Which then prevents Alien from loading as per the instructions supplied on: http://www.squeaksource.com/Alien.html -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

Jan. 11, 2010

Re: [Pharo-project] Some notes we took during our experience

by Mariano Martinez Peck

On Thu, Jan 7, 2010 at 4:21 PM, Mariano Martinez Peck <marianopeck(a)gmail.com > wrote: > > > On Thu, Jan 7, 2010 at 4:02 PM, StÃ©phane Ducasse < > stephane.ducasse(a)inria.fr> wrote: > >> Hi >> >> marcus jannik and me got fun and fixed some cycles. >> >> Here are some extra notes I took >> >> - PreferenceBrowserMorph and friends should be removed. >> - MethodReference could be closer to CompiledMethod (to avoid cycle >> with tools when in fact people >> just wanted to report aCompiled method list and return a >> methodReference list). >> - Services -> split into Services-Morphic and Services-Core -> but >> services should be removed for now >> >> - Symbol could has its own scanning function (isLiteralSymbol) to >> avoid to depend on Scanner >> - URL should be replaced by URI >> - WordArrayForSegment -> ImageSegment >> > Would you rather WordArrayForSegment to be in the package 'System-Object Storage' where it is ImageSegment right now than being in Collections-Arrayed ? > > I will try to look for this. I have several things in my ImageSegment to-do > list :) > > >> - FilePackage -> Tools >> - TextDiffBuilder -> in its own package. >> >> Stef >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >> > >

Jan. 11, 2010

[Pharo-project] [ANN] ClaseUseDiscovery tool

by Mariano Martinez Peck

Hi folks: As part of my PhD I started to work with ImageSegment. This week I clean up all the stuff related to class discovery from there. I then wanted to put that extracted code in a separate class. I didn't like that code at all, so at the end I finished rewriting completely this stuff. The cool thing is that: - I remove all that stuff from ImageSegment (which has nothing to do with that) - I remove that stuff from classes like Behavior and ClassDescription - I put all these as an external package that can be perfectly loaded in core - I rewrote that in style that I like much more. At least for me, it is much more easy to understand and to use now - I wrote test cases - I wrote good class comment. The repository is: http://www.squeaksource.com/ClaseUseDiscovery and the license is MIT. It is a little project, but maybe someone find it useful. This project only works with Pharo images newer than 1.1 version 11142 I paste here the class comment of CUDClassesUseDiscoverer: Cheers Mariano A CUDClassesUseDiscoverer is a little class which can be used to discovering used and unused classes during a period of time. If you want to build minimal images, it is useful also to detect for example, the amount of classes that are used just to do a simple message invokating. It helps also when some classes are being used and you don't know why. Little examples of how to use it: | aDiscoverer | aDiscoverer := CUDClassesUseDiscoverer startDiscovery. ......DO SOMETHING HERE...... aDiscoverer stopDiscovery. After that, you can: aDiscoverer activeClassesDuringDiscovery. aDiscoverer inactiveClassesDuringDiscovery. aDiscoverer activeClassesByCategoryDuringDiscovery. aDiscoverer swapOutInactiveClassesDuringDiscovery. For more details, see the tests in class CUDClassesUseDiscovererTest. Explanation: The Virtual Machine pays attention not to crash totally when you put a nil in a Method Category. In addition, there is a possibility to force a MethodDictionary fault (message induceMDFault) with the message. This method copies the methodDict in the organization slot. Actually, it creates an Array with the methodDict and the organization and set it in the organization slot. Finally, it sets a nil to the methodDict instance variable. After this is done, when that class is used and the methodDict is asked (it is very important to acces the methodDict using the accessor) it checks whether it is nil or not. If it is nil, it will search for that methodDict in the Array of the organizaction it will then copy it to the methodDict instance variable and will log in Smalltalk at: #MDFaultDict. Sometimes you will see that you have classes that were used but you do not know why or who used it. To solve this question, we can have that log. But in order to have that trace, you have to use the message startDiscoveryWithTrace instead of startDiscovery. The log will be then put in that dictionary and you can inspect it after the discovery. Taking that explanation in mind, you can follow these steps: 1) Send the message CUDClassesUseDiscoverer startDiscovery or startDiscoveryWithTrace which forces a MethodDictionary fault in all classes. Yes! It sends the message induceMDFault to all classes except classes like Array, Object, Class, Message and MethodDictionary. After this, all classes but those ones, will have the methodDict with nil. 2) Do a few things, use the system for a while. Maybe even save and resume the image. 3) Stop the discovery. This will stop everything and restore all remaining MethodDictionary faults. It will also process which classes were active and which ones were not. 4) Now, it is easy. All classes that still have a nil in the mehodDict is because they were not used, and those who have something different from nil is because they were. With the message activeClassesDuringDiscovery or inactiveClassesDuringDiscovery you can and return the active or unactive classes respectively. There is also the message swapOutInactiveClassesDuringDiscovery which makes up segments by grouping unused classes by system category and write them to disk. There are some categories that are excluded because they have problems, like System and Kernel. If you started the discovery with the message startDiscoveryWithTrace, after doing all those steps you can then inspect Smalltalk at: #MDFaultDict and you will see that in the dictionary you will have an element per class. That element is a ByteString that contains all the stack trace. This shows you all the messages and why that class was used. This approach of swapping out inactive classes have two big problems: - The granularity is too high: the class. As soon as only one method is invoked, the whole class is recovered. - It is very common to recover all classes doing a simple action. For example, searching for implementors or senders of a certain method. Because of this, when you are developing in your Smalltalk image is very likely that all classes are recovered in less than 5 minutes of work. It is not useful. Browsing or editing code seems not to be a problem, thus. Maybe this approach has more sense in production images rather than development.

Jan. 11, 2010

Re: [Pharo-project] [update 1.1] #11151

by Alain Plantec

yep! :) Now, the preferences to settings migration is over for the core. It means that the core can leave without Preferences (but not pharo-dev so far). The old preferences browser can still be opened (<world>/Tools/More.../PreferencesBrowser). The problem we had with updates 11149->11150 shows us that the core can also leave without settings :) Now, I fix comments and write documentation so that people can define settings for pharo-dev. Alain Marcus Denker a Ã©crit : > 11150 > ----- > > Issue 1593 : Packages: strange MC packages in need of cleanup (again) > > MC lost lots of classes... so this loads all affected packages (the settings ones). > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > >

Jan. 11, 2010

[Pharo-project] [update 1.1] #11151

by Marcus Denker

11150 ----- Issue 1593 : Packages: strange MC packages in need of cleanup (again) MC lost lots of classes... so this loads all affected packages (the settings ones).

Jan. 11, 2010