June 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] pharo/linux/alien

by danil osipchuk

Argh, just noticed recent email from Eliot about new vm (I've been in a digest mode). printf examples work there(qsort prim fails but that's probably not a vm issue) 2011/6/20 danil osipchuk <danil.osipchuk(a)gmail.com> > Hello, all > > Is there a combination of Pharo image, linux vm and alien packages which is > known to be working? > > I've tried one-click 1.2.1 Pharo releases with following: > > Gofer it > squeaksource:'MetacelloRepository' ; > package: 'ConfigurationOfAlien'; > load. > > ConfigurationOfAlien loadCore. > ConfigurationOfAlien loadTests. > ConfigurationOfAlien loadLibC. > > > Both cog and plain-vm releases load the above fine. However the plain vm > doesn't have IA32ABI in >>Smalltalk listBuiltinModules (neither does have > external plugin) and cog vm segfaults on every example i tried. Any clues? > > regards, > Danil > > Segmentation fault Mon Jun 20 12:24:14 2011 > > > C stack backtrace: > ./Contents/Linux/squeakvm[0x805ad7a] > ./Contents/Linux/squeakvm[0x805b0ef] > [0xb77cd40c] > /lib/i686/cmov/libc.so.6(_IO_vfprintf+0x133)[0xb7638773] > /lib/i686/cmov/libc.so.6(vsprintf+0x8c)[0xb765a34c] > /lib/i686/cmov/libc.so.6(sprintf+0x2b)[0xb764255b] > ./Contents/Linux/squeakvm(callIA32IntegralReturn+0x2bd)[0x80fbdf9] > ./Contents/Linux/squeakvm[0x80fba1c] > ./Contents/Linux/squeakvm[0x806bcf5] > ./Contents/Linux/squeakvm[0x8086a84] > ./Contents/Linux/squeakvm(interpret+0x5ea5)[0x8063d15] > ./Contents/Linux/squeakvm[0x8072e1d] > ./Contents/Linux/squeakvm[0x8078c9d] > ./Contents/Linux/squeakvm(interpret+0x18)[0x805de88] > ./Contents/Linux/squeakvm(main+0x266)[0x805cf37] > /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb760f455] > ./Contents/Linux/squeakvm[0x8059fd1] > > > Smalltalk stack dump: > 0xbfb33bdc I LibCLibrary>sprintfWithArguments: 2027450576: a(n) LibCLibrary > 0xbfb33c0c I AlienExamples>exampleSprintf 2027449520: a(n) AlienExamples > 0xbfb33c24 M AlienExamples class>? 2022153212: a(n) AlienExamples class > .... cut > basicNew: > sizeFieldPut: > addressFieldPut: > primFFICallResult:withArguments: > Aborted > > >

June 20, 2011

Re: [Pharo-project] About not using full Pharo

by Damien Cassou

On Mon, Jun 20, 2011 at 6:20 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > My image was always built from PharoCore. It was done this way before > PharoDev existed, and I saw no reason to change. > > I thought people were supposed to build their own image (tailored to their > needs). IIRC, PharoDev was initially provided as a convenience for those > that wanted such an artifact. > > The concept of PharoDev evolved - I recall discussion about the "branding" > of PharoDev as Pharo, and relegating PharoCore to the role of build process > artifact. The discussion did not raise any flags for me, until now, when the > discussion has turned to "eliminating" PharoCore. > > I've never mentioned this before because I never thought I was doing > anything wrong by building my own image from PharoCore. You didn't do anything wrong :-). As Mariano pointed out, the process became so simple that everyone can now build his own images. I was asking because if a lot of you don't like one package or always add the same packages, we should know it to improve Pharo so that it suits most people. -- Damien Cassou http://damiencassou.seasidehosting.st "Lambdas are relegated to relative obscurity until Java makes them popular by not having them." James Iry

June 20, 2011

[Pharo-project] [update 1.3] #13269

by Marcus Denker

13269 ----- Issue 4425: Finder cant find #first: http://code.google.com/p/pharo/issues/detail?id=4425 Issue 4427: FlatMessageListBrowser add browse users http://code.google.com/p/pharo/issues/detail?id=4427 -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

June 20, 2011

[Pharo-project] pharo/linux/alien

by danil osipchuk

Hello, all Is there a combination of Pharo image, linux vm and alien packages which is known to be working? I've tried one-click 1.2.1 Pharo releases with following: Gofer it squeaksource:'MetacelloRepository' ; package: 'ConfigurationOfAlien'; load. ConfigurationOfAlien loadCore. ConfigurationOfAlien loadTests. ConfigurationOfAlien loadLibC. Both cog and plain-vm releases load the above fine. However the plain vm doesn't have IA32ABI in >>Smalltalk listBuiltinModules (neither does have external plugin) and cog vm segfaults on every example i tried. Any clues? regards, Danil Segmentation fault Mon Jun 20 12:24:14 2011 C stack backtrace: ./Contents/Linux/squeakvm[0x805ad7a] ./Contents/Linux/squeakvm[0x805b0ef] [0xb77cd40c] /lib/i686/cmov/libc.so.6(_IO_vfprintf+0x133)[0xb7638773] /lib/i686/cmov/libc.so.6(vsprintf+0x8c)[0xb765a34c] /lib/i686/cmov/libc.so.6(sprintf+0x2b)[0xb764255b] ./Contents/Linux/squeakvm(callIA32IntegralReturn+0x2bd)[0x80fbdf9] ./Contents/Linux/squeakvm[0x80fba1c] ./Contents/Linux/squeakvm[0x806bcf5] ./Contents/Linux/squeakvm[0x8086a84] ./Contents/Linux/squeakvm(interpret+0x5ea5)[0x8063d15] ./Contents/Linux/squeakvm[0x8072e1d] ./Contents/Linux/squeakvm[0x8078c9d] ./Contents/Linux/squeakvm(interpret+0x18)[0x805de88] ./Contents/Linux/squeakvm(main+0x266)[0x805cf37] /lib/i686/cmov/libc.so.6(__libc_start_main+0xe5)[0xb760f455] ./Contents/Linux/squeakvm[0x8059fd1] Smalltalk stack dump: 0xbfb33bdc I LibCLibrary>sprintfWithArguments: 2027450576: a(n) LibCLibrary 0xbfb33c0c I AlienExamples>exampleSprintf 2027449520: a(n) AlienExamples 0xbfb33c24 M AlienExamples class>? 2022153212: a(n) AlienExamples class .... cut basicNew: sizeFieldPut: addressFieldPut: primFFICallResult:withArguments: Aborted

June 20, 2011

Re: [Pharo-project] new Cog linux VM available

by Bernat Romagosa

Great, Eliot. Thanks! -- Bernat Romagosa

June 20, 2011

Re: [Pharo-project] Draft chapter on Settings Framework for PBE2 now online

by Alain Plantec

Hi Hernan, First you have to declare your own keyword. To do that, extend SettingTreeBuilder for your application by adding a #myAppSettings method. SettingTreeBuilder>>myAppSettings <settingPragmaProcessor> "Process a <myAppSettings> pragma" currentPragma methodClass theNonMetaClass perform: currentPragma selector with: self. Then you can use <myAppSettings> as you do with <systemsettings> for the settings declarations. In order to open the SettingBrowser, just indicate the keywords you want: (SettingBrowser browse: #('myAppSettings' 'systemsettings')) or (SettingBrowser browse: #('myAppSettings' )) I will update the settings chapter with that. Cheers Alain Le 20/06/2011 00:18, HernÃ¡n Morales Durand a Ã©crit : > Hi Oscar, > > I would like to build my own application settings in a separated > Settings browser, can I use<myAppSettings> like the<systemsettings> > pragma? any example code how to do it? > > Cheers, > > 2011/2/8 Oscar Nierstrasz<oscar.nierstrasz(a)gmail.com>: >> Hi Folks, >> >> There is now a draft chapter on the Settings Framework for "Pharo by Example 2" available for review: >> >> https://gforge.inria.fr/frs/download.php/28243/Settings.pdf >> >> http://pharobyexample.org/ >> >> Cheers, >> - on >> >> >>

June 20, 2011

Re: [Pharo-project] Fuel - a fast object deployment tool

by Martin Dias

I had some problems when I tried to write a package export tool with fuel. I wanted to store the classes without method extensions of other packages. Maybe with the objectToSerializeIn: idea I can write: Class>>objectToSerializeIn: anFLAnalyser ^(anFLAnalyser shouldAvoidForeignProtocol: self) ifFalse: [self] ifTrue: [self copyWithoutForeignProtocol] Cheers, Martin On Mon, Jun 20, 2011 at 1:48 AM, Martin Dias <tinchodias(a)gmail.com> wrote: > I think the substitution should be done during the graph trace. Following > with the example, if a proxy replaces an object, the proxy represents a > subgraph that is appended and so it should be traced. > > For that we should keep track of the substitutions. I'm not sure how > complex is that but is think it's not so difficult. > > Seems to be a great idea, we have to try it. I like that avoids writing > inst var names as strings. I have no idea if with *slots* implemented then > we will be able to return inst vars as first-class objects... but anyway > this looks like the a nice solution. > > So, we have this as a pending issue as well as the id virtualization. > Thanks for the ideas and the discussion! > > Martin > > > On Fri, Jun 17, 2011 at 7:09 PM, Nicolas Cellier < > nicolas.cellier.aka.nice(a)gmail.com> wrote: > >> 2011/6/17 Eliot Miranda <eliot.miranda(a)gmail.com>: >> > >> > >> > On Fri, Jun 17, 2011 at 2:39 PM, Nicolas Cellier >> > <nicolas.cellier.aka.nice(a)gmail.com> wrote: >> >> >> >> 2011/6/17 Eliot Miranda <eliot.miranda(a)gmail.com>: >> >> > >> >> > >> >> > On Fri, Jun 17, 2011 at 1:26 AM, Martin Dias <tinchodias(a)gmail.com> >> >> > wrote: >> >> >> >> >> >> Hi Eliot, >> >> >> I am very happy to read your mail. >> >> >> >> >> >> On Wed, Jun 15, 2011 at 3:29 PM, Eliot Miranda >> >> >> <eliot.miranda(a)gmail.com> >> >> >> wrote: >> >> >>> >> >> >>> Hi Martin & Mariano, >> >> >>> regarding filtering. Yesterday my colleague Yaron and I >> >> >>> successfully >> >> >>> finished our port of Fuel to Newspeak and are successfully using it >> to >> >> >>> save >> >> >>> and restore our data sets; thank you, its a cool framework. We had >> to >> >> >>> implement two extensions, the first of which the ability to save >> and >> >> >>> restore >> >> >>> Newspeak classes, which is complex because these are instantiated >> >> >>> classes >> >> >>> inside instantiated Newspeak modules, not static Smalltalk classes >> in >> >> >>> the >> >> >>> Smalltalk dictionary. The second extension is the ability to map >> >> >>> specific >> >> >>> objects to nil, to prune objects on the way out. I want to discuss >> >> >>> this >> >> >>> latter extension. >> >> >>> In our data set we have a set of references to objects that are >> >> >>> logically >> >> >>> not persistent and hence not to be saved. I'm sure that this will >> be >> >> >>> a >> >> >>> common case. The requirement is for the pickling system to prune >> >> >>> certain >> >> >>> objects, typically by arranging that when an object graph is >> pickled, >> >> >>> references to the pruned objects are replaced by references to nil. >> >> >>> One way >> >> >>> of doing this is as described below, by specifiying per-class lists >> of >> >> >>> instance variables whose referents shoudl not be saved. But this >> can >> >> >>> be >> >> >>> clumsy; there may be references to objects one wants to prune from >> >> >>> e.g. more >> >> >>> than one class, in which case one may have to provide multiple >> lists >> >> >>> of the >> >> >>> relevant inst vars; there may be references to objects one wants to >> >> >>> prune >> >> >>> from e.g. collections (e.g. sets and dictionaries) in which case >> the >> >> >>> instance variable list approach just doesn't work. >> >> >>> Here are two more general schemes. VFirst, most directly, Fuel >> could >> >> >>> provide two filters, implemented in the default mapper, or the core >> >> >>> analyser. One is a set of classes whose instances are not to be >> >> >>> saved. Any >> >> >>> reference to an instance of a class in the toBePrunedClasses set is >> >> >>> saved as >> >> >>> nil. The other is a set of instances that are not to be saved, and >> >> >>> also any >> >> >>> reference to an instance in the toBePruned set is saved as nil. >> Why >> >> >>> have >> >> >>> both? It can be convenient and efficient to filter by class (in >> our >> >> >>> case we >> >> >>> had many instances of a specific class, all of which should be >> >> >>> filtered, and >> >> >>> finding them could be time consuming), but filtering by class can >> be >> >> >>> too >> >> >>> inflexible, there may indeed be specific instances to exclude >> (thing >> >> >>> for >> >> >>> example of part of the object graph that functions as a cache; >> pruning >> >> >>> the >> >> >>> specific objects in the cache is the right thing to do; pruning all >> >> >>> instances of classes whose instances exist in the cache may prune >> too >> >> >>> much). >> >> >>> As an example here's how we implemented pruning. Our system is >> called >> >> >>> Glue, and we start with a mapper for Glue objects, FLGlueMapper: >> >> >>> FLMapper subclass: #FLGlueMapper >> >> >>> instanceVariableNames: 'prunedObjectClasses newspeakClassesCluster >> >> >>> modelClasses' >> >> >>> classVariableNames: '' >> >> >>> poolDictionaries: '' >> >> >>> category: 'Fuel-Core-Mappers' >> >> >>> It accepts newspeak objects and filters instances in the >> >> >>> prunedObjectsClasses set, and as a side-effect collects certain >> >> >>> classes that >> >> >>> we need in a manifest: >> >> >>> FLGlueMapper>>accepts: anObject >> >> >>> "Tells if the received object is handled by this analyzer. We want >> to >> >> >>> hand-off >> >> >>> instantiated Newspeak classes to the newspeakClassesCluster, and we >> >> >>> want >> >> >>> to record other model classes. We want to filter-out instances of >> any >> >> >>> class >> >> >>> in prunedObjectClasses." >> >> >>> ^anObject isBehavior >> >> >>> ifTrue: >> >> >>> [(self isInstantiatedNewspeakClass: anObject) >> >> >>> ifTrue: [true] >> >> >>> ifFalse: >> >> >>> [(anObject inheritsFrom: GlueDataObject) ifTrue: >> >> >>> [modelClasses add: anObject]. >> >> >>> false]] >> >> >>> ifFalse: >> >> >>> [prunedObjectClasses includes: anObject class] >> >> >>> It prunes by mapping instances of the prunedObjectClasses to a >> special >> >> >>> cluster. It can do this in visitObject: since any newspeak objects >> it >> >> >>> is >> >> >>> accepting will be visited in its visitClassOrTrait: method (i.e. >> it's >> >> >>> implicit that all arguments to visitObjects: are instances of the >> >> >>> prunedObjectsClasses set). >> >> >>> FLGlueMapper>>visitObject: anObject >> >> >>> analyzer >> >> >>> mapAndTrace: anObject >> >> >>> to: FLPrunedObjectsCluster instance >> >> >>> into: analyzer clustersWithBaselevelObjects >> >> >>> FLPrunedObjectsCluster is a specialization of the nil,true,false >> >> >>> cluster >> >> >>> that maps its objects to nil: >> >> >>> FLNilTrueFalseCluster subclass: #FLPrunedObjectsCluster >> >> >>> instanceVariableNames: '' >> >> >>> classVariableNames: '' >> >> >>> poolDictionaries: '' >> >> >>> category: 'Fuel-Core-Clusters' >> >> >>> FLPrunedObjectsCluster >>serialize: aPrunedObject on: aWriteStream >> >> >>> super serialize: nil on: aWriteStream >> >> >>> >> >> >>> So this would generalize by the analyser having an e.g. >> >> >>> FLPruningMapper >> >> >>> as the first mapper, and this having a prunedObjects and a >> >> >>> priunedObjectClasses set and going something like this: >> >> >>> FLPruningMapper>>accepts: anObject >> >> >>> ^(prunedObjects includes: anObject) or: [prunedObjectClasses >> includes: >> >> >>> anObject class] >> >> >>> FLPruningMapper >>visitObject: anObject >> >> >>> analyzer >> >> >>> mapAndTrace: anObject >> >> >>> to: FLPrunedObjectsCluster instance >> >> >>> into: analyzer clustersWithBaselevelObjects >> >> >>> and then one would provide accessors in FLSerialzer and/or >> FLAnalyser >> >> >>> to >> >> >>> add objects and classes to the prunedObjects >> and prunedObjectClasses >> >> >>> set. >> >> >>> For efficiency one could arrange that the FLPruningMapper was not >> >> >>> added >> >> >>> to the sequence of mappers unless and until objects or classes were >> >> >>> added >> >> >>> to the prunedObjects and prunedObjectClasses set. >> >> >> >> >> >> Excellent. I love the botanical metaphor of pruning! Of course we >> can >> >> >> include FLPruningMapper and FLPrunedObjectsCluster in Fuel. >> >> >> >> >> >> We are also interested in pruning objects but not necessarily >> replacing >> >> >> them by nil, but for another user defined objects. For example >> proxies. >> >> >> We >> >> >> can extend the pruning stuff for doing that. >> >> > >> >> > That was an idea Yaron came up with. That instead of >> >> > using fuelIgnoredInstanceVariableNames one uses e.g. >> >> > Object>>objectToSerialize >> >> > ^self >> >> > and then if one wants to prune specific inst vars in MyClass one >> >> > implements >> >> > MyClass>>objectToSerialize >> >> > ^self shallowCopy prepareForSerialization >> >> >> >> Hi Eliot, >> >> >> >> I'm not convinced by the shallowCopy solution, except for the simple >> >> structures. >> >> If object graph is complex (have share nodes, loops, ...) then you >> >> gonna end up in a replication problem equivalent to the one Fuel is >> >> trying to solve. >> > >> > The assumption is that the analyser would create a maximum of one proxy >> per >> > object in the graph (default, no proxy) and that it would map objects >> with >> > proxies to their proxies. So if proxies only nilled out inst vars I >> don't >> > see a problem. What's attractive about this is that it provides a >> general >> > solution to a couple of problems, a) how to replace a class of objects >> by >> > some substitute (e.g. nil), b) how to prune state that needn't be saved. >> It >> > is also conceptually simple; one just creates a proxy instance; no >> defining >> > metadata, such as inst var names, and hence the code is always >> up-to-date >> > (e.g. a class redefine won't automatically uncover renamed inst vars in >> > serialization metadata). >> >> Ah, OK, it occurs after the graph analysis, which I did not catch at first >> read. >> Now I understand better. >> >> Nicolas >> >> >> >> >> Nicolas >> >> >> >> > MyClass>>prepareForSerialization >> >> > instVarIDontWantToSerialize := nil. >> >> > ^self >> >> > and for objects one doesn't want to serlalize one implements >> >> > MyNotToBeSerializedClass>>objectToSerialize >> >> > ^nil >> >> > So its more general. But I would pass the analyser in as an >> argument, >> >> > which >> >> > would allow things like >> >> > MyPerhapsNotToBeSerializedClass>>objectToSerializeIn: anFLAnalyser >> >> > ^(anFLAnalyser shouldPrune: self) >> >> > ifFalse: [self] >> >> > ifTrue: [nil] >> >> > which would of course be the default in Object: >> >> > Object>>objectToSerializeIn: anFLAnalyser >> >> > ^(anFLAnalyser shouldPrune: self) ifFalse:: [self] >> >> > >> >> >> >> >> >> >> >> >>> >> >> >>> I think both Yaron and I feel the Fuel framework is comprehensible >> and >> >> >>> flexible. We enjoyed using it and while we took two passes at >> coming >> >> >>> up >> >> >>> with the pruning scheme we liked (our first was based on not >> >> >>> serializing >> >> >>> specific ins vars and was much more complex than our second, based >> on >> >> >>> pruning instances of specific classes) we got there quickly and >> will >> >> >>> very >> >> >>> little frustration along the way. Thank you very much. >> >> >> >> >> >> :-) thank you! >> >> >> >> >> >>> >> >> >>> Finally, a couple of things. First, it may be more flexible to >> >> >>> implement >> >> >>> fuelCluster as fuelClusterIn: anFLAnalyser so that if one is trying >> to >> >> >>> override certain parts of the mapping framework an implementation >> can >> >> >>> access >> >> >>> the analyser to find existing clusters, e.g. >> >> >>> MyClass>>fuelClusterIn: anFLAnalyser >> >> >>> ^self shouldBeInASpecialCluster >> >> >>> ifTrue: [anFLAnalyser clusterWithId: MySpecialCluster id] >> >> >>> ifFalse: [super fuelClusterIn: anFLAnalyser] >> >> >>> This makes it easier to find a specific unique cluster to handle a >> >> >>> group >> >> >>> of objects specially. >> >> >> >> >> >> I can't imagine a concrete example but I see that it is more >> >> >> flexible... >> >> >> the cluster obtained via double dispatch can be anything polymorphic >> >> >> with >> >> >> MySpecialCluster... that's the point? >> >> > >> >> > To be honest I'm not sure. But passing in the analyser in things >> like >> >> > fuelCluster or objectToSerialize is I think a good idea as it >> provides a >> >> > convenient communication path which in turn provides considerable >> >> > flexibility. >> >> >> >> >> >> >> >> >>> >> >> >>> Lastly, the class-side cluster ids are a bit of a pain. It would >> be >> >> >>> nice >> >> >>> to know a) are these byte values or general integer values, i.e. >> can >> >> >>> there >> >> >>> be more than 256 types of cluster?, and b) is there any meaning to >> the >> >> >>> ids? >> >> >>> For example, are clusters ordered by id, or is this just an >> integer >> >> >>> tag? >> >> >>> Also, some class-side code to assign an unused id would be nice. >> >> >>> You might think of virtualizing the id scheme. For example, if >> >> >>> FLCluster >> >> >>> maintained a weak array of all its subclasses then the id of a >> cluster >> >> >>> could >> >> >>> be the index in the array, and the array could be cleaned up >> >> >>> occasionally. >> >> >>> Then each fuel serialization could start with the list of cluster >> >> >>> class >> >> >>> names and ids, so that specific values of ids are specific to a >> >> >>> particular >> >> >>> serialization. >> >> >> >> >> >> I do agree, these ids are an heritage from the first prototypes of >> >> >> fuel, >> >> >> they should be revised. a) yes, it is encoded in only one byte; b) >> just >> >> >> an >> >> >> integer tag, the only purpose of the id was for decoding fast: read >> a >> >> >> byte >> >> >> and then look in a dictionary for the corresponding cluster >> instance. >> >> >> We >> >> >> could even store the cluster class name but that's inefficient. >> >> > >> >> > Yes, but how inefficient? What's the size of all the cluster names? >> >> > FLCluster allSubclasses inject: 0 into: [:t :c| t + c name size + >> 1] >> >> > 670 >> >> > >> >> > So you'd add less than a kilobyte to the size of each serialization >> and >> >> > get >> >> > complete freedom from ids. Something to think about. >> >> >> >> >> >> Virtualizing the id scheme is a good idea. Much more elegant and >> >> >> extensible. The current mechanism not only limits the number of >> >> >> possible >> >> >> clusters, but also "user defined" extensions can collide, for >> example >> >> >> if >> >> >> your Glue cluster id is the same of the Moose cluster id. >> >> >> >> >> >> I added an issue in our tracker. >> >> >> >> >> >> If it makes sense, maybe the weak array you suggest can be also used >> to >> >> >> avoid instantiating lots of FLObjectCluster like we are doing in >> >> >> Object: >> >> >> >> >> >> fuelCluster >> >> >> ^ self class isVariable >> >> >> ifTrue: [ FLVariableObjectCluster for: self class ] >> >> >> ifFalse: [ FLFixedObjectCluster for: self class ] >> >> >> >> >> >> the second time you send fuelCluster to an object, it can reuse the >> >> >> cluster instance. >> >> > >> >> > Right. I think that's important, and is one reason why I think >> passing >> >> > in >> >> > the analyser is important, because it allows certain objects to >> discover >> >> > existing clusters in the analyzer and join them if they want to, >> instead >> >> > of >> >> > having to invent and maintain their own cluster uniquing solution >> >> > . >> >> >>> >> >> >>> again thanks for a great framework. >> >> >> >> >> >> Thanks for your words and the feedback. Is Glue published somewhere? >> >> > >> >> > No, and its extremely proprietary :) Newspeak however is available >> and >> >> > we >> >> > may end up maintaining a port of Fuel for Newspeak. >> >> > best regards, >> >> > Eliot >> >> > >> >> >> >> >> >> regards >> >> >> Martin >> >> >> >> >> >> >> >> >>> >> >> >>> best, >> >> >>> Eliot >> >> >> >> >> >> >> >> >>> >> >> >>> On Mon, Jun 13, 2011 at 10:16 AM, Mariano Martinez Peck >> >> >>> <marianopeck(a)gmail.com> wrote: >> >> >>>> >> >> >>>> >> >> >>>> On Thu, Jun 9, 2011 at 3:35 AM, Eliot Miranda >> >> >>>> <eliot.miranda(a)gmail.com> >> >> >>>> wrote: >> >> >>>>> >> >> >>>>> Hi Martin and Mariano, >> >> >>>>> a couple of questions. What's the right way to exclude >> certain >> >> >>>>> objects from the serialization? Is there a way of excluding >> certain >> >> >>>>> inst >> >> >>>>> vars from certain objects? >> >> >>>> >> >> >>>> >> >> >>>> Eliot and the rest....Martin implemented this feature in >> >> >>>> Fuel-MartinDias.258. For the moment, we decided to put >> >> >>>> #fuelIgnoredInstanceVariableNames at class side. >> >> >>>> >> >> >>>> Behavior >> fuelIgnoredInstanceVariableNames >> >> >>>> "Indicates which variables have to be ignored during >> >> >>>> serialization." >> >> >>>> >> >> >>>> ^#() >> >> >>>> >> >> >>>> >> >> >>>> MyClass class >> fuelIgnoredInstanceVariableNames >> >> >>>> ^ #('instVar1') >> >> >>>> >> >> >>>> >> >> >>>> The impact in speed is nothing, so this is good. Now....we were >> >> >>>> thinking >> >> >>>> if it is common to need that 2 different instances of the same >> class >> >> >>>> need >> >> >>>> different instVars to ignore. Is this common ? do you usually need >> >> >>>> this ? >> >> >>>> We checked in SIXX and it is at instance side. Java uses the >> prefix >> >> >>>> 'transient' so it is at class side... >> >> >>>> >> >> >>>> thanks >> >> >>>> >> >> >>>> >> >> >>>> -- >> >> >>>> Mariano >> >> >>>> http://marianopeck.wordpress.com >> >> >>>> >> >> >>> >> >> >> >> >> > >> >> > >> > >> > >> > -- >> > best, >> > Eliot >> > >> >> >

June 20, 2011

Re: [Pharo-project] About not using full Pharo

by Lukas Renggli

> I've read that some of you are installing their own set of packages on > top of Pharo-Core and not using Pharo. Why is that so? What don't you > like in Pharo? Why didn't you tell us before? Uhh? Don't tell me you didn't know? All images I build on my Jenkins are based on PharoCore since we setup that infrastructure. I even unload packages from PharoCore, because I think it contains far to much cruft (https://github.com/renggli/builder/blob/master/scripts/pharo-cleanup.st) This is also something I repeatedly brought up. I don't use Pharo mostly because of OCompletion. It looks ugly and I get annoyed to have it constantly suggesting me stuff I didn't ask for. No other IDE I know has such an aggressive and annoying completion behavior. Also the Pharo images usually contain very old code, for example the image I just downloaded from pharo-project.org uses a version of OmniBrowser with the broken scrollbar. Something that I fixed in the beginning of this year, I think. Personally I don't need many of the packages loaded into Pharo: AutmaticMethodCategorizer (something similar is in OB-Standard for a long time), Autotest (something similar is in OB-SUnitIntegration for a long time), HelpSystem, MemoryMonitor, MorphicExtras, Sound, ArchiveViewer, SqueakCompatiblity, BitBltPen, MovieMorph, ObjectMetaTools, Nile, ProfStef, ScriptManager, XMLSupport (I load that only when needed), XMLWriter. > If we don't have this information, you might not like the new > Pharo-Core and you will have nothing anymore to base your images on. > Speak now or forever hold your peace :-) Since the beginning of 2011, 3757 of my images were downloaded from people other than me. Not sure who this is, why they do it, and if this number is any significant? Maybe it helps you for your statistics? Cheers, Lukas -- Lukas Renggli www.lukas-renggli.ch

June 20, 2011

Re: [Pharo-project] Fuel - a fast object deployment tool

by Martin Dias

I think the substitution should be done during the graph trace. Following with the example, if a proxy replaces an object, the proxy represents a subgraph that is appended and so it should be traced. For that we should keep track of the substitutions. I'm not sure how complex is that but is think it's not so difficult. Seems to be a great idea, we have to try it. I like that avoids writing inst var names as strings. I have no idea if with *slots* implemented then we will be able to return inst vars as first-class objects... but anyway this looks like the a nice solution. So, we have this as a pending issue as well as the id virtualization. Thanks for the ideas and the discussion! Martin On Fri, Jun 17, 2011 at 7:09 PM, Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com> wrote: > 2011/6/17 Eliot Miranda <eliot.miranda(a)gmail.com>: > > > > > > On Fri, Jun 17, 2011 at 2:39 PM, Nicolas Cellier > > <nicolas.cellier.aka.nice(a)gmail.com> wrote: > >> > >> 2011/6/17 Eliot Miranda <eliot.miranda(a)gmail.com>: > >> > > >> > > >> > On Fri, Jun 17, 2011 at 1:26 AM, Martin Dias <tinchodias(a)gmail.com> > >> > wrote: > >> >> > >> >> Hi Eliot, > >> >> I am very happy to read your mail. > >> >> > >> >> On Wed, Jun 15, 2011 at 3:29 PM, Eliot Miranda > >> >> <eliot.miranda(a)gmail.com> > >> >> wrote: > >> >>> > >> >>> Hi Martin & Mariano, > >> >>> regarding filtering. Yesterday my colleague Yaron and I > >> >>> successfully > >> >>> finished our port of Fuel to Newspeak and are successfully using it > to > >> >>> save > >> >>> and restore our data sets; thank you, its a cool framework. We had > to > >> >>> implement two extensions, the first of which the ability to save and > >> >>> restore > >> >>> Newspeak classes, which is complex because these are instantiated > >> >>> classes > >> >>> inside instantiated Newspeak modules, not static Smalltalk classes > in > >> >>> the > >> >>> Smalltalk dictionary. The second extension is the ability to map > >> >>> specific > >> >>> objects to nil, to prune objects on the way out. I want to discuss > >> >>> this > >> >>> latter extension. > >> >>> In our data set we have a set of references to objects that are > >> >>> logically > >> >>> not persistent and hence not to be saved. I'm sure that this will > be > >> >>> a > >> >>> common case. The requirement is for the pickling system to prune > >> >>> certain > >> >>> objects, typically by arranging that when an object graph is > pickled, > >> >>> references to the pruned objects are replaced by references to nil. > >> >>> One way > >> >>> of doing this is as described below, by specifiying per-class lists > of > >> >>> instance variables whose referents shoudl not be saved. But this > can > >> >>> be > >> >>> clumsy; there may be references to objects one wants to prune from > >> >>> e.g. more > >> >>> than one class, in which case one may have to provide multiple lists > >> >>> of the > >> >>> relevant inst vars; there may be references to objects one wants to > >> >>> prune > >> >>> from e.g. collections (e.g. sets and dictionaries) in which case the > >> >>> instance variable list approach just doesn't work. > >> >>> Here are two more general schemes. VFirst, most directly, Fuel > could > >> >>> provide two filters, implemented in the default mapper, or the core > >> >>> analyser. One is a set of classes whose instances are not to be > >> >>> saved. Any > >> >>> reference to an instance of a class in the toBePrunedClasses set is > >> >>> saved as > >> >>> nil. The other is a set of instances that are not to be saved, and > >> >>> also any > >> >>> reference to an instance in the toBePruned set is saved as nil. Why > >> >>> have > >> >>> both? It can be convenient and efficient to filter by class (in our > >> >>> case we > >> >>> had many instances of a specific class, all of which should be > >> >>> filtered, and > >> >>> finding them could be time consuming), but filtering by class can be > >> >>> too > >> >>> inflexible, there may indeed be specific instances to exclude (thing > >> >>> for > >> >>> example of part of the object graph that functions as a cache; > pruning > >> >>> the > >> >>> specific objects in the cache is the right thing to do; pruning all > >> >>> instances of classes whose instances exist in the cache may prune > too > >> >>> much). > >> >>> As an example here's how we implemented pruning. Our system is > called > >> >>> Glue, and we start with a mapper for Glue objects, FLGlueMapper: > >> >>> FLMapper subclass: #FLGlueMapper > >> >>> instanceVariableNames: 'prunedObjectClasses newspeakClassesCluster > >> >>> modelClasses' > >> >>> classVariableNames: '' > >> >>> poolDictionaries: '' > >> >>> category: 'Fuel-Core-Mappers' > >> >>> It accepts newspeak objects and filters instances in the > >> >>> prunedObjectsClasses set, and as a side-effect collects certain > >> >>> classes that > >> >>> we need in a manifest: > >> >>> FLGlueMapper>>accepts: anObject > >> >>> "Tells if the received object is handled by this analyzer. We want > to > >> >>> hand-off > >> >>> instantiated Newspeak classes to the newspeakClassesCluster, and we > >> >>> want > >> >>> to record other model classes. We want to filter-out instances of > any > >> >>> class > >> >>> in prunedObjectClasses." > >> >>> ^anObject isBehavior > >> >>> ifTrue: > >> >>> [(self isInstantiatedNewspeakClass: anObject) > >> >>> ifTrue: [true] > >> >>> ifFalse: > >> >>> [(anObject inheritsFrom: GlueDataObject) ifTrue: > >> >>> [modelClasses add: anObject]. > >> >>> false]] > >> >>> ifFalse: > >> >>> [prunedObjectClasses includes: anObject class] > >> >>> It prunes by mapping instances of the prunedObjectClasses to a > special > >> >>> cluster. It can do this in visitObject: since any newspeak objects > it > >> >>> is > >> >>> accepting will be visited in its visitClassOrTrait: method (i.e. > it's > >> >>> implicit that all arguments to visitObjects: are instances of the > >> >>> prunedObjectsClasses set). > >> >>> FLGlueMapper>>visitObject: anObject > >> >>> analyzer > >> >>> mapAndTrace: anObject > >> >>> to: FLPrunedObjectsCluster instance > >> >>> into: analyzer clustersWithBaselevelObjects > >> >>> FLPrunedObjectsCluster is a specialization of the nil,true,false > >> >>> cluster > >> >>> that maps its objects to nil: > >> >>> FLNilTrueFalseCluster subclass: #FLPrunedObjectsCluster > >> >>> instanceVariableNames: '' > >> >>> classVariableNames: '' > >> >>> poolDictionaries: '' > >> >>> category: 'Fuel-Core-Clusters' > >> >>> FLPrunedObjectsCluster >>serialize: aPrunedObject on: aWriteStream > >> >>> super serialize: nil on: aWriteStream > >> >>> > >> >>> So this would generalize by the analyser having an e.g. > >> >>> FLPruningMapper > >> >>> as the first mapper, and this having a prunedObjects and a > >> >>> priunedObjectClasses set and going something like this: > >> >>> FLPruningMapper>>accepts: anObject > >> >>> ^(prunedObjects includes: anObject) or: [prunedObjectClasses > includes: > >> >>> anObject class] > >> >>> FLPruningMapper >>visitObject: anObject > >> >>> analyzer > >> >>> mapAndTrace: anObject > >> >>> to: FLPrunedObjectsCluster instance > >> >>> into: analyzer clustersWithBaselevelObjects > >> >>> and then one would provide accessors in FLSerialzer and/or > FLAnalyser > >> >>> to > >> >>> add objects and classes to the prunedObjects and prunedObjectClasses > >> >>> set. > >> >>> For efficiency one could arrange that the FLPruningMapper was not > >> >>> added > >> >>> to the sequence of mappers unless and until objects or classes were > >> >>> added > >> >>> to the prunedObjects and prunedObjectClasses set. > >> >> > >> >> Excellent. I love the botanical metaphor of pruning! Of course we can > >> >> include FLPruningMapper and FLPrunedObjectsCluster in Fuel. > >> >> > >> >> We are also interested in pruning objects but not necessarily > replacing > >> >> them by nil, but for another user defined objects. For example > proxies. > >> >> We > >> >> can extend the pruning stuff for doing that. > >> > > >> > That was an idea Yaron came up with. That instead of > >> > using fuelIgnoredInstanceVariableNames one uses e.g. > >> > Object>>objectToSerialize > >> > ^self > >> > and then if one wants to prune specific inst vars in MyClass one > >> > implements > >> > MyClass>>objectToSerialize > >> > ^self shallowCopy prepareForSerialization > >> > >> Hi Eliot, > >> > >> I'm not convinced by the shallowCopy solution, except for the simple > >> structures. > >> If object graph is complex (have share nodes, loops, ...) then you > >> gonna end up in a replication problem equivalent to the one Fuel is > >> trying to solve. > > > > The assumption is that the analyser would create a maximum of one proxy > per > > object in the graph (default, no proxy) and that it would map objects > with > > proxies to their proxies. So if proxies only nilled out inst vars I > don't > > see a problem. What's attractive about this is that it provides a > general > > solution to a couple of problems, a) how to replace a class of objects by > > some substitute (e.g. nil), b) how to prune state that needn't be saved. > It > > is also conceptually simple; one just creates a proxy instance; no > defining > > metadata, such as inst var names, and hence the code is always up-to-date > > (e.g. a class redefine won't automatically uncover renamed inst vars in > > serialization metadata). > > Ah, OK, it occurs after the graph analysis, which I did not catch at first > read. > Now I understand better. > > Nicolas > > >> > >> Nicolas > >> > >> > MyClass>>prepareForSerialization > >> > instVarIDontWantToSerialize := nil. > >> > ^self > >> > and for objects one doesn't want to serlalize one implements > >> > MyNotToBeSerializedClass>>objectToSerialize > >> > ^nil > >> > So its more general. But I would pass the analyser in as an argument, > >> > which > >> > would allow things like > >> > MyPerhapsNotToBeSerializedClass>>objectToSerializeIn: anFLAnalyser > >> > ^(anFLAnalyser shouldPrune: self) > >> > ifFalse: [self] > >> > ifTrue: [nil] > >> > which would of course be the default in Object: > >> > Object>>objectToSerializeIn: anFLAnalyser > >> > ^(anFLAnalyser shouldPrune: self) ifFalse:: [self] > >> > > >> >> > >> >> > >> >>> > >> >>> I think both Yaron and I feel the Fuel framework is comprehensible > and > >> >>> flexible. We enjoyed using it and while we took two passes at > coming > >> >>> up > >> >>> with the pruning scheme we liked (our first was based on not > >> >>> serializing > >> >>> specific ins vars and was much more complex than our second, based > on > >> >>> pruning instances of specific classes) we got there quickly and will > >> >>> very > >> >>> little frustration along the way. Thank you very much. > >> >> > >> >> :-) thank you! > >> >> > >> >>> > >> >>> Finally, a couple of things. First, it may be more flexible to > >> >>> implement > >> >>> fuelCluster as fuelClusterIn: anFLAnalyser so that if one is trying > to > >> >>> override certain parts of the mapping framework an implementation > can > >> >>> access > >> >>> the analyser to find existing clusters, e.g. > >> >>> MyClass>>fuelClusterIn: anFLAnalyser > >> >>> ^self shouldBeInASpecialCluster > >> >>> ifTrue: [anFLAnalyser clusterWithId: MySpecialCluster id] > >> >>> ifFalse: [super fuelClusterIn: anFLAnalyser] > >> >>> This makes it easier to find a specific unique cluster to handle a > >> >>> group > >> >>> of objects specially. > >> >> > >> >> I can't imagine a concrete example but I see that it is more > >> >> flexible... > >> >> the cluster obtained via double dispatch can be anything polymorphic > >> >> with > >> >> MySpecialCluster... that's the point? > >> > > >> > To be honest I'm not sure. But passing in the analyser in things like > >> > fuelCluster or objectToSerialize is I think a good idea as it provides > a > >> > convenient communication path which in turn provides considerable > >> > flexibility. > >> >> > >> >> > >> >>> > >> >>> Lastly, the class-side cluster ids are a bit of a pain. It would be > >> >>> nice > >> >>> to know a) are these byte values or general integer values, i.e. can > >> >>> there > >> >>> be more than 256 types of cluster?, and b) is there any meaning to > the > >> >>> ids? > >> >>> For example, are clusters ordered by id, or is this just an integer > >> >>> tag? > >> >>> Also, some class-side code to assign an unused id would be nice. > >> >>> You might think of virtualizing the id scheme. For example, if > >> >>> FLCluster > >> >>> maintained a weak array of all its subclasses then the id of a > cluster > >> >>> could > >> >>> be the index in the array, and the array could be cleaned up > >> >>> occasionally. > >> >>> Then each fuel serialization could start with the list of cluster > >> >>> class > >> >>> names and ids, so that specific values of ids are specific to a > >> >>> particular > >> >>> serialization. > >> >> > >> >> I do agree, these ids are an heritage from the first prototypes of > >> >> fuel, > >> >> they should be revised. a) yes, it is encoded in only one byte; b) > just > >> >> an > >> >> integer tag, the only purpose of the id was for decoding fast: read a > >> >> byte > >> >> and then look in a dictionary for the corresponding cluster instance. > >> >> We > >> >> could even store the cluster class name but that's inefficient. > >> > > >> > Yes, but how inefficient? What's the size of all the cluster names? > >> > FLCluster allSubclasses inject: 0 into: [:t :c| t + c name size + > 1] > >> > 670 > >> > > >> > So you'd add less than a kilobyte to the size of each serialization > and > >> > get > >> > complete freedom from ids. Something to think about. > >> >> > >> >> Virtualizing the id scheme is a good idea. Much more elegant and > >> >> extensible. The current mechanism not only limits the number of > >> >> possible > >> >> clusters, but also "user defined" extensions can collide, for example > >> >> if > >> >> your Glue cluster id is the same of the Moose cluster id. > >> >> > >> >> I added an issue in our tracker. > >> >> > >> >> If it makes sense, maybe the weak array you suggest can be also used > to > >> >> avoid instantiating lots of FLObjectCluster like we are doing in > >> >> Object: > >> >> > >> >> fuelCluster > >> >> ^ self class isVariable > >> >> ifTrue: [ FLVariableObjectCluster for: self class ] > >> >> ifFalse: [ FLFixedObjectCluster for: self class ] > >> >> > >> >> the second time you send fuelCluster to an object, it can reuse the > >> >> cluster instance. > >> > > >> > Right. I think that's important, and is one reason why I think > passing > >> > in > >> > the analyser is important, because it allows certain objects to > discover > >> > existing clusters in the analyzer and join them if they want to, > instead > >> > of > >> > having to invent and maintain their own cluster uniquing solution > >> > . > >> >>> > >> >>> again thanks for a great framework. > >> >> > >> >> Thanks for your words and the feedback. Is Glue published somewhere? > >> > > >> > No, and its extremely proprietary :) Newspeak however is available > and > >> > we > >> > may end up maintaining a port of Fuel for Newspeak. > >> > best regards, > >> > Eliot > >> > > >> >> > >> >> regards > >> >> Martin > >> >> > >> >> > >> >>> > >> >>> best, > >> >>> Eliot > >> >> > >> >> > >> >>> > >> >>> On Mon, Jun 13, 2011 at 10:16 AM, Mariano Martinez Peck > >> >>> <marianopeck(a)gmail.com> wrote: > >> >>>> > >> >>>> > >> >>>> On Thu, Jun 9, 2011 at 3:35 AM, Eliot Miranda > >> >>>> <eliot.miranda(a)gmail.com> > >> >>>> wrote: > >> >>>>> > >> >>>>> Hi Martin and Mariano, > >> >>>>> a couple of questions. What's the right way to exclude > certain > >> >>>>> objects from the serialization? Is there a way of excluding > certain > >> >>>>> inst > >> >>>>> vars from certain objects? > >> >>>> > >> >>>> > >> >>>> Eliot and the rest....Martin implemented this feature in > >> >>>> Fuel-MartinDias.258. For the moment, we decided to put > >> >>>> #fuelIgnoredInstanceVariableNames at class side. > >> >>>> > >> >>>> Behavior >> fuelIgnoredInstanceVariableNames > >> >>>> "Indicates which variables have to be ignored during > >> >>>> serialization." > >> >>>> > >> >>>> ^#() > >> >>>> > >> >>>> > >> >>>> MyClass class >> fuelIgnoredInstanceVariableNames > >> >>>> ^ #('instVar1') > >> >>>> > >> >>>> > >> >>>> The impact in speed is nothing, so this is good. Now....we were > >> >>>> thinking > >> >>>> if it is common to need that 2 different instances of the same > class > >> >>>> need > >> >>>> different instVars to ignore. Is this common ? do you usually need > >> >>>> this ? > >> >>>> We checked in SIXX and it is at instance side. Java uses the prefix > >> >>>> 'transient' so it is at class side... > >> >>>> > >> >>>> thanks > >> >>>> > >> >>>> > >> >>>> -- > >> >>>> Mariano > >> >>>> http://marianopeck.wordpress.com > >> >>>> > >> >>> > >> >> > >> > > >> > > > > > > > -- > > best, > > Eliot > > > >

June 20, 2011

Re: [Pharo-project] About not using full Pharo

by Yanni Chiu

On 19/06/11 4:38 PM, Damien Cassou wrote: > Hi, > > I've read that some of you are installing their own set of packages on > top of Pharo-Core and not using Pharo. Why is that so? What don't you > like in Pharo? Why didn't you tell us before? My image was always built from PharoCore. It was done this way before PharoDev existed, and I saw no reason to change. I thought people were supposed to build their own image (tailored to their needs). IIRC, PharoDev was initially provided as a convenience for those that wanted such an artifact. The concept of PharoDev evolved - I recall discussion about the "branding" of PharoDev as Pharo, and relegating PharoCore to the role of build process artifact. The discussion did not raise any flags for me, until now, when the discussion has turned to "eliminating" PharoCore. I've never mentioned this before because I never thought I was doing anything wrong by building my own image from PharoCore.

June 20, 2011