December 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Fuel - a fast object deployment tool

by Mariano Martinez Peck

On Wed, Dec 8, 2010 at 10:33 PM, Adrian Lienhard <adi(a)netstyle.ch> wrote: > Hi Martin, > > I took some application for which we use image segments to test Fuel > > - With Fuel serializing and writing to disk took 330s. File size is 16.1MB > - With image segments saving takes 4s and the file size is 2.4MB > but how are you using ImageSegment? just the primitive? because in order to compare it to Fuel, you should write all objects, including "outPointers". So you should use #writeForExportOn: or similar... funny it is only 4s in IS, since it has to do a #become, a full GC mark phase, etc... > > - When loading, I got a low space warning because the primitive > newMethod:header: failed. > - Loading with image segments takes 11 seconds > > I wonder what your rational is for "objects are much more times loaded than > stored". That's Fuel purpose. And it is useful for version systems, where you may commit once (a specfic version for example), but load hundred of times. > In our case its exactly the other way round. We store very often (like > every couple of minutes if there are changes), but only load when we restart > an image (this may be weeks). > > So I guess Fuel is not the best approach for you :) > HTH, > Adrian > > On Dec 8, 2010, at 17:50 , Martin Dias wrote: > > > Hi all > > > > Last months I and Tristan have been working on Fuel project, an object > > binary serialization tool. The idea is that objects are much more > > times loaded than stored, therefore it is worth to spend time while > > storing in order to have faster loading and user experience. We > > present an implementation of a pickle format that is based on > > clustering similar objects. > > > > There is a summary of the project below, but more complete information > > is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel > > > > The implementation still needs a lot of work to be really useful, > > optimizations should be done, but we'll be glad to get feedback of the > > community. > > > > > > = Pickle format = > > > > The pickle format and the serialization algorithm main idea, is > > explained in this slides: > > > > http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example > > > > > > = Current features = > > > > - Class shape changing (when a variable has been added, or removed, or > > its index changed) > > - Serialize most of the basic objects. > > - Serialize (almost) any CompiledMethod > > - Detection of global or class variables > > - Support for cyclic object graphs > > - Tests > > > > > > = Next steps = > > > > - Improve version checking. > > - Optimize performance. > > - Serialize more kinds of objects: > > -- Class with its complete description. > > -- Method contexts > > -- Active block closures > > -- Continuation > > - Some improvements for the user: > > -- pre and post actions to be executed. > > -- easily say 'this object is singleton'. > > - Partial loading of a stored graph. > > - Fast statistics/brief info extraction of a stored graph. > > - ConfigurationOfFuel. > > - Be able to deploy materialization behavior only (independent from > > the serialization behavior) > > > > > > = Download = > > > > In a Pharo 1.1 or 1.1.1 evaluate: > > > > Gofer new > > squeaksource: 'Fuel'; > > version: 'Fuel-MartinDias.74'; > > version: 'FuelBenchmarks-MartinDias.4'; > > load. > > > > > > = Benchmarks = > > > > You can run benchmarks executing this line (results in Transcript): > > > > FLBenchmarks newBasic run. > > > > > > Thank you! > > Martin Dias > > > > >

Dec. 8, 2010

Re: [Pharo-project] Finder issue?

by Francisco Ortiz PeÃ±aloza

Great! Didn't know that but what about AppRegistry? I didn't see much of it, don't know if its an old idea or have a bad implementation but would it be better to use some dynamic binding mechanism so given any other day someone could change the editor globally like *MorphicTextEditor default *works. hmm i think i'm missing something Thanks, Francisco On Wed, Dec 8, 2010 at 7:00 PM, Benjamin < benjamin.vanryseghem.pharo(a)gmail.com> wrote: > > On Dec 8, 2010, at 8:37 PM, Francisco Ortiz PeÃ±aloza wrote: > > Hi, > > after loading last Shout on PharoCore the new Finder isn't working > properly. Let's see > > PluggableShoutMorph>>initialize registers Shout in AppRegistry as the > default MorphicTextEditor but FinderUI>>buildSourceTextArea uses a > PluggableTextMorph. > > I think it should be using the more dinamic *MorphicTextEditor default*insted of the hardcoded PluggableTextMorph. Am i right? > > > In 1.2, it was panned that PluggableTextMorph and PluggableShoutMorph > should been merged. In fact, I've posted a fix to allow PluggableTextMorph > to handle text styling, this way Shout isn't needed anymore :) > > > Thank you for your review, > > > Ben > >

Dec. 8, 2010

Re: [Pharo-project] Fuel - a fast object deployment tool

by Adrian Lienhard

On Dec 8, 2010, at 22:33 , Levente Uzonyi wrote: > On Wed, 8 Dec 2010, Mariano Martinez Peck wrote: > > Well, I doubt if speed is really important, since you're loading everything at most once. And the current tools are really fast IMHO. The following numbers are from Squeak: > [ Compiler recompileAll ] timeToRun. 29083. > CompiledMethod allInstances size. 60701. > [ CompiledMethod allInstancesDo: [ :each | each getSource ] ] timeToRun. 1133. > So the compiler is compiling 2087 methods per second on average. You can load 53575 methods per second from a file on average. If it's zipped, than it may be a bit slower, say a factor of 2-3x slowdown. So you can still load and compile more than 1800 methods per second. I guess thats fast enough. > > Even if Fuel can be 10x faster, it doesn't really make a difference IMHO. It would be interesting to thoroughly profile MC to figure where it spends all its time (with large projects it gets very very slow, like several minutes to just show the merge diffs between two branches). Adrian

Dec. 8, 2010

Re: [Pharo-project] Fuel - a fast object deployment tool

by Adrian Lienhard

Hi Martin, I took some application for which we use image segments to test Fuel - With Fuel serializing and writing to disk took 330s. File size is 16.1MB - With image segments saving takes 4s and the file size is 2.4MB - When loading, I got a low space warning because the primitive newMethod:header: failed. - Loading with image segments takes 11 seconds I wonder what your rational is for "objects are much more times loaded than stored". In our case its exactly the other way round. We store very often (like every couple of minutes if there are changes), but only load when we restart an image (this may be weeks). HTH, Adrian On Dec 8, 2010, at 17:50 , Martin Dias wrote: > Hi all > > Last months I and Tristan have been working on Fuel project, an object > binary serialization tool. The idea is that objects are much more > times loaded than stored, therefore it is worth to spend time while > storing in order to have faster loading and user experience. We > present an implementation of a pickle format that is based on > clustering similar objects. > > There is a summary of the project below, but more complete information > is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel > > The implementation still needs a lot of work to be really useful, > optimizations should be done, but we'll be glad to get feedback of the > community. > > > = Pickle format = > > The pickle format and the serialization algorithm main idea, is > explained in this slides: > > http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example > > > = Current features = > > - Class shape changing (when a variable has been added, or removed, or > its index changed) > - Serialize most of the basic objects. > - Serialize (almost) any CompiledMethod > - Detection of global or class variables > - Support for cyclic object graphs > - Tests > > > = Next steps = > > - Improve version checking. > - Optimize performance. > - Serialize more kinds of objects: > -- Class with its complete description. > -- Method contexts > -- Active block closures > -- Continuation > - Some improvements for the user: > -- pre and post actions to be executed. > -- easily say 'this object is singleton'. > - Partial loading of a stored graph. > - Fast statistics/brief info extraction of a stored graph. > - ConfigurationOfFuel. > - Be able to deploy materialization behavior only (independent from > the serialization behavior) > > > = Download = > > In a Pharo 1.1 or 1.1.1 evaluate: > > Gofer new > squeaksource: 'Fuel'; > version: 'Fuel-MartinDias.74'; > version: 'FuelBenchmarks-MartinDias.4'; > load. > > > = Benchmarks = > > You can run benchmarks executing this line (results in Transcript): > > FLBenchmarks newBasic run. > > > Thank you! > Martin Dias >

Dec. 8, 2010

Re: [Pharo-project] Fuel - a fast object deployment tool

by Levente Uzonyi

On Wed, 8 Dec 2010, Mariano Martinez Peck wrote: > On Wed, Dec 8, 2010 at 9:05 PM, Alexandre Bergel <alexandre(a)bergel.eu>wrote: > >>>> It look cool! >>>> Is this a better ImageSegment? >>> >>> there is no VM code implied and you do not get all the problems of >> imageSegment. >>> You should ask is it better than SmartRefStreams. >>> Yes >>> Now the code has not been optimized >> >> Ah ok! How to deal with stubs is the difference with ImageSegment then. >> >> > The difference is that in Fuel there is no managment for "shared objects". > Fuel is not for swapping (there are no stubs/proxies and becomes), and the > idea is that you write on a file, ALL (not only those objects that are ONLY > accessible by the roots, like in ImageSegment) the objects reachable from > user defined roots. > > Fuel is similar to ReferenceStream and subclasses, a XML serializer, or any > serializers, etc...but MOSTLY, to Parcels. It is very similar to VisualWorks > parcel. > > Another important thing is that it is binary. The idea is that maybe in a > future we have a Monticello that doesn't need a mcz (code) + Compiler, but > instead, just load binary code (fuel files). This would be muuuch faster > that current Monticello, and even more, for minimal images, we wouldn't need > a Compiler. > Finally, Fuel is designed (like Parcels) to be very fast at loading time :) Well, I doubt if speed is really important, since you're loading everything at most once. And the current tools are really fast IMHO. The following numbers are from Squeak: [ Compiler recompileAll ] timeToRun. 29083. CompiledMethod allInstances size. 60701. [ CompiledMethod allInstancesDo: [ :each | each getSource ] ] timeToRun. 1133. So the compiler is compiling 2087 methods per second on average. You can load 53575 methods per second from a file on average. If it's zipped, than it may be a bit slower, say a factor of 2-3x slowdown. So you can still load and compile more than 1800 methods per second. I guess thats fast enough. Even if Fuel can be 10x faster, it doesn't really make a difference IMHO. Levente > > Cheers > > Mariano > > > > >> It looks exciting! >> >> Alexandre >> >> >>> >>>> >>>> Alexandre >>>> >>>> >>>> On 8 Dec 2010, at 13:50, Martin Dias wrote: >>>> >>>>> Hi all >>>>> >>>>> Last months I and Tristan have been working on Fuel project, an object >>>>> binary serialization tool. The idea is that objects are much more >>>>> times loaded than stored, therefore it is worth to spend time while >>>>> storing in order to have faster loading and user experience. We >>>>> present an implementation of a pickle format that is based on >>>>> clustering similar objects. >>>>> >>>>> There is a summary of the project below, but more complete information >>>>> is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel >>>>> >>>>> The implementation still needs a lot of work to be really useful, >>>>> optimizations should be done, but we'll be glad to get feedback of the >>>>> community. >>>>> >>>>> >>>>> = Pickle format = >>>>> >>>>> The pickle format and the serialization algorithm main idea, is >>>>> explained in this slides: >>>>> >>>>> http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example >>>>> >>>>> >>>>> = Current features = >>>>> >>>>> - Class shape changing (when a variable has been added, or removed, or >>>>> its index changed) >>>>> - Serialize most of the basic objects. >>>>> - Serialize (almost) any CompiledMethod >>>>> - Detection of global or class variables >>>>> - Support for cyclic object graphs >>>>> - Tests >>>>> >>>>> >>>>> = Next steps = >>>>> >>>>> - Improve version checking. >>>>> - Optimize performance. >>>>> - Serialize more kinds of objects: >>>>> -- Class with its complete description. >>>>> -- Method contexts >>>>> -- Active block closures >>>>> -- Continuation >>>>> - Some improvements for the user: >>>>> -- pre and post actions to be executed. >>>>> -- easily say 'this object is singleton'. >>>>> - Partial loading of a stored graph. >>>>> - Fast statistics/brief info extraction of a stored graph. >>>>> - ConfigurationOfFuel. >>>>> - Be able to deploy materialization behavior only (independent from >>>>> the serialization behavior) >>>>> >>>>> >>>>> = Download = >>>>> >>>>> In a Pharo 1.1 or 1.1.1 evaluate: >>>>> >>>>> Gofer new >>>>> squeaksource: 'Fuel'; >>>>> version: 'Fuel-MartinDias.74'; >>>>> version: 'FuelBenchmarks-MartinDias.4'; >>>>> load. >>>>> >>>>> >>>>> = Benchmarks = >>>>> >>>>> You can run benchmarks executing this line (results in Transcript): >>>>> >>>>> FLBenchmarks newBasic run. >>>>> >>>>> >>>>> Thank you! >>>>> Martin Dias >>>>> >>>> >>>> -- >>>> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >>>> Alexandre Bergel http://www.bergel.eu >>>> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >> >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> >> >> >> >> >

Dec. 8, 2010

[Pharo-project] Issue 3389 in pharo: should add SimpleMorphic in core

by pharo＠googlecode.com

Status: Fixed Owner: stephane.ducasse Labels: Milestone-1.2 New issue 3389 by stephane.ducasse: should add SimpleMorphic in core http://code.google.com/p/pharo/issues/detail?id=3389 so that it does not rot. Now I pubished it into PharoTaskForces

Dec. 8, 2010

Re: [Pharo-project] Finder issue?

by Benjamin

On Dec 8, 2010, at 8:37 PM, Francisco Ortiz PeÃ±aloza wrote: > Hi, > > after loading last Shout on PharoCore the new Finder isn't working > properly. Let's see > > PluggableShoutMorph>>initialize registers Shout in AppRegistry as > the default MorphicTextEditor but FinderUI>>buildSourceTextArea uses > a PluggableTextMorph. > > I think it should be using the more dinamic MorphicTextEditor > default insted of the hardcoded PluggableTextMorph. Am i right? In 1.2, it was panned that PluggableTextMorph and PluggableShoutMorph should been merged. In fact, I've posted a fix to allow PluggableTextMorph to handle text styling, this way Shout isn't needed anymore :) Thank you for your review, Ben

Dec. 8, 2010

[Pharo-project] Issue 3390 in pharo: should add SimpleMorphic in core

by pharo＠googlecode.com

Status: Fixed Owner: stephane.ducasse Labels: Milestone-1.2 New issue 3390 by stephane.ducasse: should add SimpleMorphic in core http://code.google.com/p/pharo/issues/detail?id=3390 so that it does not rot. Now I pubished it into PharoTaskForces

Dec. 8, 2010

[Pharo-project] ODBC, loadFFI and friends!

by Torsten Bergmann

Did you see http://www.pharocasts.com/2010/12/access-database-through-odbc.html Bye T. -- GRATIS! Movie-FLAT mit Ã¼ber 300 Videos. Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome

Dec. 8, 2010

Re: [Pharo-project] Fuel - a fast object deployment tool

by Mariano Martinez Peck

On Wed, Dec 8, 2010 at 9:05 PM, Alexandre Bergel <alexandre(a)bergel.eu>wrote: > >> It look cool! > >> Is this a better ImageSegment? > > > > there is no VM code implied and you do not get all the problems of > imageSegment. > > You should ask is it better than SmartRefStreams. > > Yes > > Now the code has not been optimized > > Ah ok! How to deal with stubs is the difference with ImageSegment then. > > The difference is that in Fuel there is no managment for "shared objects". Fuel is not for swapping (there are no stubs/proxies and becomes), and the idea is that you write on a file, ALL (not only those objects that are ONLY accessible by the roots, like in ImageSegment) the objects reachable from user defined roots. Fuel is similar to ReferenceStream and subclasses, a XML serializer, or any serializers, etc...but MOSTLY, to Parcels. It is very similar to VisualWorks parcel. Another important thing is that it is binary. The idea is that maybe in a future we have a Monticello that doesn't need a mcz (code) + Compiler, but instead, just load binary code (fuel files). This would be muuuch faster that current Monticello, and even more, for minimal images, we wouldn't need a Compiler. Finally, Fuel is designed (like Parcels) to be very fast at loading time :) Cheers Mariano > It looks exciting! > > Alexandre > > > > > >> > >> Alexandre > >> > >> > >> On 8 Dec 2010, at 13:50, Martin Dias wrote: > >> > >>> Hi all > >>> > >>> Last months I and Tristan have been working on Fuel project, an object > >>> binary serialization tool. The idea is that objects are much more > >>> times loaded than stored, therefore it is worth to spend time while > >>> storing in order to have faster loading and user experience. We > >>> present an implementation of a pickle format that is based on > >>> clustering similar objects. > >>> > >>> There is a summary of the project below, but more complete information > >>> is available here: http://rmod.lille.inria.fr/web/pier/software/Fuel > >>> > >>> The implementation still needs a lot of work to be really useful, > >>> optimizations should be done, but we'll be glad to get feedback of the > >>> community. > >>> > >>> > >>> = Pickle format = > >>> > >>> The pickle format and the serialization algorithm main idea, is > >>> explained in this slides: > >>> > >>> http://www.slideshare.net/tinchodias/fuel-serialization-in-an-example > >>> > >>> > >>> = Current features = > >>> > >>> - Class shape changing (when a variable has been added, or removed, or > >>> its index changed) > >>> - Serialize most of the basic objects. > >>> - Serialize (almost) any CompiledMethod > >>> - Detection of global or class variables > >>> - Support for cyclic object graphs > >>> - Tests > >>> > >>> > >>> = Next steps = > >>> > >>> - Improve version checking. > >>> - Optimize performance. > >>> - Serialize more kinds of objects: > >>> -- Class with its complete description. > >>> -- Method contexts > >>> -- Active block closures > >>> -- Continuation > >>> - Some improvements for the user: > >>> -- pre and post actions to be executed. > >>> -- easily say 'this object is singleton'. > >>> - Partial loading of a stored graph. > >>> - Fast statistics/brief info extraction of a stored graph. > >>> - ConfigurationOfFuel. > >>> - Be able to deploy materialization behavior only (independent from > >>> the serialization behavior) > >>> > >>> > >>> = Download = > >>> > >>> In a Pharo 1.1 or 1.1.1 evaluate: > >>> > >>> Gofer new > >>> squeaksource: 'Fuel'; > >>> version: 'Fuel-MartinDias.74'; > >>> version: 'FuelBenchmarks-MartinDias.4'; > >>> load. > >>> > >>> > >>> = Benchmarks = > >>> > >>> You can run benchmarks executing this line (results in Transcript): > >>> > >>> FLBenchmarks newBasic run. > >>> > >>> > >>> Thank you! > >>> Martin Dias > >>> > >> > >> -- > >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > >> Alexandre Bergel http://www.bergel.eu > >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > >> > >> > >> > >> > >> > >> > > > > > > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > > >

Dec. 8, 2010