October 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] When will Fuel file format stabilize?

by Igor Stasenko

On 11 October 2011 22:52, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > > On Tue, Oct 11, 2011 at 9:10 AM, StÃ©phane Ducasse > <stephane.ducasse(a)inria.fr> wrote: >> >> Mariano >> >> I think that evolution and migration is key if we want to use fuel for >> loading code. >> This is why in MCZ you have always the source files. In case of trouble. > > Exactly! and even more, you NEED the code if you are managing code ;)Â so > you need it always, in case of problem or not. > >> >> It is important to use the current situation as case for evolution. >> I like the idea of igor of having class that represent different versions. >> > > > We will think about it. > > Yes, think. :) The lesson that i learn by staring at VMMaker, events handling in image and other places: never ever spread bit-twiddling logic over multiple places. If you cannot avoid bit-twiddling, make sure to put it into a single place, so rest of the code could be kept clean of it. -- Best regards, Igor Stasenko.

Oct. 12, 2011

Re: [Pharo-project] [squeak-dev] Re: Migrating Complex in a separate package

by Nicolas Cellier

Thanks all for your answers. Too many words for a single post, so I compiled a few comments and Complex implementation reflections in this blog entry: http://smallissimo.blogspot.com/2011/10/about-complex-in-squeak.html I'm not satisfied with current status because we are mixing two paradigms. Note that putting Complex outside the image is also an easy solution for playing with paradigms without causing undesired side effects to Complex-unaware-Applications. But maybe it's too easy to divert like this, and we should better think harder ;) Nicolas 2011/10/11 Ricardo Moran <richi.moran(a)gmail.com>: > > On Tue, Oct 11, 2011 at 9:51 AM, Juan Vuletich <juan(a)jvuletich.org> wrote: >> >> Juan Vuletich wrote: >>> >>> Hi Folks, >>> >>> Rodney Polkinghorne wrote: >>>> >>>> ... >>>> >>>> extra fact: >>>> - Complex is absent from Cuis >>>> >>> >>> Not for long :) . I have already integrated it and will be in next Cuis >>> release. I think Complex belongs in the base system. >> >> Oh, forgot to say. I also think that >> Â -1 sqrt = 1i >> should be true, and I'm working on it (together with some other tweaks to >> #sqrt and friends I'll publish when finished). > > +1. I was just about to comment on that one :) > It always bothered me that "-1 sqrt" wouldn't answer "1i". It seems like an > incomplete implementation, being that we have the class Complex. > At the time I "fixed" it by just relying on the fact that the sqrt primitive > fails when the receiver is negative, but that solution seems very hackish. I > wonder what your solution looks like? :) > Cheers, > Richo > >> >> Thoughts? >> >> Cheers, >> Juan Vuletich >> > > > > >

Oct. 11, 2011

Re: [Pharo-project] [squeak-dev] Re: Migrating Complex in a separate package

by Chris Muller

>>> - the name of the class can be Complex or ComplexNumber >>> Specifically I don't like isComplex, many objects could respond true >>> because complicated; >>> isComplexNumber is much more explicit. >>> We could also think of having complex expressions in a symbolic >>> algebra, and isComplexNumber would be true only for a literal value.. >>> >>> What I could eventually do is publish an old Complex in package >>> Complex for backard compatibility and an updated ComplexNumber in a >>> Math-Complex package... >>> >>> How many of you use Complex ? >>> What do you think of these proposals ? >>> > > I do not have a strong position between Complex and ComplexNumber. I think > Complex is not that bad. But I think we should be careful about namespace consumption. "Complex" is very broad and ambiguous and could potentially create headaches for other domains that want to define a class named Complex.

Oct. 11, 2011

Re: [Pharo-project] [squeak-dev] Re: CogDroid Status Update

by Mariano Martinez Peck

Congratulations! continue with your work :) On Mon, Oct 10, 2011 at 8:57 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > wrote: > Thanks a lot Dimitry. What you are doing is great! > > Stef > On Oct 10, 2011, at 7:06 PM, Dimitry Golubovsky wrote: > > > Hi, > > > > I would like to update the Squeak/Pharo community on the CogDroid > > development status, summarize the changes made since the first public > > alpha release, and discuss further directions of the development. > > > > I have uploaded two new alpha release packages: > > > > for ARM devices: > > http://squeakvm-tablet.googlecode.com/files/CogDroid-alpha-20111009.apk > > > > for Android-x86: > > > http://squeakvm-tablet.googlecode.com/files/CogDroid86-alpha-20111010.apk > > > > I have created a wiki page [1] which summarizes the changes made to > > CogDroid since the first public alpha release. > > > > Briefly, these changes are: > > > > - Partial integration with Android intent system, so CogDroid can be > > launched with a specified Smalltalk image file and a startup script by > > tapping a home screen shortcut icon. Such icons can be created from > > Smalltalk code running in CogDroid. See also [2] for technical > > details. > > > > - Added build for the x86 Android platform. Android-x86 can run on > > Intel-compatible CPUs in desktops and netbooks, and also in virtual > > machines. > > > > - Using a PC with physical keyboard and mouse attached brings some > > benefits in terms of user input, so functionality was added to support > > special keys such as Ctrl- and Shift- modifiers, and also arrow and > > home/end keys. This may be also of use on ARM tablets with keyboard > > and mouse attached, or on dockable phones. I am unable to test this > > myself, but your feedback is always welcome. > > > > At this moment, I am going to change the direction of my efforts > > towards deeper studying of Android itself, which may give me some > > ideas how to integrate CogDroid better with Android services (e. g. > > sound interface is an open question, but I need to study more, whether > > OSS/ALSA is available under the hood, or some Java wrapper is needed). > > > > This means that for some time I will be pushing updates less > > frequently, except for bugs and critical issues, if reported. OTOH > > this is a better time to integrate my developer's branch with the > > mainline. So, Igor and Eliot please contact me to discuss the > > technical moments. > > > > I would like to thank the whole Squeak/Pharo community for helping me > > with the development and testing. The Benchmark page [3] is a source > > of information people may find helpful choosing an Android device to > > run CogDroid on. > > > > Your feedback is always welcome. Please add your comments to the Alpha > > Progress page [1], including your ideas of Android applications that > > might be created with CogDroid, and what functionality you think needs > > to be added. > > > > Thanks again, and hope to hear from you! > > > > ------------------- > > [1] http://code.google.com/p/squeakvm-tablet/wiki/CogDroidAlphaProgress > > [2] > http://code.google.com/p/squeakvm-tablet/wiki/CogDroidIntentsAndShortcuts > > [3] http://code.google.com/p/squeakvm-tablet/wiki/TinyBenchmark > > > > -- > > Dimitry Golubovsky > > > > Anywhere on the Web > > > > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Mariano Martinez Peck

On Tue, Oct 11, 2011 at 9:10 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr > wrote: > Mariano > > I think that evolution and migration is key if we want to use fuel for > loading code. > This is why in MCZ you have always the source files. In case of trouble. > Exactly! and even more, you NEED the code if you are managing code ;) so you need it always, in case of problem or not. > > It is important to use the current situation as case for evolution. > I like the idea of igor of having class that represent different versions. > > We will think about it. > Stef > > On Oct 11, 2011, at 8:55 AM, Philippe Marschall wrote: > > > On 10/08/2011 10:42 PM, Mariano Martinez Peck wrote: > >> s > >> > >>>> > >>>> This is IMHO more than necessary for Fuel to become a production ready > >>>> serializer and I'd say Fuel is now "old enough" to become such :) > >>> > >>> Yes. > >>> Now what I would love is that even if fuel changes that the evolution > of > >>> information > >>> is taken into account because like that it will be exercised for real. > >>> > >>> > >> No, that's impossible, and if posible, it is not worth it. Migrating > from an > >> old format to a new one is extremelly complicated and innecessary. The > >> easiest way to solve this is to take the correct version of Fuel, > >> materialize the graph from the stream, load new version of Fuel, and > >> seriaize it again. That the easiest, more secure, and more practical > >> approach I can see. > > > > That is horribly naÃ¯ve an excludes fuel from a lot of use cases. You > > can't use fuel for "archiving" objects outside of the image because you > > will never know whether you will be able to read them in again because > > the format changes. You will always need to have "live" ones in the > image. > > > > That means you can't use fuel for anything Monticello related because > > you may never be able to load those versions in again because the file > > format has changed in the mean time. > > > > Cheers > > Philippe > > > > > > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Mariano Martinez Peck

On Tue, Oct 11, 2011 at 8:55 AM, Philippe Marschall < philippe.marschall(a)netcetera.ch> wrote: > On 10/08/2011 10:42 PM, Mariano Martinez Peck wrote: > > s > > > >>> > >>> This is IMHO more than necessary for Fuel to become a production ready > >>> serializer and I'd say Fuel is now "old enough" to become such :) > >> > >> Yes. > >> Now what I would love is that even if fuel changes that the evolution of > >> information > >> is taken into account because like that it will be exercised for real. > >> > >> > > No, that's impossible, and if posible, it is not worth it. Migrating from > an > > old format to a new one is extremelly complicated and innecessary. The > > easiest way to solve this is to take the correct version of Fuel, > > materialize the graph from the stream, load new version of Fuel, and > > seriaize it again. That the easiest, more secure, and more practical > > approach I can see. > > That is horribly naÃ¯ve an excludes fuel from a lot of use cases. You > can't use fuel for "archiving" objects outside of the image because you > will never know whether you will be able to read them in again because > the format changes. You will always need to have "live" ones in the image. > > No. That's incorrect. You won't be able to do that ONLY if you update Fuel to a new image that breaks format. You can still continue to use the same version and you will never have that problem. So, again, why you need to update Fuel? Why SmartRefStream does not have this problem? because it hasn't changed in the last 10 years. So..do the same, take an specific Fuel version and keep it for 10 years. Just update it to make it work in Pharo without changing the format and you are done. > That means you can't use fuel for anything Monticello related because > you may never be able to load those versions in again because the file > format has changed in the mean time. > I guess that in the end, if someone can really do something for Monticello on top of Fuel it will be like 2 years from now, and at some point Fuel format will be stable. And as Stef says...you always have the code there in case of problems. > Cheers > Philippe > > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Mariano Martinez Peck

On Tue, Oct 11, 2011 at 5:51 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 11 October 2011 00:57, Stefan Marr <pharo(a)stefan-marr.de> wrote: > > > > On 10 Oct 2011, at 23:25, Igor Stasenko wrote: > > > >> On 11 October 2011 00:19, Schwab,Wilhelm K <bschwab(a)anest.ufl.edu> > wrote: > >>> True, but what about reading data from five years ago? I have ten year > old Dolphin STB that I can still load. > >>> > >>> > >> to my experience a 10 years old data has very small chances to be > relevant :) > > > > You might want to reconsider that: > > Some countries have laws, that regulate how long and what has to be > archived: > > > > Switzerland: > > http://www.itandlaw.ch/html/publikationen/pdf/KMUMagazin04.pdf > > Germany: > > http://www.recht-im-internet.de/themen/archivierung.htm > > > > 10 years is considered the minimum. > > > > Then how about following idea: help implementing format migration code :) > > I think what can be done in Fuel is to have a separate class per each > binary format version, > so when you do format changes, instead of modifying existing class you > creating a new one > and put new stuff there. > Then at a point of detecting incoming data, it could decide which > class to use for deserialization. > Yes, but the problem is that in Fuel we have a flatten design where we group the encoding/decoding of objects in "Cluster" classes. So, the format is not only defined by FLSerializer and FLMaterializer but also with its 20 cluster classes. So it is not that easy... > > > But that might not be what Pharo or Fuel is supposed to be used for. > > > > -- > > Stefan Marr > > Software Languages Lab > > Vrije Universiteit Brussel > > Pleinlaan 2 / B-1050 Brussels / Belgium > > http://soft.vub.ac.be/~smarr > > Phone: +32 2 629 2974 > > Fax: +32 2 629 3525 > > > > > > > > > > -- > Best regards, > Igor Stasenko. > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Mariano Martinez Peck

On Tue, Oct 11, 2011 at 2:33 AM, Yanni Chiu <yanni(a)rogers.com> wrote: > On 10/10/11 2:43 PM, Mariano Martinez Peck wrote: > >> >> On Mon, Oct 10, 2011 at 6:50 PM, Yanni Chiu <yanni(a)rogers.com >> >> For the case where a lot of serialized files need to be converted, >> having two versions in memory is almost a necessity; >> >> >> heheheheh how people could live with ReferenceStream for 10 years? >> >> otherwise, the Fuel code update would have to be applied for each >> file being converted. >> >> >> but answer my question: why would you need to update Fuel so frequently ? >> > > I don't need to update Fuel frequently. I'm currently "stuck" at some > pre-1.6 version of Fuel (some combination of package versions that I managed > to get working at the time, using early Fuel releases). You are an early adopter!!! cool :) > I want to migrate to a newer version at some point (to use some new Fuel > features), but the hold up is that I would have to migrate about a dozen > serialization files - probably manually, since a dozen is roughly at my > threshold to automate. However, I expect that the number of files to migrate > would be a larger number in the future. At that point I'd automate the: > deserialize with old Fuel, update to newer Fuel and serialize again. Exactly. That's what I meant. From my point of view it is easier to provide such scripts rather than to find a complex automagic conversion between versions. Imagine something like: FuelMigrator migrateFromFuelVersion: '1.6' to: '1.7 streams: aListOfStreams basically it can use gofer/metacello to load the version of fuel, iterate over each file and serialize/materialize. > The migration procedure seems safer to me, if it did not load code in the > middle of the process. There's lots of workarounds for this issue, so it's > not a high priority. > > > A separate issue is that at some point I wanted to serialize >> something differently, in two different contexts. In one case, >> serializing a Pier kernel, I want Fuel to just do its thing. In the >> other case, when exporting a portion a Pier kernel, I want to fine >> tune (i.e. map to different objects) the objects being >> exported/imported. There is a hook to allow the object swap, but the >> swap would then take place for the other case of serializing to >> whole kernel (where the desired behaviour was for no such mapping). >> >> >> That's an interesting point....Let me see if I understood. Let's say >> that you want to exclude certain instance variables of a class. What you >> mean is that you want to apply this hook (or another one) in an specific >> serialization and not always? because right now with Fuel (and SIXX, and >> I guess StOMP also) since you put the hooks in methods of the class, >> they are always taken into account. How would the API for the user >> should be? you can you specify for an specific serialization run that >> certain hook of certain object/class must be applied and some should not ? >> > > Yes, you've got the idea. No idea what the API should be for such > functionality - probably a serialization context (passed in at the start) > that answers configuration parameters to use to pick an object mapping > method. Interesting. I will open an issue for this. > > > Apart from that, do you have a real example where you need such behavior >> ? Beause it looks quite complex/rare scenario ;) >> > > My serialization code is not top of mind, but I think the issue I had was > that when exporting a portion of a Pier kernel, for importing into another > Pier kernel, the security objects should be nil'ed out. However, when using > Fuel to serialize the kernel, the security objects must be serialized > instead of being nil'ed out. Well...that can be simple workarrounded with a simple if depending on what you want. So depending what you do, you can serialize or not the security objects. This can be done without needing what you say. It may not be the best solution, but a workarround. > Not sure why I don't currently have a bug - probably, the security is being > inherited from the parent structures when they're nil. I'll see the problem > once I try to set different security on lower down structures - they'll be > lost on restart. > Ok. Check: http://rmod.lille.inria.fr/web/pier/software/Fuel/Hooks http://forum.world.st/Any-serializer-can-customize-serialization-strategy-t… Cheers > > -- > Yanni > > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] [squeak-dev] Any serializer can customize serialization strategy?

by Mariano Martinez Peck

On Tue, Oct 11, 2011 at 12:56 AM, HernÃ¡n Morales Durand < hernan.morales(a)gmail.com> wrote: > Thanks Mariano, > > That's pretty much what I want, but... why to expose the visitor interface? > ;) > Good question. Honesty, it was the only way we have found to do it with Fuel. I think (but I am not sure) that one of the drawbacks of the pickle format is that it analysis phase (as we call it in Fuel) make some hooks more complicated to implement than a classic serializer. > Just to give a little feedback of what I have in my mind: > > mySerializer setEngine: ExternalCompressor for: ( MyClass -> #myStringIVar > ) > > Well, you can also create an specific Fuel cluster for MyClass that does exactly that. Basically you have to create FLMyClassCluster, implement methods like #serializeWith: and #materializeFrom: and there you serialize/materialize the specific #myStringIVar with a special way and the rest as normal. Then you subclass the FLDefaultMapper, say YourMapper and you implement #visitXXX Then you have to implement MyClass >> visitXXX. Of course, you should be able to generalize such cluster and be able to reuse it for different classes or instVars. I am not saying this is the best way. I am just telling you one possible solution you can do by extending Fuel for your needs. Of course, you may need to understand how Fuel works first. But of course, we can help if you want. > or > > mySerializer setEngine: ExternalCompressor for: incomingStream > > Now with machines generating daily 6 TBytes of raw data, one of the > keys will be data compression. > it looks like ;) > 2011/10/10 Mariano Martinez Peck <marianopeck(a)gmail.com>: > > > > > > On Mon, Oct 10, 2011 at 9:35 PM, HernÃ¡n Morales Durand > > <hernan.morales(a)gmail.com> wrote: > >> > >> Is there any serializer (Fuel/StOMP/Ma object > >> serialization/BOSS/SRP/SIXX??) which let me attach an external > >> compressor/decompressor for specific Strings to the > >> serialization/deserialization process? > > > > In Fuel you can do: > > > > String >> fuelSubstitution > > ^ ExternalCompressor compress: self > > > > String >> fuelAccept: aVisitor > > ^ (ExternalCompressor shouldCompressString: self) > > ifTrue: [aVisitor visitSubstitution: self] > > ifFalse: [super fuelAccept: aVisitor] > > > > But the problem is that if #compress: answers an instance of String, you > > should be careful that if #shouldCompressString: asnwers true to the > > compressed string, then you will end up in an infinitive loop. > > > > See: http://rmod.lille.inria.fr/web/pier/software/Fuel/Hooks and > > FLHookedSubstitutionTest > > > > I think StOMP should support this as well: > > http://stomp.smalltalk-users.jp/home/how-to-use-stomp/hook-methods > > > > The previous hook you let you replace the original String with its > > compression during serialization. Which make sense because if you are > going > > to compress it is better to do it during serialization. > > > >> With external I mean it could > >> be in a separate binary (I could arrange the binding though). > > > > I am not sure if I understood. > > > > > > > >> > >> Thanks > >> > >> HernÃ¡n > >> > > > > > > > > -- > > Mariano > > http://marianopeck.wordpress.com > > > > > > -- Mariano http://marianopeck.wordpress.com

Oct. 11, 2011

Re: [Pharo-project] [update 1.4] #14191

by StÃ©phane Ducasse

I rollbacked this fix for now and we are working with alain to find a solution. Stef On Oct 11, 2011, at 4:00 PM, StÃ©phane Ducasse wrote: > Apparently this fixes has a problem so we will have to investigate and I will rollback as soon as I have access to a decent network. > (I hope around 10 pm). > > On Oct 11, 2011, at 3:34 PM, stephane ducasse wrote: > >> >> 14191 >> ----- >> >> Issue 4904: MorphTreeMorph improvements. Thanks Alain Plantec. >> - Speed improvement: 10 to 12 times faster ! >> the speed is now very near from what is possible with PluggableListMorph. >> the TimeProfiler is now very efficient. >> >> - made MorphTreeModel composable (see MCSliceMaker which is not a MorphTreeModel subclass anymore) >> >> - Start to use Annoucements instead of change/update (see MorphTreeSelectionChanged announcement) >> - no item forgotten when sliding with the mouse for auto multiselection (as in the TestRunner, try to select items with the mouse, some items are not selected if the mouse slides too fast) >> - no need for TickList* (8 classes) and no need for a CheckList class >> >> Stef > >

Oct. 11, 2011