October 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] When will Fuel file format stabilize?

by Mariano Martinez Peck

On Sat, Oct 15, 2011 at 11:49 AM, Philippe Marschall <kustos(a)gmx.net> wrote: > On 11.10.2011 21:51, Mariano Martinez Peck wrote: > > > > > > On Tue, Oct 11, 2011 at 8:55 AM, Philippe Marschall > > <philippe.marschall(a)netcetera.ch > > <mailto:philippe.marschall@netcetera.ch>> wrote: > > > > On 10/08/2011 10:42 PM, Mariano Martinez Peck wrote: > > > s > > > > > >>> > > >>> This is IMHO more than necessary for Fuel to become a production > > ready > > >>> serializer and I'd say Fuel is now "old enough" to become such :) > > >> > > >> Yes. > > >> Now what I would love is that even if fuel changes that the > > evolution of > > >> information > > >> is taken into account because like that it will be exercised for > > real. > > >> > > >> > > > No, that's impossible, and if posible, it is not worth it. > > Migrating from an > > > old format to a new one is extremelly complicated and innecessary. > The > > > easiest way to solve this is to take the correct version of Fuel, > > > materialize the graph from the stream, load new version of Fuel, > and > > > seriaize it again. That the easiest, more secure, and more > practical > > > approach I can see. > > > > That is horribly naÃ¯ve an excludes fuel from a lot of use cases. You > > can't use fuel for "archiving" objects outside of the image because > you > > will never know whether you will be able to read them in again > because > > the format changes. You will always need to have "live" ones in the > > image. > > > > > > No. That's incorrect. You won't be able to do that ONLY if you update > > Fuel to a new image that breaks format. > > You can still continue to use the same version and you will never have > > that problem. So, again, why you need to update Fuel? > > Because it's old software. Bugs may not get fixed. It may not work in > newer Pharo versions. I may have dependencies on other libraries that > may require a new version of Fuel. You name it. > > I understand. What I mean is that depending on the changes and the amount of work, you can just adapt Fuel to new versions of Pharo but without changing its format. I mean....say you were in Fuel 1.4. You don't need to move to Fuel 1.8 just because. You can just try to fix 1.4 to make it work in latest pharo. That's EXACTLY what we do with ReferenceStream and friends. The only difference is that the Pharo team does that because it is in the core. But yes, it may happen that ideed you will require a new version of Fuel. > > Why SmartRefStream does not have this problem? because it hasn't > > changed in the last 10 years. > > So..do the same, take an specific Fuel version and keep it for 10 years. > > Just update it to make it work in Pharo without changing the format and > > you are done. > > > > > > > > That means you can't use fuel for anything Monticello related because > > you may never be able to load those versions in again because the > file > > format has changed in the mean time. > > > > > > I guess that in the end, if someone can really do something for > > Monticello on top of Fuel it will be like 2 years from now, and at some > > point Fuel format will be stable. > > And as Stef says...you always have the code there in case of problems. > > Monticello is just an example for a case where I want to store objects > outside of the image. > > Well, then you can just wait until we finish. Give us one year more. Instead of doing 1-year-long releases, we like to do 3 months releases. But again, that doesn't mean you have to update... > Cheers > Philippe > > > -- Mariano http://marianopeck.wordpress.com

Oct. 15, 2011

Re: [Pharo-project] When will Fuel file format stabilize?

by Philippe Marschall

On 11.10.2011 21:51, Mariano Martinez Peck wrote: > > > On Tue, Oct 11, 2011 at 8:55 AM, Philippe Marschall > <philippe.marschall(a)netcetera.ch > <mailto:philippe.marschall@netcetera.ch>> wrote: > > On 10/08/2011 10:42 PM, Mariano Martinez Peck wrote: > > s > > > >>> > >>> This is IMHO more than necessary for Fuel to become a production > ready > >>> serializer and I'd say Fuel is now "old enough" to become such :) > >> > >> Yes. > >> Now what I would love is that even if fuel changes that the > evolution of > >> information > >> is taken into account because like that it will be exercised for > real. > >> > >> > > No, that's impossible, and if posible, it is not worth it. > Migrating from an > > old format to a new one is extremelly complicated and innecessary. The > > easiest way to solve this is to take the correct version of Fuel, > > materialize the graph from the stream, load new version of Fuel, and > > seriaize it again. That the easiest, more secure, and more practical > > approach I can see. > > That is horribly naÃ¯ve an excludes fuel from a lot of use cases. You > can't use fuel for "archiving" objects outside of the image because you > will never know whether you will be able to read them in again because > the format changes. You will always need to have "live" ones in the > image. > > > No. That's incorrect. You won't be able to do that ONLY if you update > Fuel to a new image that breaks format. > You can still continue to use the same version and you will never have > that problem. So, again, why you need to update Fuel? Because it's old software. Bugs may not get fixed. It may not work in newer Pharo versions. I may have dependencies on other libraries that may require a new version of Fuel. You name it. > Why SmartRefStream does not have this problem? because it hasn't > changed in the last 10 years. > So..do the same, take an specific Fuel version and keep it for 10 years. > Just update it to make it work in Pharo without changing the format and > you are done. > > > > That means you can't use fuel for anything Monticello related because > you may never be able to load those versions in again because the file > format has changed in the mean time. > > > I guess that in the end, if someone can really do something for > Monticello on top of Fuel it will be like 2 years from now, and at some > point Fuel format will be stable. > And as Stef says...you always have the code there in case of problems. Monticello is just an example for a case where I want to store objects outside of the image. Cheers Philippe

Oct. 15, 2011

Re: [Pharo-project] P2P library in Smalltalk

by David T. Lewis

On Fri, Oct 14, 2011 at 10:32:44PM +0200, St?phane Ducasse wrote: > > On Oct 14, 2011, at 5:32 PM, Johannes Rasche wrote: > > > Stef, > > > > can you please explain what DVG is ? > > project on squeaksource. > I did not try to load it to pharo. I think it is this: MCHttpRepository location: 'http://www.squeaksource.com/DGV' user: '' password: '' But the project name is 'Digital Society for the Past' so it is not too obvious how to find it unless you understand the Dutch Digitaal Genootschap voor het Verleden ;) Dave > > > > > Johannes > > > > Am 14.10.2011 um 10:30 schrieb St?phane Ducasse: > > > >> found it in DVG > >> > >> Stef > >> > >> On Oct 14, 2011, at 10:04 AM, G?ran Krampe wrote: > >> > >>> Cees de Groot also implemented a P2P library a while back: > >>> > >>> > >>> http://wiki.squeak.org/squeak/5629 > >>> > >>> regards, G?ran > >>> > >> > >> > > > > >

Oct. 14, 2011

Re: [Pharo-project] 1.3 and scamper html package; unexpected emergency evaluator

by Schwab,Wilhelm K

Stef, Other than some simple customizations of the image, I used the one-click (on 32 bit Ubuntu 10.04), loaded the package and evaluated the incorrect line. I'll see if I can trip it up with minimal steps. Bill ________________________________________ From: pharo-project-bounces(a)lists.gforge.inria.fr [pharo-project-bounces(a)lists.gforge.inria.fr] On Behalf Of StÃ©phane Ducasse [stephane.ducasse(a)inria.fr] Sent: Friday, October 14, 2011 4:38 PM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] 1.3 and scamper html package; unexpected emergency evaluator On Oct 14, 2011, at 10:28 PM, Schwab,Wilhelm K wrote: > Stef, all, > > I am doing some well-intentioned reverse-engineering of a system (aka trying to read uncommented code) and have been looking at some html that it produces. Scamper's parser was helpful some years back, so I looked on SqueakSource and found two HTML packages. I had the validation package installed, so I decided to add the scamper parser package. > > Weirdness #1: I was curious about what was going to happen when I installed a second HTML* package. I did NOT expect the existing package(s) to be replaced :( Didn't save that image. yes package as jst string conventions so ... > > I owe you guys some serious testing on 1.3, so I decided to use the one-click to run the scamper package. My plan was to view source on a locally-produced page, select-all+copy and then pull the text from the clipboard to parse it. I *should* have done this: > > HtmlParser parse:Clipboard default clipboardText string readStream. > > Aside from every other item being an empty HtmlAttributes (probably very correct if a little odd looking), it does pretty much exactly what I had in mind. I said I should have done the above. What I in fact did was > > HtmlParser parse:Clipboard default clipboardText string readStream. > > (the #string send is missing). > > Weirdness #2: It's interesting, because I get a vm error (text all over the main window with an offer to open an emergency evaluator) saying *** System error handling failed ***. > > Hitting any key clears the error and displays a walkback that will open a debugger revealing that #replaceHtmlCharRefs is missing, or so that appears to be the problem. Of course the real problem is that I evaluated the wrong expression, but I should simply get the walkback, not the callstack. Is there something I can do to help unravel the cause of the problem? I used the 1.3 one-click from the web site. I do not know. Indeed it looks like the handler did not took place and we get the emergency something. Would be good to get a simple way to reproduce the behavior. > > Bill > > > >

Oct. 14, 2011

Re: [Pharo-project] [squeak-dev] Any serializer can customize serialization strategy?

by Martin Dias

On Fri, Oct 14, 2011 at 2:40 AM, HernÃ¡n Morales Durand < hernan.morales(a)gmail.com> wrote: > Hi Martin > > 2011/10/13 Martin Dias <tinchodias(a)gmail.com>: > > Hi Hernan, > > Could your "specific Strings" be instances of a variant of RemoteString > that > > compresses and decompresses? Then they will occupy less memory in your > image > > and it would be easier to serializer and deserialize the proxy object. > > I'd like to keep my objects cross-dialect, so if that's not an issue > it would be ok to use a RemoteString. I don't see why it would be > easier to serialize/deserialize? > I don't know it has sense, also it depends on your case... but if MyRemoteString just has a instVar 'filename' or 'fileIndex' to fetch the real string from disk when necessary... it should be easy to serialize instances of such MyRemoteString. > > > Thanks for the feedback, I also don't like the visitor interface we have > for > > customizations. In fact, I think it is not a visitor but a > double-dispatch. > > And I would prefer to merge the two hooks Mariano proposed to something > > like: > > String >> > > fuelMapOn: aDefaultMapper (ExternalCompressor shouldCompressString: self) > > ifTrue: [aDefaultMapper mapSubstitution: self by: (ExternalCompressor > > compress: self)] ifFalse: [super fuelMapOn: aDefaultMapper] > > > > Do you like more this way? > > Absolutely :) > thanks... probably for Fuel 1.8! > > > Martin > > > > On Tue, Oct 11, 2011 at 4:10 PM, Mariano Martinez Peck > > <marianopeck(a)gmail.com> wrote: > >> > >> > >> On Tue, Oct 11, 2011 at 12:56 AM, HernÃ¡n Morales Durand > >> <hernan.morales(a)gmail.com> wrote: > >>> > >>> Thanks Mariano, > >>> > >>> That's pretty much what I want, but... why to expose the visitor > >>> interface? ;) > >> > >> Good question. Honesty, it was the only way we have found to do it with > >> Fuel. I think (but I am not sure) that one of the drawbacks of the > pickle > >> format is that it analysis phase (as we call it in Fuel) make some hooks > >> more complicated to implement than a classic serializer. > >> > >>> > >>> Just to give a little feedback of what I have in my mind: > >>> > >>> mySerializer setEngine: ExternalCompressor for: ( MyClass -> > >>> #myStringIVar ) > >>> > >> > >> Well, you can also create an specific Fuel cluster for MyClass that does > >> exactly that. Basically you have to create FLMyClassCluster, implement > >> methods like #serializeWith: and #materializeFrom: and there you > >> serialize/materialize the specific #myStringIVar with a special way and > the > >> rest as normal. Then you subclass the FLDefaultMapper, say YourMapper > and > >> you implement #visitXXX > >> Then you have to implement MyClass >> visitXXX. > >> Of course, you should be able to generalize such cluster and be able to > >> reuse it for different classes or instVars. > >> > >> I am not saying this is the best way. I am just telling you one possible > >> solution you can do by extending Fuel for your needs. Of course, you may > >> need to understand how Fuel works first. But of course, we can help if > you > >> want. > >> > >>> > >>> or > >>> > >>> mySerializer setEngine: ExternalCompressor for: incomingStream > >>> > >>> Now with machines generating daily 6 TBytes of raw data, one of the > >>> keys will be data compression. > >>> > >> > >> it looks like ;) > >> > >> > >>> > >>> 2011/10/10 Mariano Martinez Peck <marianopeck(a)gmail.com>: > >>> > > >>> > > >>> > On Mon, Oct 10, 2011 at 9:35 PM, HernÃ¡n Morales Durand > >>> > <hernan.morales(a)gmail.com> wrote: > >>> >> > >>> >> Is there any serializer (Fuel/StOMP/Ma object > >>> >> serialization/BOSS/SRP/SIXX??) which let me attach an external > >>> >> compressor/decompressor for specific Strings to the > >>> >> serialization/deserialization process? > >>> > > >>> > In Fuel you can do: > >>> > > >>> > String >> fuelSubstitution > >>> > ^ ExternalCompressor compress: self > >>> > > >>> > String >> fuelAccept: aVisitor > >>> > ^ (ExternalCompressor shouldCompressString: self) > >>> > ifTrue: [aVisitor visitSubstitution: self] > >>> > ifFalse: [super fuelAccept: aVisitor] > >>> > > >>> > But the problem is that if #compress: answers an instance of String, > >>> > you > >>> > should be careful that if #shouldCompressString: asnwers true to the > >>> > compressed string, then you will end up in an infinitive loop. > >>> > > >>> > See: http://rmod.lille.inria.fr/web/pier/software/Fuel/Hooks and > >>> > FLHookedSubstitutionTest > >>> > > >>> > I think StOMP should support this as well: > >>> > http://stomp.smalltalk-users.jp/home/how-to-use-stomp/hook-methods > >>> > > >>> > The previous hook you let you replace the original String with its > >>> > compression during serialization. Which make sense because if you are > >>> > going > >>> > to compress it is better to do it during serialization. > >>> > > >>> >> With external I mean it could > >>> >> be in a separate binary (I could arrange the binding though). > >>> > > >>> > I am not sure if I understood. > >>> > > >>> > > >>> > > >>> >> > >>> >> Thanks > >>> >> > >>> >> HernÃ¡n > >>> >> > >>> > > >>> > > >>> > > >>> > -- > >>> > Mariano > >>> > http://marianopeck.wordpress.com > >>> > > >>> > > >>> > >> > >> > >> > >> -- > >> Mariano > >> http://marianopeck.wordpress.com > >> > > > > > >

Oct. 14, 2011

Re: [Pharo-project] 1.3 and scamper html package; unexpected emergency evaluator

by StÃ©phane Ducasse

On Oct 14, 2011, at 10:28 PM, Schwab,Wilhelm K wrote: > Stef, all, > > I am doing some well-intentioned reverse-engineering of a system (aka trying to read uncommented code) and have been looking at some html that it produces. Scamper's parser was helpful some years back, so I looked on SqueakSource and found two HTML packages. I had the validation package installed, so I decided to add the scamper parser package. > > Weirdness #1: I was curious about what was going to happen when I installed a second HTML* package. I did NOT expect the existing package(s) to be replaced :( Didn't save that image. yes package as jst string conventions so ... > > I owe you guys some serious testing on 1.3, so I decided to use the one-click to run the scamper package. My plan was to view source on a locally-produced page, select-all+copy and then pull the text from the clipboard to parse it. I *should* have done this: > > HtmlParser parse:Clipboard default clipboardText string readStream. > > Aside from every other item being an empty HtmlAttributes (probably very correct if a little odd looking), it does pretty much exactly what I had in mind. I said I should have done the above. What I in fact did was > > HtmlParser parse:Clipboard default clipboardText string readStream. > > (the #string send is missing). > > Weirdness #2: It's interesting, because I get a vm error (text all over the main window with an offer to open an emergency evaluator) saying *** System error handling failed ***. > > Hitting any key clears the error and displays a walkback that will open a debugger revealing that #replaceHtmlCharRefs is missing, or so that appears to be the problem. Of course the real problem is that I evaluated the wrong expression, but I should simply get the walkback, not the callstack. Is there something I can do to help unravel the cause of the problem? I used the 1.3 one-click from the web site. I do not know. Indeed it looks like the handler did not took place and we get the emergency something. Would be good to get a simple way to reproduce the behavior. > > Bill > > > >

Oct. 14, 2011

Re: [Pharo-project] BUG on converting to Fraction

by StÃ©phane Ducasse

norberto you just experience our daily life :), doing/fixing is more difficult than talking :) This is not a problem, if you fix the problem let us know. I appreciate that you tried to fix it. Stef On Oct 14, 2011, at 5:30 PM, Norberto Manzanos wrote: > I'm afraid the fix cause more problems than solutions. I don't know what is happening, and I couldn't go deep, but many things are working bad. > So I have to implement temporarily String #asFraction to fullfill my requirements. > > On Thu, Oct 13, 2011 at 10:43 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > thanks you > > http://code.google.com/p/pharo/issues/detail?id=4909 > > Stef > > On Oct 13, 2011, at 2:39 PM, Norberto Manzanos wrote: > > > On both Squeak 3.9 (I supose others too) and Pharo: > > Fraction readFromString: '1/2' ---> 1 > > '1/2' asNumber ---> 1 > > > > fix > > > > Number #readFrom: stringOrStream > > "Answer a number as described on aStream. The number may > > include a leading radix specification, as in 16rFADE" > > | value base aStream sign | > > aStream _ (stringOrStream isString) > > ifTrue: [ReadStream on: stringOrStream] > > ifFalse: [stringOrStream]. > > (aStream nextMatchAll: 'NaN') ifTrue: [^ Float nan]. > > sign _ (aStream peekFor: $-) ifTrue: [-1] ifFalse: [1]. > > (aStream nextMatchAll: 'Infinity') ifTrue: [^ Float infinity * sign]. > > base _ 10. > > value _ Integer readFrom: aStream base: base. > > "this line " > > (aStream peekFor: $/) ifTrue:[^Fraction numerator: value denominator: (aStream upTo: $\) asInteger]. > > " added " > > (aStream peekFor: $r) > > > > ifTrue: > > ["<base>r<integer>" > > (base _ value) < 2 ifTrue: [^self error: 'Invalid radix']. > > (aStream peekFor: $-) ifTrue: [sign _ sign negated]. > > value _ Integer readFrom: aStream base: base]. > > ^ self readRemainderOf: value from: aStream base: base withSign: sign. > > > > -- > > Norberto Manzanos > > Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS) > > FaHCE/UNLP - CONICET > > Calle 48 e/ 6 y 7 s/NÂº - 8Âº piso - oficina 803 > > Tel: +54-221-4230125 interno 262 > > > > > > -- > Norberto Manzanos > Instituto de Investigaciones en Humanidades y Ciencias Sociales (IdIHCS) > FaHCE/UNLP - CONICET > Calle 48 e/ 6 y 7 s/NÂº - 8Âº piso - oficina 803 > Tel: +54-221-4230125 interno 262

Oct. 14, 2011

Re: [Pharo-project] P2P library in Smalltalk

by StÃ©phane Ducasse

On Oct 14, 2011, at 5:32 PM, Johannes Rasche wrote: > Stef, > > can you please explain what DVG is ? project on squeaksource. I did not try to load it to pharo. > > Johannes > > Am 14.10.2011 um 10:30 schrieb StÃ©phane Ducasse: > >> found it in DVG >> >> Stef >> >> On Oct 14, 2011, at 10:04 AM, GÃ¶ran Krampe wrote: >> >>> Cees de Groot also implemented a P2P library a while back: >>> >>> >>> http://wiki.squeak.org/squeak/5629 >>> >>> regards, GÃ¶ran >>> >> >> > >

Oct. 14, 2011

Re: [Pharo-project] Native Windows

by StÃ©phane Ducasse

On Oct 14, 2011, at 6:19 PM, Sean P. DeNigris wrote: > What are the current options/future-possibilities? > > I was playing with Fffenestri's DisplayHostWindow, which kind of works in > 1.3. What ever happened to that project? What is the status of Mars? Are > there any other avenues? For the moment igor is working on a zoomable canvas with a cairo back-end so no time for the multiple native windows. But I would love to have that too. > > Thanks. > Sean > > -- > View this message in context: http://forum.world.st/Native-Windows-tp3905413p3905413.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

Oct. 14, 2011

[Pharo-project] 1.3 and scamper html package; unexpected emergency evaluator

by Schwab,Wilhelm K

Stef, all, I am doing some well-intentioned reverse-engineering of a system (aka trying to read uncommented code) and have been looking at some html that it produces. Scamper's parser was helpful some years back, so I looked on SqueakSource and found two HTML packages. I had the validation package installed, so I decided to add the scamper parser package. Weirdness #1: I was curious about what was going to happen when I installed a second HTML* package. I did NOT expect the existing package(s) to be replaced :( Didn't save that image. I owe you guys some serious testing on 1.3, so I decided to use the one-click to run the scamper package. My plan was to view source on a locally-produced page, select-all+copy and then pull the text from the clipboard to parse it. I *should* have done this: HtmlParser parse:Clipboard default clipboardText string readStream. Aside from every other item being an empty HtmlAttributes (probably very correct if a little odd looking), it does pretty much exactly what I had in mind. I said I should have done the above. What I in fact did was HtmlParser parse:Clipboard default clipboardText string readStream. (the #string send is missing). Weirdness #2: It's interesting, because I get a vm error (text all over the main window with an offer to open an emergency evaluator) saying *** System error handling failed ***. Hitting any key clears the error and displays a walkback that will open a debugger revealing that #replaceHtmlCharRefs is missing, or so that appears to be the problem. Of course the real problem is that I evaluated the wrong expression, but I should simply get the walkback, not the callstack. Is there something I can do to help unravel the cause of the problem? I used the 1.3 one-click from the web site. Bill

Oct. 14, 2011