December 2012 - Pharo-dev - lists.pharo.org

[Pharo-project] Smalltalks Videos

by Marcus Denker

There are now a lot of Videos online from Smalltalks, both last month and 2011. *A lot* of Pharo related videos! Playlist 2012: https://www.youtube.com/playlist?list=PLCGAAdUizzH31VumrhrK2HHepHu3DBpY0 Playlist 2011: https://www.youtube.com/playlist?list=PLE90823AFF9C8177B Marcus

Dec. 3, 2012

Re: [Pharo-project] Performance Tip: Using buffered streams

by Sven Van Caekenberghe

On 03 Dec 2012, at 12:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > There are ZnCharacterReadStream and ZnCharacterWriteStream that add decoding/encoding to binary streams. They could indeed be used as an alternative. But that was not my point. I reran the examples using a binary file stream to disable the decoding and then using ZnCharacterReadStream to handle the decoding: [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun. 3502 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1214 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun 2869 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun 1280 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: (ZnCharacterReadStream on: stream binary)) next ] ] timeToRun. 1655 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: (ZnCharacterReadStream on: stream binary encoding: 'latin1')) next ] ] timeToRun. 1574 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: (ZnCharacterReadStream on: fstream binary) do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1753 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: (ZnCharacterReadStream on: fstream binary encoding: 'latin1') do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1655 This shows several things (for this particular test case): - ZnCharacterReadStream is at least as good as the builtin decoding streams (here twice as good) - UTF-8 (ZnUTF8Encoder) and Latin1 (ZnNullEncoder) decoding are close together (because the input is ASCII) - ZnBufferedReadStream does not help when the stream is pretty good (ZnCharacterReadStream actually uses a 1 character buffer to support peek, which is important to speed up this use case) - this is to be expected: it adds a layer, but its fetch next buffer logic still loops using #next, decoding possibly Unicode characters - the super fast runs (the 2nd and the 4th) are actually cheating because they skip decoding - thanks for that observation ! Sven PS: While doing these explorations, I actually updated ZnUTF8Encoder quite a bit, both when handling ASCII, which is the majority of characters in most cases, as well as in the general case; see the latest version. -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Dec. 3, 2012

[Pharo-project] pharo-2.0-tests Â» win - Build # 496 - Failure!

by jenkins-pharo.ci.inria.fr＠jenkins-server.lille.inria.fr

Dec. 3, 2012

Re: [Pharo-project] Vim Keys?

by Goubier Thierry

Le 03/12/2012 15:23, Camillo Bruni a Ã©crit : > > On 2012-12-03, at 10:02, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: > >> Le 03/12/2012 13:31, Camillo Bruni a Ã©crit : >>>> Well, it's allmost there. The only thing that I didn't manage to push was the removal of the global, class-linked maps (it was rejected)... This is the only step missing for your vim shortcuts to work. >>> >>> Maybe I can revive the discussion here, sadly it died on the tracker :/. >> >>> I would really love to see your changes in the image! >>> However it slightly (changes/breaks depending on the viewer :P) some focus-cycling in certain tools. >> >> There is two different issues : >> >> vim-like shortcuts and focus. >> >> First, vim-like (and emacs, and whatever); this requires cutting off matching of the default shortcuts (since those often conflicts). These defaults being non configurable, Sean (for vim) and I (for my dynamically loaded / matched to current menu shortcuts), had to disconnect from matching the default class-based keymaps by on-the-fly patching of a target Morph KMDispatcher :() Hugly hack, this one. > > I think that should be fine. I remember I did something similar once with Keymapping (in the early days). Doru needed the very same for Moose, so we tried to make it work. I ended up with creating a subclass of KMDispatcher or so, which has a local key-mapping. So there are more or less elegant solutions to this problem (making it work is the main priority :P) Yes, but, still, the code to find the right KMDispatcher to replace is ugly, especially for PluggableTextMorph (since you have not one, but two KMDispatcher to replace). Once done, it works perfectly. You just keep that knowledge somewhere to be able to recreate it (PluggableTextMorph... Hum... Two KMDispatcher to find and replace; MorphTreeMorph... one KMDispatcher instance to replace :) ). >> Second, keyboard focus navigation. First, what I did is the following : correct a bug or two in the Morphic based focus navigation code. And, second, move the focus navigation shortcuts to Keymapping. Nothing else. > > yeah this is how it is supposed to be, no special trickery needed otherwise. neat! > >> However, this exposed one fact about Morphic: that the way to add submorphs is LIGT (Last In Goes to Top i.e: in front of the previous submorphs). This is great for a drawing editor, and bad for an UI design tool, such as Spec. So Spec loads in an apparent reverse order (at least relative to the focus order), and this is what you see. > > right > >>> Is there a way to come up with a more or less compatible things as the focus list? >>> Aka a nice and simple interface on top of your changes to achieve the same thing? >> >> Yes. Just have Spec use on:do: (or load the right keymap) for focus change, instead of using eventHandler. I do believe this change is just around the corner... >> >> Once this is done, any focus order you set in Spec will apply first when focus tabbing. >> >> However, you may then see design faults : if you forget one element in your focus ordering in Spec, you will never manage to focus it by the keyboard. Whereas the Morphic one will always tab through all elements whatever way you build the GUI. > > right > >> IMO, the best would be Spec being able to order submorphs correctly and to then rely on the Morphic focus code. But there must be a good reason for Spec not doing it. I just don't know why? I tried changing a bit Spec for that (changing the default ordering where I could see it, then changing a few Specs here and there to make sure the GUI would build and tab right) and couldn't find anything wrong with it, but I just checked on a few GUI samples. > > I think the only thing with major focus issues was the refactoring-changes browser. > - do a class-rename with the refactoring menu > - try to tab through the changes in the list And what is happening then? Do you tab out the list? >> (This is also because the focus switch code in Morphic is a lot shorter than the Spec one, and I tend to believe the shorter the better (and no duplication). But that's just me:)). > > I think I don't know enough of the internal details, but I saw the 2 sides of the discussion: > - Ben said that it won't work without an explicit focus-order list There is maybe something about Spec component-based approach and composing focus navigation there. The next focus code in Spec is fairly complex when you look at it from the composition angle. Another issue is : do you need to have Spec handling the focus navigation for Spec to work well on multi targets (Morphic, Amber, HTML5, etc...) ? This would be a very important question for me if this multi-toolkit aspect is one of the design goal for Spec. > - I loaded your code, and roughly all tools still worked :) (minor issues with the changes browser for instance) And, since then, you were bypassing Spec focus order allmost everywhere :) > Conclusion: > - Your approach seems to make sense as an approximation of the common case, which is important. > - But we need custom control in certain cases. Yes. This is possible in both (either with the Morphic one or the Spec focus order) by setting the focus yourself when you want it. Just call takeKeyboardFocus on the Morph you want. You just have to take care that you don't throw the focus code in an endless loop by shortcutting the order (that was the bug in Morphic : it would loop forever because of a send navigation key to an owner Morph somewhere :( ). >>> We desperately need this refactoring, and if we don't use it, just because of the focus list, I think it will take another year until they make it into the image :( >> >> I thought it was integrated. I'll check. It is integrated; what I guess is missing is the Spec override. > > http://code.google.com/p/pharo/issues/detail?id=6736 > http://code.google.com/p/pharo/issues/detail?id=6657 > > > thanks for looking into it! It's a pleasure to work in Pharo, you know that feeling :) Thierry -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Dec. 3, 2012

Re: [Pharo-project] Vim Keys?

by Camillo Bruni

On 2012-12-03, at 10:02, Goubier Thierry <thierry.goubier(a)cea.fr> wrote: > Le 03/12/2012 13:31, Camillo Bruni a Ã©crit : >>> Well, it's allmost there. The only thing that I didn't manage to push was the removal of the global, class-linked maps (it was rejected)... This is the only step missing for your vim shortcuts to work. >> >> Maybe I can revive the discussion here, sadly it died on the tracker :/. > >> I would really love to see your changes in the image! >> However it slightly (changes/breaks depending on the viewer :P) some focus-cycling in certain tools. > > There is two different issues : > > vim-like shortcuts and focus. > > First, vim-like (and emacs, and whatever); this requires cutting off matching of the default shortcuts (since those often conflicts). These defaults being non configurable, Sean (for vim) and I (for my dynamically loaded / matched to current menu shortcuts), had to disconnect from matching the default class-based keymaps by on-the-fly patching of a target Morph KMDispatcher :() Hugly hack, this one. I think that should be fine. I remember I did something similar once with Keymapping (in the early days). Doru needed the very same for Moose, so we tried to make it work. I ended up with creating a subclass of KMDispatcher or so, which has a local key-mapping. So there are more or less elegant solutions to this problem (making it work is the main priority :P) > Second, keyboard focus navigation. First, what I did is the following : correct a bug or two in the Morphic based focus navigation code. And, second, move the focus navigation shortcuts to Keymapping. Nothing else. yeah this is how it is supposed to be, no special trickery needed otherwise. neat! > However, this exposed one fact about Morphic: that the way to add submorphs is LIGT (Last In Goes to Top i.e: in front of the previous submorphs). This is great for a drawing editor, and bad for an UI design tool, such as Spec. So Spec loads in an apparent reverse order (at least relative to the focus order), and this is what you see. right >> Is there a way to come up with a more or less compatible things as the focus list? >> Aka a nice and simple interface on top of your changes to achieve the same thing? > > Yes. Just have Spec use on:do: (or load the right keymap) for focus change, instead of using eventHandler. I do believe this change is just around the corner... > > Once this is done, any focus order you set in Spec will apply first when focus tabbing. > > However, you may then see design faults : if you forget one element in your focus ordering in Spec, you will never manage to focus it by the keyboard. Whereas the Morphic one will always tab through all elements whatever way you build the GUI. right > IMO, the best would be Spec being able to order submorphs correctly and to then rely on the Morphic focus code. But there must be a good reason for Spec not doing it. I just don't know why? I tried changing a bit Spec for that (changing the default ordering where I could see it, then changing a few Specs here and there to make sure the GUI would build and tab right) and couldn't find anything wrong with it, but I just checked on a few GUI samples. I think the only thing with major focus issues was the refactoring-changes browser. - do a class-rename with the refactoring menu - try to tab through the changes in the list > (This is also because the focus switch code in Morphic is a lot shorter than the Spec one, and I tend to believe the shorter the better (and no duplication). But that's just me:)). I think I don't know enough of the internal details, but I saw the 2 sides of the discussion: - Ben said that it won't work without an explicit focus-order list - I loaded your code, and roughly all tools still worked :) (minor issues with the changes browser for instance) Conclusion: - Your approach seems to make sense as an approximation of the common case, which is important. - But we need custom control in certain cases. >> We desperately need this refactoring, and if we don't use it, just because of the focus list, I think it will take another year until they make it into the image :( > > I thought it was integrated. I'll check. It is integrated; what I guess is missing is the Spec override. http://code.google.com/p/pharo/issues/detail?id=6736 http://code.google.com/p/pharo/issues/detail?id=6657 thanks for looking into it!

Dec. 3, 2012

Re: [Pharo-project] Vim Keys?

by Goubier Thierry

Le 03/12/2012 13:31, Camillo Bruni a Ã©crit : >> Well, it's allmost there. The only thing that I didn't manage to push was the removal of the global, class-linked maps (it was rejected)... This is the only step missing for your vim shortcuts to work. > > Maybe I can revive the discussion here, sadly it died on the tracker :/. > I would really love to see your changes in the image! > However it slightly (changes/breaks depending on the viewer :P) some focus-cycling in certain tools. There is two different issues : vim-like shortcuts and focus. First, vim-like (and emacs, and whatever); this requires cutting off matching of the default shortcuts (since those often conflicts). These defaults being non configurable, Sean (for vim) and I (for my dynamically loaded / matched to current menu shortcuts), had to disconnect from matching the default class-based keymaps by on-the-fly patching of a target Morph KMDispatcher :() Hugly hack, this one. Second, keyboard focus navigation. First, what I did is the following : correct a bug or two in the Morphic based focus navigation code. And, second, move the focus navigation shortcuts to Keymapping. Nothing else. However, this exposed one fact about Morphic: that the way to add submorphs is LIGT (Last In Goes to Top i.e: in front of the previous submorphs). This is great for a drawing editor, and bad for an UI design tool, such as Spec. So Spec loads in an apparent reverse order (at least relative to the focus order), and this is what you see. > Is there a way to come up with a more or less compatible things as the focus list? > Aka a nice and simple interface on top of your changes to achieve the same thing? Yes. Just have Spec use on:do: (or load the right keymap) for focus change, instead of using eventHandler. I do believe this change is just around the corner... Once this is done, any focus order you set in Spec will apply first when focus tabbing. However, you may then see design faults : if you forget one element in your focus ordering in Spec, you will never manage to focus it by the keyboard. Whereas the Morphic one will always tab through all elements whatever way you build the GUI. IMO, the best would be Spec being able to order submorphs correctly and to then rely on the Morphic focus code. But there must be a good reason for Spec not doing it. I just don't know why? I tried changing a bit Spec for that (changing the default ordering where I could see it, then changing a few Specs here and there to make sure the GUI would build and tab right) and couldn't find anything wrong with it, but I just checked on a few GUI samples. (This is also because the focus switch code in Morphic is a lot shorter than the Spec one, and I tend to believe the shorter the better (and no duplication). But that's just me:)). > We desperately need this refactoring, and if we don't use it, just because of the focus list, I think it will take another year until they make it into the image :( I thought it was integrated. I'll check. It is integrated; what I guess is missing is the Spec override. Thierry -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Dec. 3, 2012

Re: [Pharo-project] Vim Keys?

by Camillo Bruni

> Well, it's allmost there. The only thing that I didn't manage to push was the removal of the global, class-linked maps (it was rejected)... This is the only step missing for your vim shortcuts to work. Maybe I can revive the discussion here, sadly it died on the tracker :/. I would really love to see your changes in the image! However it slightly (changes/breaks depending on the viewer :P) some focus-cycling in certain tools. Is there a way to come up with a more or less compatible things as the focus list? Aka a nice and simple interface on top of your changes to achieve the same thing? We desperately need this refactoring, and if we don't use it, just because of the focus list, I think it will take another year until they make it into the image :(

Dec. 3, 2012

Re: [Pharo-project] Performance Tip: Using buffered streams

by Sven Van Caekenberghe

On 03 Dec 2012, at 12:22, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > In other words, to get comparable runtimes, you'd have to include a ZnEncodedStream doing UTF-8 decoding on top of the stack used by the STON-reader. There are ZnCharacterReadStream and ZnCharacterWriteStream that add decoding/encoding to binary streams. They could indeed be used as an alternative. But that was not my point. > (At which point you're well on the way to reimplementing the modular streams of Xtreams, but with a more traditional API :) ) Many people, including myself are interested in Xtreams - many fun things will happen in the future ;-) -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Dec. 3, 2012

Re: [Pharo-project] Performance Tip: Using buffered streams

by Sven Van Caekenberghe

Hendrik, Thanks for entering this discussion, it is important territory, for many users. We should get the basics right. On 03 Dec 2012, at 12:10, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > The reduced read times you are seeing, is lack of decoding, not lack of buffering. I should have been more clear: it is a combination of buffering & decoding (see further). > BufferedReadStream uses readInto:startingAt:count:, which isn't reimplemented for MultiByteFileStream to do proper decoding, and thus inherits the one from StandardFileStream. If that is so (and it seems to be the case), then that is an important bug, wouldn't you agree ? > The filestreams already do read buffering (albeit with a smaller buffer size), so when Fuel uses binary mode (which skips decoding), the performance is pretty much the same. Yeah, in binary mode, MultiByteFilestream does not do encoding. Still, IMO that is not the only reason for the difference. However, even when I go straight to MultiByteFilestream's superclass, StandardFileStream, which does not do decoding, the additional buffering helps a lot: [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun 2869 [ StandardFileStream fileNamed: '/tmp/numbers.json' do: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun 1280 I said from the start that the access pattern matters: parser code is a sweet spot here. Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Dec. 3, 2012

Re: [Pharo-project] Performance Tip: Using buffered streams

by Henrik Sperre Johansen

On 03.12.2012 12:10, Henrik Sperre Johansen wrote: > On 03.12.2012 09:30, Sven Van Caekenberghe wrote: >> On 03 Dec 2012, at 07:25, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >>> Hi Mariano, >>> >>> On 02 Dec 2012, at 23:43, Mariano Martinez Peck >>> <marianopeck(a)gmail.com> wrote: >>> >>>> This is really really interesting, because it means a speed up of >>>> 2x when reading :) >>>> Where can I get the last version of ZnBufferedReadStream? >>> I knew you would be interested: it is in the latest Zn, either >>> http://mc.stfx.eu or squeaksource (not in the metacello yet). >>> It should work on binary streams too, I don't know whether it would >>> make a lot of difference, I would guess so. >>> Maybe Fuel uses lots of 'deterministic reading' (size prefixed) >>> instead of parsing. >> Well, I just tried. The good news is, it works, the bad news is, it >> does not make a difference, either way: >> >> | data | >> data := STONTestMap classTreeExtended. >> [ FLSerializer serialize: data toFileNamed: '/tmp/all.fuel' ] >> timeToRun. 4967 >> >> [ FLMaterializer materializeFromFileNamed: '/tmp/all.fuel' ] >> timeToRun. 1095 >> >> [ '/tmp/all.fuel' asFileReference readStreamDo: [ :fstream | >> ZnBufferedReadStream on: fstream binary do: [ :stream | >> FLMaterializer newDefault materializeFrom: stream ] ] ] >> timeToRun. 1043 >> >> So I guess Fuel is already pretty optimised, especially for reading ;-) >> >> I think ZnBufferedReadStream only makes a difference for parsers that >> are sending lots of little #next, #peek and #atEnd messages in loops. >> Binary protocols are typically more deterministic, they know ahead >> what they have to read and don't have to do #peek or #atEnd tests all >> the time. >> >> Sven >> >> -- >> Sven Van Caekenberghe >> http://stfx.eu >> Smalltalk is the Red Pill > The reduced read times you are seeing, is lack of decoding, not lack > of buffering. > > BufferedReadStream uses readInto:startingAt:count:, which isn't > reimplemented for MultiByteFileStream to do proper decoding, and thus > inherits the one from StandardFileStream. > > The filestreams already do read buffering (albeit with a smaller > buffer size), so when Fuel uses binary mode (which skips decoding), > the performance is pretty much the same. > > Cheers, > Henry In other words, to get comparable runtimes, you'd have to include a ZnEncodedStream doing UTF-8 decoding on top of the stack used by the STON-reader. (At which point you're well on the way to reimplementing the modular streams of Xtreams, but with a more traditional API :) ) Cheers, Henry

Dec. 3, 2012