February 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] ||

by Marcus Denker

> On 05 Feb 2015, at 16:35, Levente Uzonyi <leves(a)elte.hu> wrote: > > On Thu, 5 Feb 2015, Marcus Denker wrote: > >> >>> On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker(a)inria.fr> wrote: >>> >>> >>>> On 04 Feb 2015, at 22:04, Levente Uzonyi <leves(a)elte.hu> wrote: >>>> >>>> A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. >>>> Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine? >>> >>> I have not yet benchmarked itâ¦ PetitParser as it is is too slow, but we will soon have a faster version (factor 10). >>> >>> We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. >>> (there is a setting 'AST based coloringâ in Pharo3 and Pharo4, but it is turned off by default). >>> >>> One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined >>> by the AST nodes and structural navigation in the editor. >>> > > Rebuilding the whole AST after every keystroke is possible, but keeping it real-time is a bit challenging. > > I would love to see an editor, which works on the AST directly - aka it maps the source code to AST nodes, and just updates the smallest possible subtree at each keystroke. Implementing such editor has it's challenges ofc, like typing a single character can invalidate the whole code, but the editor should keep the AST somehow in that case too. > Yes! Marcus

Feb. 5, 2015

Re: [Pharo-dev] ||

by Levente Uzonyi

On Thu, 5 Feb 2015, Marcus Denker wrote: > >> On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> >>> On 04 Feb 2015, at 22:04, Levente Uzonyi <leves(a)elte.hu> wrote: >>> >>> A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. >>> Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine? >> >> I have not yet benchmarked itâ¦ PetitParser as it is is too slow, but we will soon have a faster version (factor 10). >> >> We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. >> (there is a setting 'AST based coloringâ in Pharo3 and Pharo4, but it is turned off by default). >> >> One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined >> by the AST nodes and structural navigation in the editor. >> Rebuilding the whole AST after every keystroke is possible, but keeping it real-time is a bit challenging. I would love to see an editor, which works on the AST directly - aka it maps the source code to AST nodes, and just updates the smallest possible subtree at each keystroke. Implementing such editor has it's challenges ofc, like typing a single character can invalidate the whole code, but the editor should keep the AST somehow in that case too. > > Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978? > > What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an > extreme, while today we are obsessed to find reasons why we can not do anything that makes the system > slower or use more memory than yesterday. And that even with resources growing every yearâ¦ > > This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people > who will see these ~7000 objects as pure wasteâ¦ while I would say that we have already *now* the resources to be > even more radical. I think this is a different thing. Improvements are always welcome, as long as they don't step on your toes. About Slots: I don't see their advantages yet (other than bitfields, but those are so rare in Smalltalk that I implement them causes no trouble. And they are just an optimization over using multiple fields.). Levente > > Marcus > > >

Feb. 5, 2015

Re: [Pharo-dev] [CI] help needed: integration of Fuel changes with new version number

by Max Leske

> On 05 Feb 2015, at 15:19, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > >> On 05 Feb 2015, at 11:35, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >> Hi guys >> >> ClÃ©ment asked us a while ago to update Fuel for Spur support and we did (https://pharo.fogbugz.com/f/cases/14688/Change-CompiledMethod-header-serial… <https://pharo.fogbugz.com/f/cases/14688/Change-CompiledMethod-header-serial…>). The problem now: Fuel is an integral part of the integration process and the change to the Fuel version number means that the automatic integration will not work. Hereâs why: >> >> This slice can't be automatically integrated because I changed the Fuel version number. Now, when the image worker reads the result from the validation, Fuel raises an exception because of the different version numbers in the parent and child image. >> >> We should probably add a mechanism that ensures that the original Fuel version is reloaded after the validation, such that the Fuel file is written by the same version that then reads it in the parent. >> >> Can someone please manually integrate or (which would be way cooler) adapt the integration process such that future updates to Fuel can be automatically integrated? >> > Ok, I have put it on Fix To Include (after checking that it merges well) and will include it with the next update. > > Marcus > Great, thanks.

Feb. 5, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/40477 Home: https://github.com/pharo-project/pharo-core

Feb. 5, 2015

[pharo-project/pharo-core] 6a5096: 40477

by GitHub

Branch: refs/heads/4.0 Home: https://github.com/pharo-project/pharo-core Commit: 6a509693b22a9920f2a59f5c5194e612c27af9fc https://github.com/pharo-project/pharo-core/commit/6a509693b22a9920f2a59f5c… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-02-05 (Thu, 05 Feb 2015) Changed paths: M Fuel.package/FLCompiledMethodCluster.class/instance/serialize%2Fmaterialize/materializeInstanceWith_.st M Fuel.package/FLCompiledMethodCluster.class/instance/serialize%2Fmaterialize/serializeInstance_with_.st M Fuel.package/FLMaterializer.class/class/protected/currentVersion.st M Fuel.package/FLSerializer.class/class/protected/currentVersion.st A Kernel.package/Delay.class/instance/delaying/waitOtherwise_.st M Morphic-Core.package/WorldState.class/instance/update cycle/interCyclePause_.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - scripts/script477.st A ScriptLoader40.package/ScriptLoader.class/instance/pharo - updates/update40477.st M ScriptLoader40.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 40477 14688 Change CompiledMethod header serialization to use signed integer https://pharo.fogbugz.com/f/cases/14688 14669 Delay refactoring (part 2a) - avoid UI locking up when timer event loop is stopped https://pharo.fogbugz.com/f/cases/14669 http://files.pharo.org/image/40/40477.zip

Feb. 5, 2015

Re: [Pharo-dev] Proposal for LiteralArray class

by Sven Van Caekenberghe

> On 05 Feb 2015, at 15:44, Nicolai Hess <nicolaihess(a)web.de> wrote: > > 2015-02-02 3:03 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > > > On Sun, Feb 1, 2015 at 3:39 AM, Nicolai Hess <nicolaihess(a)web.de> wrote: > > > 2015-02-01 10:52 GMT+01:00 Ben Coman <btc(a)openinworld.com>: > > Looking into Image locking problems [1] caused by a recursive array such as this... > > literalArray := #( 1 2 3 ). > literalArray at: 3 put: literalArray. > > I find that "literalArray printString" locks the image due to Array>>printOn: use of the recursive #shouldBePrintedAsLiteral method. Now its implementation is identical to #isLiteral and indeed "literalArray isLiteral" also locks the Image. So comparing implementors of #isLiteral... > > > > Squeak uses a Set to store all visited elements for shouldBePrintedAsLiteral and this protects against the recursive loop. > > shouldBePrintedAsLiteralVisiting: aSet > self class == Array ifFalse: > [^false]. > (aSet includes: self) ifTrue: > [^false]. > aSet add: self. > ^self allSatisfy: [:each | each shouldBePrintedAsLiteralVisiting: aSet] > > > isn't there a common pattern to handle this kind of potential endless recursion? > > At Cadence we fixed it thus: > > Object>>shouldBePrintedAsLiteral > > ^self isLiteral > > Array>>shouldBePrintedAsLiteral > > ^self class == Array > and: [self shouldBePrintedAsLiteralVisiting: (IdentitySet new: 8)] > > Object>>shouldBePrintedAsLiteralVisiting: aSet > > ^self isLiteral > > Array>>shouldBePrintedAsLiteralVisiting: aSet > self class == Array ifFalse: > [^false]. > (aSet includes: self) ifTrue: > [^false]. > aSet add: self. > ^self allSatisfy: [:each | each shouldBePrintedAsLiteralVisiting: aSet] > > > Is there something more "generic". Something we can use for any object tracing. > Isn't there something the GC uses? The GC obviously does not fall into this loop. > (It flags visited objects, but there is nothing exposed that can be used > at the image level?) > How do ImageSegment or Fuel work with recursive structures? In Moose there is DeepTraverser which does something similar it seems. FUEL & STON do this too. > Nicolai > > > > Object>>isLiteral ^false > Boolean>>isLiteral ^true > Character>>isLiteral ^true > Integer>>isLiteral ^true > String>>isLiteral ^true > UndefinedObject>>isLiteral ^true > > ByteArray>>isLiteral ^self class == ByteArray > Float>>isLiteral ^self isFinite "^(self - self) = 0.0" > ScaledDecimal>>isLiteral ^denominator = 1 or: [(10 raisedTo: scale)\\denominator = 0] > > Array>>isLiteral ^self class == Array and: [self allSatisfy: [:each | each isLiteral]] > > ...I find most are very basic (might even say deterministic), with the recursion of Array>>isLiteral seeming an annomaly. Also, the big IF condition in Array>>printOn: smells like a design decision being made at runtime (Valloud AMCOS p12). > > Array>>printOn: aStream > self shouldBePrintedAsLiteral ifTrue: [self printAsLiteralFormOn: aStream. ^ self]. > self isSelfEvaluating ifTrue: [self printAsSelfEvaluatingFormOn: aStream. ^ self]. > super printOn: aStream > > Flipping between two printString formats seems like selecting between two class types. Indeed, if we had a LiteralArray class, there would be no need for its printOn: to recursively search to determine its form, thus allowing #printStringLimitedTo: to do its thing to protect against infinite recursion. > > Also, instead of a recursive Array>>isLiteral we'd have something like > LiteralArray>>isLiteral ^true > Array>>isLiteral ^false > which seems to align much better with the pattern of the other #isLiteral implementors. > > I notice there is both RBArrayNode and RBLiteralArrayNode. > > So what are the wider concerns that might apply? > (In particular, I'm not sure how the #isSelfEvaluating (which is also recursive) fits into the big picture) > > cheers -ben > > [1] https://www.mail-archive.com/pharo-dev@lists.pharo.org/msg25156.html > > > > > -- > best, > Eliot

Feb. 5, 2015

Re: [Pharo-dev] TROSMTileProvider

by Sven Van Caekenberghe

Thierry, Onil, I rounded up my experiments with a 2nd commit on my experimental branch: === Name: Roassal2-SvenVanCaekenberghe.720 Author: SvenVanCaekenberghe Time: 5 February 2015, 3:40:43.211681 pm UUID: 121ffc22-4a30-4153-bef1-e019a92862bc Ancestors: Roassal2-SvenVanCaekenberghe.719 Experimental branch. Second version. Some more changes after discussing with Thierry @ Pharo Days and hours of experimenting Added TROSMTileProvider>>#cachedTileNamed: and #memoryCachedTileNamed: Rewrote/simplified TROSMShape>>#getTile: to no longer fork a 'get tile' process when the tile is cached in memory or on the file system Removed all hacks trying to optimize the number of times #signalUpdate was called - made no difference. Tried several logging approaches but that did not reveal anything special. The observed slowdown for tiles in the file cache but not the memory cache is still there, even though loading about 10 tiles takes less than 40 ms (see #testEmptyMemoryCacheParisLevel7) === Summary, it should be faster, but it is not. Forking and network downloading are gone, each of about 10 tiles takes less than 4 ms, the total observed slowdown is still about 1 to 2 seconds - with no extra #signalUpdate. I don't know why and I don't understand. I hope somebody can have a look, I feel that we can/should make this better still. Sven > On 28 Jan 2015, at 16:58, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Hi Sven, > > Thanks for the work. I'll have a try (if I manage to get the time) and we'll meet tomorrow for sure :) > > Thierry > > 2015-01-28 14:43 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Hi Thierry, Onil, > > First let me iterate what I wrote last week, the OSM stuff in Roassal is very cool, very well written. Great work. > > Since I have some experience with OSM maps, tiles and serving them, I had a look to see if I could optimise the tile drawing/loading, which is visibly a bit slow, one would assume because of the loading of the tiles over the network. > > I just committed (based on code that I wrote some time ago): > > === > Name: Roassal2-SvenVanCaekenberghe.719 > Author: SvenVanCaekenberghe > Time: 28 January 2015, 2:31:50.462461 pm > UUID: 0978fb6a-d6d0-4d86-a9ae-1f2f83a0b6a2 > Ancestors: Roassal2-AlexandreBergel.718 > > Experimental branch. > > Introduction of TROSMTileProvider to optimize getting OSM tiles, adds one shared memory cache and a local file system cache, predefines and reuses 6 http clients spread over different hosts. > > Modifies TROSMShape>>#getTile: and adds TROSMShape>>#tileNamed: > === > > The strange thing is though, that I can optimise the loading (as benchmarked separately), but in most cases (not when they are in the memory cache, but when they are in the file cache) the drawing still has the same pauses as before - which I don't understand, at all. But it is a bit hard to explain. > > Anyway, Thierry, I will see you tomorrow and the day thereafter and I hope I can show you what I mean. > > Regards, > > Sven > > >

Feb. 5, 2015

Re: [Pharo-dev] Proposal for LiteralArray class

by Nicolai Hess

2015-02-02 3:03 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > > > On Sun, Feb 1, 2015 at 3:39 AM, Nicolai Hess <nicolaihess(a)web.de> wrote: > >> >> >> 2015-02-01 10:52 GMT+01:00 Ben Coman <btc(a)openinworld.com>: >> >>> >>> Looking into Image locking problems [1] caused by a recursive array such >>> as this... >>> >>> literalArray := #( 1 2 3 ). >>> literalArray at: 3 put: literalArray. >>> >>> I find that "literalArray printString" locks the image due to >>> Array>>printOn: use of the recursive #shouldBePrintedAsLiteral method. Now >>> its implementation is identical to #isLiteral and indeed "literalArray >>> isLiteral" also locks the Image. So comparing implementors of #isLiteral... >>> >>> >>> >> >> Squeak uses a Set to store all visited elements for >> shouldBePrintedAsLiteral and this protects against the recursive loop. >> >> shouldBePrintedAsLiteralVisiting: aSet >> self class == Array ifFalse: >> [^false]. >> (aSet includes: self) ifTrue: >> [^false]. >> aSet add: self. >> ^self allSatisfy: [:each | each shouldBePrintedAsLiteralVisiting: >> aSet] >> >> >> isn't there a common pattern to handle this kind of potential endless >> recursion? >> > > At Cadence we fixed it thus: > > Object>>shouldBePrintedAsLiteral > > ^self isLiteral > > Array>>shouldBePrintedAsLiteral > > ^self class == Array > and: [self shouldBePrintedAsLiteralVisiting: (IdentitySet new: 8)] > > Object>>shouldBePrintedAsLiteralVisiting: aSet > > ^self isLiteral > > Array>>shouldBePrintedAsLiteralVisiting: aSet > self class == Array ifFalse: > [^false]. > (aSet includes: self) ifTrue: > [^false]. > aSet add: self. > ^self allSatisfy: [:each | each shouldBePrintedAsLiteralVisiting: aSet] > Is there something more "generic". Something we can use for any object tracing. Isn't there something the GC uses? The GC obviously does not fall into this loop. (It flags visited objects, but there is nothing exposed that can be used at the image level?) How do ImageSegment or Fuel work with recursive structures? Nicolai > > Object>>isLiteral ^false >>> Boolean>>isLiteral ^true >>> Character>>isLiteral ^true >>> Integer>>isLiteral ^true >>> String>>isLiteral ^true >>> UndefinedObject>>isLiteral ^true >>> >>> ByteArray>>isLiteral ^self class == ByteArray >>> Float>>isLiteral ^self isFinite "^(self - self) = 0.0" >>> ScaledDecimal>>isLiteral ^denominator = 1 or: [(10 raisedTo: >>> scale)\\denominator = 0] >>> >>> Array>>isLiteral ^self class == Array and: [self allSatisfy: [:each | >>> each isLiteral]] >>> >>> ...I find most are very basic (might even say deterministic), with the >>> recursion of Array>>isLiteral seeming an annomaly. Also, the big IF >>> condition in Array>>printOn: smells like a design decision being made at >>> runtime (Valloud AMCOS p12). >>> >>> Array>>printOn: aStream >>> self shouldBePrintedAsLiteral ifTrue: [self printAsLiteralFormOn: >>> aStream. ^ self]. >>> self isSelfEvaluating ifTrue: [self printAsSelfEvaluatingFormOn: >>> aStream. ^ self]. >>> super printOn: aStream >>> >>> Flipping between two printString formats seems like selecting between >>> two class types. Indeed, if we had a LiteralArray class, there would be no >>> need for its printOn: to recursively search to determine its form, thus >>> allowing #printStringLimitedTo: to do its thing to protect against infinite >>> recursion. >>> >>> Also, instead of a recursive Array>>isLiteral we'd have something like >>> LiteralArray>>isLiteral ^true >>> Array>>isLiteral ^false >>> which seems to align much better with the pattern of the other >>> #isLiteral implementors. >>> >>> I notice there is both RBArrayNode and RBLiteralArrayNode. >>> >>> So what are the wider concerns that might apply? >>> (In particular, I'm not sure how the #isSelfEvaluating (which is also >>> recursive) fits into the big picture) >>> >>> cheers -ben >>> >>> [1] https://www.mail-archive.com/pharo-dev@lists.pharo.org/msg25156.html >>> >> >> > > > -- > best, > Eliot >

Feb. 5, 2015

Re: [Pharo-dev] [CI] help needed: integration of Fuel changes with new version number

by Marcus Denker

> On 05 Feb 2015, at 11:35, Max Leske <maxleske(a)gmail.com> wrote: > > Hi guys > > ClÃ©ment asked us a while ago to update Fuel for Spur support and we did (https://pharo.fogbugz.com/f/cases/14688/Change-CompiledMethod-header-serial… <https://pharo.fogbugz.com/f/cases/14688/Change-CompiledMethod-header-serial…>). The problem now: Fuel is an integral part of the integration process and the change to the Fuel version number means that the automatic integration will not work. Hereâs why: > > This slice can't be automatically integrated because I changed the Fuel version number. Now, when the image worker reads the result from the validation, Fuel raises an exception because of the different version numbers in the parent and child image. > > We should probably add a mechanism that ensures that the original Fuel version is reloaded after the validation, such that the Fuel file is written by the same version that then reads it in the parent. > > Can someone please manually integrate or (which would be way cooler) adapt the integration process such that future updates to Fuel can be automatically integrated? > Ok, I have put it on Fix To Include (after checking that it merges well) and will include it with the next update. Marcus

Feb. 5, 2015

Re: [Pharo-dev] [CI] is integrator actually integrating?

by Max Leske

> On 05 Feb 2015, at 15:11, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > >> On 05 Feb 2015, at 15:08, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >> Iâm subscribed to a couple of resolved (and validated) issues on fogbugz which donât seem to get integrated, although the integrator runs once a day. Is something going wrong there? >> >> Issues to integrate: >> >> https://pharo.fogbugz.com/f/cases/14669/Delay-refactoring-part-2a-avoid-UI-… <https://pharo.fogbugz.com/f/cases/14669/Delay-refactoring-part-2a-avoid-UI-…> >> https://pharo.fogbugz.com/f/cases/14064/Improvements-to-checking-terminatio… <https://pharo.fogbugz.com/f/cases/14064/Improvements-to-checking-terminatio…> >> > > > It is not an automatic process. > > 1) that a change passes the monkey does not mean that it is good. It always needs human review > â> if that happens, we put it on âfix to includeâ > > 2) even issues on âfix to includedâ are integrated by a human. The tool is quite good, though (just a couple of kicks) > > Keep in mind that not many people review issuesâ¦ > > Marcus > Oops! Both of those have been reviewed by humans. Iâll put them on âfix to includeâ (damn, Iâm sure Iâve asked the same question beforeâ¦ sorry). Thanks Marcus

Feb. 5, 2015