April 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Do we have the new primitive?? [WAS] Re: IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

> >>> Hi Eliot/Levente. What is the status of this? Do we have already the new >>> primitive? If true, how can we adapt LargeIdentitySet to use such new >>> primitive? >>> >> >> AFAIK the new primitive is not implemented yet. Adding the primitive to >> the interpreter VM is very easy, but it seems to be a lot more complicated >> (to me) to add it to Cog, because the receiver can be a MethodContext which >> needs special handling. >> I'll rewrite both LargeIdentitySet and LargeIdentityDictionary when the >> primitive is ready. >> > > Thanks Levente. So we should wait Eliot. > I will ping again in a couple of weeks/months ;) > > Ping. So I did it :) Eliot if you tell us what it is needed maybe I can push Esteban or Igor (or me?) to do it ;) Thanks! > > >> >> >> Levente >> >> >> >>> Thanks! >>> >>> >>> >>> >>> >>> >>> >>> >>> Levente >>> >>> >>> thanks >>> >>> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi < >>> leves(a)elte.hu> wrote: >>> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>> >>> How about my numbers? :) >>> >>> "Preallocate objects, so we won't count gc time." >>> n := 1000000. >>> objects := Array new: n streamContents: [ :stream | >>> n timesRepeat: [ stream nextPut: Object new ] ]. >>> >>> set := IdentitySet new: n. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "4949" >>> >>> set := LargeIdentitySet new. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "331" >>> >>> set := (PluggableSet new: n) >>> hashBlock: [ :object | object identityHash * 4096 + object >>> class >>> identityHash * 64 ]; "Change this to #basicIdentityHash in >>> Pharo" >>> equalBlock: [ :a :b | a == b ]; >>> yourself. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "5511" >>> >>> >>> I also have a LargeIdentityDictionary, which is relatively >>> fast, but not >>> as fast as LargeIdentitySet, because (for some unknown >>> reason) we don't >>> have a primitive that could support it. If we had a primitive >>> like >>> primitive 132 which would return the index of the element if >>> found or 0 if >>> not, then we could have a really fast LargeIdentityDictionary. >>> >>> >>> Levente >>> >>> Hehe yes, if writing a version fully exploiting the limited range, >>> that's >>> probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>> > >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls >>> to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as >>> I >>> used above in 871 ms. (181 with preallocation). >>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>> Henry >>> >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >>> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

April 26, 2012

Re: [Pharo-project] PetitSmalltalk: now with a self-contained grammar for number literals

by Frank Shearar

On 26 April 2012 16:01, Levente Uzonyi <leves(a)elte.hu> wrote: > On Thu, 26 Apr 2012, Frank Shearar wrote: > >> On 26 April 2012 03:35, Levente Uzonyi <leves(a)elte.hu> wrote: >>> >>> On Wed, 25 Apr 2012, Levente Uzonyi wrote: >>> >>>> On Wed, 25 Apr 2012, Frank Shearar wrote: >>>> >>>>> On 20 April 2012 15:15, Frank Shearar <frank.shearar(a)gmail.com> wrote: >>>>>> >>>>>> >>>>>> On 20 April 2012 03:51, Levente Uzonyi <leves(a)elte.hu> wrote: >>>>>>> >>>>>>> >>>>>>> On Thu, 19 Apr 2012, Frank Shearar wrote: >>>>>>> >>>>>>>> I found a serious bug in parsing numbers with negative exponents. It >>>>>>>> was completey broken, in fact, parsing 1e-1 as 10, not 1 / 10. >>>>>>>> Anyway. >>>>>>>> This version fixes that, and adds a bunch of tests demonstrating >>>>>>>> that >>>>>>>> number parsing will return rationals if it can. >>>>>>>> >>>>>>>> It's significantly slower than Squeak's SqNumberParser: >>>>>>>> >>>>>>>> Time millisecondsToRun: [100000 timesRepeat: [SqNumberParser parse: >>>>>>>> '1234567890']] => 466 >>>>>>>> >>>>>>>> Time millisecondsToRun: [100000 timesRepeat: >>>>>>>> [PPSmalltalkNumberParser >>>>>>>> parse: '1234567890']] => 32082 >>>>>>>> >>>>>>>> I've attached a MessageTally spying on the latter: I've not much >>>>>>>> skill >>>>>>>> in reading these, but nothing leaps out at me as being obviously >>>>>>>> awful. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Didn't check the code, just the tally, and I think that >>>>>>> PPSmalltalkNumberParser(PPSmalltalkNumberGrammar)>>digitsBase: is >>>>>>> begging >>>>>>> for optimization. It's probably also the cause of the high amount of >>>>>>> garbage >>>>>>> which causes significant amount of time spent with garbage >>>>>>> collection. >>>>>>> It's also interesting is that the finalization process does so much >>>>>>> work, >>>>>>> there may be something wrong with your image. >>>>>> >>>>>> >>>>>> >>>>>> Thanks for taking a look, Levente. >>>>>> >>>>>> I'd expect digitsBase: to dominate the running costs, given that we're >>>>>> parsing numbers. >>>>>> >>>>>> I do make a large number of throwaway "immutable" values with a >>>>>> Builder-like pattern... in PPSmalltalkNumberParser >> >>>>>> #makeNumberFrom:base:. That, I would imagine, could explain the >>>>>> garbage? >>>>>> >>>>>> If I may, what do you look for when reading the MessageTally? How do >>>>>> you tell, for instance, that there's excessive garbage production? >>>>>> That the incremental GCs take 7ms? (I'm reading Andreas' comments on >>>>>> http://wiki.squeak.org/squeak/4210 again.) >>>>> >>>>> >>>>> >>>>> Levente, you're quite right: #digitsBase: has now been optimised even >>>>> more, reducing the time taken to run my benchmark >>>>> >>>>> MessageTally spyOn: [Time millisecondsToRun: [100000 timesRepeat: >>>>> [PPSmalltalkNumberParser parse: '1234567890']]] >>>>> >>>>> from ~32 seconds to ~16 seconds. (Memoising was the answer: >>>>> #digitsBase: is effectively a higher-order production and, like OMeta, >>>>> PPCompositeParser doesn't memoise those. A simple class var dictionary >>>>> solves that problem. >>>> >>>> >>>> >>>> Creating a new parser (PPSmalltalkNumberParser) takes quite a while, if >>>> you extract the parser creation, then you'll get closer to the >>>> performance >>>> limit of PetitParser: >>>> >>>> | p | >>>> p := PPSmalltalkNumberParser new. >>>> [ Time millisecondsToRun: [100000 timesRepeat: [p parse: '1234567890']] >>>> ] >>>> timeProfile. "==> 2594" >>>> >>>> There are still places where you create parsers on the fly, like >>>> PPSmalltalkNumberGrammar #>> number. These should be avoided. >>> >>> >>> >>> I hacked the code a bit and got the following results: >>> >>> (1 to: 5) collect: [ :run | >>> >>> Â Â Â Â | p | >>> Â Â Â Â p := PPSmalltalkNumberParser new. >>> Â Â Â Â Smalltalk garbageCollect. >>> Â Â Â Â [ 1 to: 100000 do: [ :i | p parse: '1234567890' ] ] timeToRun ]. >>> >>> #(1198 1209 1225 1221 1204) >>> >>> It's ~6 times slower than SqNumberParser and I don't really think it can >>> get >>> much closer to it. The mcz is available here: >>> http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.62.mcz >> >> >> Ah excellent! Thanks! I see you eagerly initialise the number parsers >> where I'd gone the lazy/memoise route. Any performance implications I >> should be aware of, other than of course that eager initialisation > > > There's only one perdicate parser (instead of two + a choice parser) and a > String is passed as a collection to the PPPredicateParser, so #includes: can > use a fast primitive. > > >> means keeping 36 parsers around when maybe three or four of those will >> see regular use? (decimal, hex, binary, octal) For instance, I guess >> the running cost of Dictionary >> #at:ifAbsentPut: is greater than >> Array >> #at: because the latter just uses a primitive. > > > Lazy initialization of the parsers is also possible, but running the tests > will initialize 35 of them. I wonder if holding the parsers in a class > variable has any side effects. Anyway, I uploaded a new version which has > lazy initialization: > http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.65.mcz Er, yes. That should have been obvious. I feel a bit silly now! In which case there's nothing to decide either way. I'm unhappy using a class-side cache: it means consuming memory forever when you only need (some of) the 35 parsers during parsing. But PPCompositeParser works by initialising ALL instvars to PPDelegateParsers, so it's a bit complicated putting non-parser things in the instvars. It worked when I memoised the original digit parsers (so when I moved DigitParsers -> digitParsers) but broke when I applied the same technique elsewhere. Presumably I was lucky the first time round, and it only worked because of where in the initialisation I initialised the dictionary. If I could browbeat the grammar to use an instvar to hold the digit parsers it would make a lot of sense to switch to lazily instantiating the number parsers: you're likely only to instantiate at most four of them at a time, unless you're being very tricky with your arithmetic. frank > Levente > > >> >> Lastly, I added PetitSmalltalk-fbs.64 to >> http://ss3.gemstone.com/ss/Scratchpad-fbs/, based on your .62. It just >> normalises how signs work. Why I didn't write the exponent sign as a >> string from the beginning (like I do the number's sign), I don't know. >> At any rate, that normalises NumberMaker's API (it always takes >> Strings) and keeps the cheaper x = '-' rather than #($- '-') includes: >> x, and makes the tests all pass :) >> >> And now I'm going to add a feature request for SS3, asking for a >> RESTful interface to the repositories! >> >> frank >> >>> Levente >>> >>> >>>> >>>> >>>> Levente >>>> >>>> >>>>> >>>>> frank >>>>> >>>>>> frank >>>>>> >>>>>>> Levente >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> frank >>>>>>>> >>>>>>>> On 14 September 2011 20:26, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 3 September 2011 19:35, Nicolas Cellier >>>>>>>>> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2011/9/3 Frank Shearar <frank.shearar(a)gmail.com>: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3 September 2011 18:50, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think it is a good idea to have the number parser separate, >>>>>>>>>>>> after >>>>>>>>>>>> all it might also make sense to use it separately. >>>>>>>>>>>> >>>>>>>>>>>> It seems that the new Smalltalk grammar is significantly slower. >>>>>>>>>>>> The >>>>>>>>>>>> benchmark PPSmalltalkClassesTests class>>#benchmark: that uses >>>>>>>>>>>> the >>>>>>>>>>>> source code of the collection hierarchy and does not especially >>>>>>>>>>>> target >>>>>>>>>>>> number literals runs 30% slower. >>>>>>>>>>>> >>>>>>>>>>>> Also I see that "Number readFrom: ..." is still used within the >>>>>>>>>>>> grammar. This seems to be a bit strange, no? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yes: it's a double-parse, which is a bit lame. First, we parse >>>>>>>>>>> the >>>>>>>>>>> literal with PPSmalltalkNumberParser, which ensures that the >>>>>>>>>>> thing >>>>>>>>>>> given to Number class >> #readFrom: is a well-formed token (so, >>>>>>>>>>> in >>>>>>>>>>> particular, Squeak's Number doesn't get to see anything other >>>>>>>>>>> than >>>>>>>>>>> a >>>>>>>>>>> well-formed token). >>>>>>>>>>> >>>>>>>>>>> It sounds like you're happy with the basic concept, so maybe I >>>>>>>>>>> should >>>>>>>>>>> remove the Number class >> #readFrom: stuff, see if I can't >>>>>>>>>>> remove >>>>>>>>>>> the >>>>>>>>>>> performance issues, and resubmit the patch. >>>>>>>>>>> >>>>>>>>>>> frank >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, a NumberParser is essentially parsing, and this duplication >>>>>>>>>> sounds >>>>>>>>>> useless. >>>>>>>>>> The main feature of interest in NumberParser that I consider a >>>>>>>>>> requirement and should find its equivalence in a PetitNumberParser >>>>>>>>>> is: >>>>>>>>>> - round a decimal representation to nearest Float >>>>>>>>>> It's simple, just convert a Fraction asFloat in a single final >>>>>>>>>> step >>>>>>>>>> to >>>>>>>>>> avoid cumulating round off errors - see >>>>>>>>>> #makeFloatFromMantissa:exponent:base: >>>>>>>>>> >>>>>>>>>> The second feature of interest in NumberParser is the ability to >>>>>>>>>> parser LargeInteger efficiently by avoiding (10 * largeValue + >>>>>>>>>> digitValue) loops, and replacing them with a log(n) cost. >>>>>>>>>> This would be a simple thing to implement in a functional >>>>>>>>>> language. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hopefully this won't offend your sensibilities too much :). It >>>>>>>>> does, >>>>>>>>> in fact, use 10* loops - I wrote an experimental "front half * rear >>>>>>>>> half" recursion, which was slower in my benchmarks. >>>>>>>>> >>>>>>>>> This version has the grammar and parser doing no string->number >>>>>>>>> conversion at all. PPSmalltalkNumberMaker supplies a number of >>>>>>>>> utility >>>>>>>>> methods designed to stop one from making malformed numbers. It also >>>>>>>>> supplies a builder interface that the parser uses to construct >>>>>>>>> numbers. >>>>>>>>> >>>>>>>>> frank >>>>>>>>> >>>>>>>>>> Nicolas >>>>>>>>>> >>>>>>>>>>>> Lukas >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3 September 2011 17:18, Frank Shearar >>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3 September 2011 15:56, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3 September 2011 16:51, Frank Shearar >>>>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Lukas, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I haven't :) mainly because I'm unsure where to put it - is >>>>>>>>>>>>>>> there >>>>>>>>>>>>>>> perhaps a PP Inbox, or shall I just post the merged version, >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> what's >>>>>>>>>>>>>>> your preference? (How about an mcd between my merge and PP's >>>>>>>>>>>>>>> head?) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just put the .mcz at some public URL (dropbox, squeak source, >>>>>>>>>>>>>> ...) >>>>>>>>>>>>>> or >>>>>>>>>>>>>> attach it to a mail. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Ah, great - here it is. You'll see I've written the grammar as >>>>>>>>>>>>> a >>>>>>>>>>>>> separate class. That was really more to make what I'd done more >>>>>>>>>>>>> obvious and to minimise the change to PPSmalltalkGrammar, but >>>>>>>>>>>>> perhaps >>>>>>>>>>>>> it's not a bad idea anyway: it's easy to see the number literal >>>>>>>>>>>>> subgrammar. >>>>>>>>>>>>> >>>>>>>>>>>>> frank >>>>>>>>>>>>> >>>>>>>>>>>>>> Lukas >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>>> >>> >> >

April 26, 2012

Re: [Pharo-project] PetitSmalltalk: now with a self-contained grammar for number literals

by Levente Uzonyi

On Thu, 26 Apr 2012, Frank Shearar wrote: > On 26 April 2012 03:35, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Wed, 25 Apr 2012, Levente Uzonyi wrote: >> >>> On Wed, 25 Apr 2012, Frank Shearar wrote: >>> >>>> On 20 April 2012 15:15, Frank Shearar <frank.shearar(a)gmail.com> wrote: >>>>> >>>>> On 20 April 2012 03:51, Levente Uzonyi <leves(a)elte.hu> wrote: >>>>>> >>>>>> On Thu, 19 Apr 2012, Frank Shearar wrote: >>>>>> >>>>>>> I found a serious bug in parsing numbers with negative exponents. It >>>>>>> was completey broken, in fact, parsing 1e-1 as 10, not 1 / 10. Anyway. >>>>>>> This version fixes that, and adds a bunch of tests demonstrating that >>>>>>> number parsing will return rationals if it can. >>>>>>> >>>>>>> It's significantly slower than Squeak's SqNumberParser: >>>>>>> >>>>>>> Time millisecondsToRun: [100000 timesRepeat: [SqNumberParser parse: >>>>>>> '1234567890']] => 466 >>>>>>> >>>>>>> Time millisecondsToRun: [100000 timesRepeat: [PPSmalltalkNumberParser >>>>>>> parse: '1234567890']] => 32082 >>>>>>> >>>>>>> I've attached a MessageTally spying on the latter: I've not much skill >>>>>>> in reading these, but nothing leaps out at me as being obviously >>>>>>> awful. >>>>>> >>>>>> >>>>>> >>>>>> Didn't check the code, just the tally, and I think that >>>>>> PPSmalltalkNumberParser(PPSmalltalkNumberGrammar)>>digitsBase: is >>>>>> begging >>>>>> for optimization. It's probably also the cause of the high amount of >>>>>> garbage >>>>>> which causes significant amount of time spent with garbage collection. >>>>>> It's also interesting is that the finalization process does so much >>>>>> work, >>>>>> there may be something wrong with your image. >>>>> >>>>> >>>>> Thanks for taking a look, Levente. >>>>> >>>>> I'd expect digitsBase: to dominate the running costs, given that we're >>>>> parsing numbers. >>>>> >>>>> I do make a large number of throwaway "immutable" values with a >>>>> Builder-like pattern... in PPSmalltalkNumberParser >> >>>>> #makeNumberFrom:base:. That, I would imagine, could explain the >>>>> garbage? >>>>> >>>>> If I may, what do you look for when reading the MessageTally? How do >>>>> you tell, for instance, that there's excessive garbage production? >>>>> That the incremental GCs take 7ms? (I'm reading Andreas' comments on >>>>> http://wiki.squeak.org/squeak/4210 again.) >>>> >>>> >>>> Levente, you're quite right: #digitsBase: has now been optimised even >>>> more, reducing the time taken to run my benchmark >>>> >>>> MessageTally spyOn: [Time millisecondsToRun: [100000 timesRepeat: >>>> [PPSmalltalkNumberParser parse: '1234567890']]] >>>> >>>> from ~32 seconds to ~16 seconds. (Memoising was the answer: >>>> #digitsBase: is effectively a higher-order production and, like OMeta, >>>> PPCompositeParser doesn't memoise those. A simple class var dictionary >>>> solves that problem. >>> >>> >>> Creating a new parser (PPSmalltalkNumberParser) takes quite a while, if >>> you extract the parser creation, then you'll get closer to the performance >>> limit of PetitParser: >>> >>> | p | >>> p := PPSmalltalkNumberParser new. >>> [ Time millisecondsToRun: [100000 timesRepeat: [p parse: '1234567890']] ] >>> timeProfile. "==> 2594" >>> >>> There are still places where you create parsers on the fly, like >>> PPSmalltalkNumberGrammar #>> number. These should be avoided. >> >> >> I hacked the code a bit and got the following results: >> >> (1 to: 5) collect: [ :run | >> >> Â Â Â Â | p | >> Â Â Â Â p := PPSmalltalkNumberParser new. >> Â Â Â Â Smalltalk garbageCollect. >> Â Â Â Â [ 1 to: 100000 do: [ :i | p parse: '1234567890' ] ] timeToRun ]. >> >> #(1198 1209 1225 1221 1204) >> >> It's ~6 times slower than SqNumberParser and I don't really think it can get >> much closer to it. The mcz is available here: >> http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.62.mcz > > Ah excellent! Thanks! I see you eagerly initialise the number parsers > where I'd gone the lazy/memoise route. Any performance implications I > should be aware of, other than of course that eager initialisation There's only one perdicate parser (instead of two + a choice parser) and a String is passed as a collection to the PPPredicateParser, so #includes: can use a fast primitive. > means keeping 36 parsers around when maybe three or four of those will > see regular use? (decimal, hex, binary, octal) For instance, I guess > the running cost of Dictionary >> #at:ifAbsentPut: is greater than > Array >> #at: because the latter just uses a primitive. Lazy initialization of the parsers is also possible, but running the tests will initialize 35 of them. I wonder if holding the parsers in a class variable has any side effects. Anyway, I uploaded a new version which has lazy initialization: http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.65.mcz Levente > > Lastly, I added PetitSmalltalk-fbs.64 to > http://ss3.gemstone.com/ss/Scratchpad-fbs/, based on your .62. It just > normalises how signs work. Why I didn't write the exponent sign as a > string from the beginning (like I do the number's sign), I don't know. > At any rate, that normalises NumberMaker's API (it always takes > Strings) and keeps the cheaper x = '-' rather than #($- '-') includes: > x, and makes the tests all pass :) > > And now I'm going to add a feature request for SS3, asking for a > RESTful interface to the repositories! > > frank > >> Levente >> >> >>> >>> >>> Levente >>> >>> >>>> >>>> frank >>>> >>>>> frank >>>>> >>>>>> Levente >>>>>> >>>>>> >>>>>>> >>>>>>> frank >>>>>>> >>>>>>> On 14 September 2011 20:26, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 3 September 2011 19:35, Nicolas Cellier >>>>>>>> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> 2011/9/3 Frank Shearar <frank.shearar(a)gmail.com>: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 3 September 2011 18:50, Lukas Renggli <renggli(a)gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think it is a good idea to have the number parser separate, >>>>>>>>>>> after >>>>>>>>>>> all it might also make sense to use it separately. >>>>>>>>>>> >>>>>>>>>>> It seems that the new Smalltalk grammar is significantly slower. >>>>>>>>>>> The >>>>>>>>>>> benchmark PPSmalltalkClassesTests class>>#benchmark: that uses the >>>>>>>>>>> source code of the collection hierarchy and does not especially >>>>>>>>>>> target >>>>>>>>>>> number literals runs 30% slower. >>>>>>>>>>> >>>>>>>>>>> Also I see that "Number readFrom: ..." is still used within the >>>>>>>>>>> grammar. This seems to be a bit strange, no? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes: it's a double-parse, which is a bit lame. First, we parse the >>>>>>>>>> literal with PPSmalltalkNumberParser, which ensures that the thing >>>>>>>>>> given to Number class >> #readFrom: is a well-formed token (so, in >>>>>>>>>> particular, Squeak's Number doesn't get to see anything other than >>>>>>>>>> a >>>>>>>>>> well-formed token). >>>>>>>>>> >>>>>>>>>> It sounds like you're happy with the basic concept, so maybe I >>>>>>>>>> should >>>>>>>>>> remove the Number class >> #readFrom: stuff, see if I can't remove >>>>>>>>>> the >>>>>>>>>> performance issues, and resubmit the patch. >>>>>>>>>> >>>>>>>>>> frank >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, a NumberParser is essentially parsing, and this duplication >>>>>>>>> sounds >>>>>>>>> useless. >>>>>>>>> The main feature of interest in NumberParser that I consider a >>>>>>>>> requirement and should find its equivalence in a PetitNumberParser >>>>>>>>> is: >>>>>>>>> - round a decimal representation to nearest Float >>>>>>>>> It's simple, just convert a Fraction asFloat in a single final step >>>>>>>>> to >>>>>>>>> avoid cumulating round off errors - see >>>>>>>>> #makeFloatFromMantissa:exponent:base: >>>>>>>>> >>>>>>>>> The second feature of interest in NumberParser is the ability to >>>>>>>>> parser LargeInteger efficiently by avoiding (10 * largeValue + >>>>>>>>> digitValue) loops, and replacing them with a log(n) cost. >>>>>>>>> This would be a simple thing to implement in a functional language. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hopefully this won't offend your sensibilities too much :). It does, >>>>>>>> in fact, use 10* loops - I wrote an experimental "front half * rear >>>>>>>> half" recursion, which was slower in my benchmarks. >>>>>>>> >>>>>>>> This version has the grammar and parser doing no string->number >>>>>>>> conversion at all. PPSmalltalkNumberMaker supplies a number of >>>>>>>> utility >>>>>>>> methods designed to stop one from making malformed numbers. It also >>>>>>>> supplies a builder interface that the parser uses to construct >>>>>>>> numbers. >>>>>>>> >>>>>>>> frank >>>>>>>> >>>>>>>>> Nicolas >>>>>>>>> >>>>>>>>>>> Lukas >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3 September 2011 17:18, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3 September 2011 15:56, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3 September 2011 16:51, Frank Shearar >>>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Lukas, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I haven't :) mainly because I'm unsure where to put it - is >>>>>>>>>>>>>> there >>>>>>>>>>>>>> perhaps a PP Inbox, or shall I just post the merged version, or >>>>>>>>>>>>>> what's >>>>>>>>>>>>>> your preference? (How about an mcd between my merge and PP's >>>>>>>>>>>>>> head?) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Just put the .mcz at some public URL (dropbox, squeak source, >>>>>>>>>>>>> ...) >>>>>>>>>>>>> or >>>>>>>>>>>>> attach it to a mail. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Ah, great - here it is. You'll see I've written the grammar as a >>>>>>>>>>>> separate class. That was really more to make what I'd done more >>>>>>>>>>>> obvious and to minimise the change to PPSmalltalkGrammar, but >>>>>>>>>>>> perhaps >>>>>>>>>>>> it's not a bad idea anyway: it's easy to see the number literal >>>>>>>>>>>> subgrammar. >>>>>>>>>>>> >>>>>>>>>>>> frank >>>>>>>>>>>> >>>>>>>>>>>>> Lukas >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Lukas Renggli >>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >>> >>> >> > >

April 26, 2012

Re: [Pharo-project] Downloading a file

by Sven Van Caekenberghe

Sean, On 26 Apr 2012, at 02:23, Sean P. DeNigris wrote: > Sven Van Caekenberghe wrote >> >> Now, you are right that there should be some convenience method to make >> that easier. I'll think about it for a bit and let you know if I can come >> up with something. >> > > What about: > download: urlString to: anFSReference > "anFSReference can be either: > - a directory, where the file will be saved under its original name > - otherwise, the file will be saved as the FSReference" > > | target entity | > target := anFSReference isDirectory > ifTrue: [ anFSReference / urlString asUrl path last ] > ifFalse: [ anFSReference ]. > > entity := ZnClient new > streaming: true; > get: urlString; > entity. > target fileStreamDo: [ :stream | > stream binary. > entity writeOn: stream ]. > > This would allow really easy, flexible use like: > "Save in this directory as same name" > ZnClient download: 'http://www.pharo-project.org/images/pharo.png' to: > '/Volumes/Fast/Squeak/Main Working Images/Pharo-1.3' asFile. > > "Save under new name in this exact place" > ZnClient download: 'http://www.pharo-project.org/images/pharo.png' to: > '/Volumes/Fast/Squeak/Main Working Images/Pharo-1.3/new_name.png' asFile. I implemented ZnClient>>#downloadTo: which accepts both a directory or a filename path string. (Zinc-HTTP-SvenVanCaekenberghe.265 and Zinc-Tests-SvenVanCaekenberghe.136). ZnClient new systemPolicy; accept: ZnMimeType text; url: 'http://zn.stfx.eu/zn/numbers.txt'; downloadTo: '/Users/sven/Desktop'. I temporarily rolled back my use of FileSystem due to http://code.google.com/p/pharo/issues/detail?id=5693 and the fact that I had to hack too much to maintain 1.3 compatibility (you can still see the code in a previous version, Zinc-HTTP-SvenVanCaekenberghe.264). Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

April 26, 2012

Re: [Pharo-project] Confused about #isDirectory

by Sean P. DeNigris

Camillo Bruni-3 wrote > > no clue though why it should return false for '/private/var'... > For /private/var, <primitive: 'primitiveDirectoryLookup' module: 'FilePlugin'> returns false for the dir flag... still investigating... -- View this message in context: http://forum.world.st/Confused-about-isDirectory-tp4589441p4590063.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] [ANN] ImageWriter, the SystemTracer's son

by Edgar De Cleene

El 4/26/12 10:19 AM, "Guillermo Polito" <guillermopolito(a)gmail.com> escribiÃ³: >> Now it is fixed :).Â Tested it in a clean 2.0 image and worked :). >> Â Guillermo: I like duplicate all steps. Dowloaded a clean all in one 2.0. You have a script for loading and test Hazelnut and all your new work ? Cheers Edgar

April 26, 2012

Re: [Pharo-project] Pharo and the rest of the ecosystem

by Sean P. DeNigris

Chris Muller-3 wrote > generating plenty of marketing "buzz" with frameworks-of-the-month > (such as the ones you listed). In Cuis, Smalltalk _is_ the framework > and users are expected to build on that directly. > Pavel Krivanek-3 wrote > > the planed changes in Pharo are not autotelic but in most cases they > are replacements of current buggy and unmaintainable subsytems. > Yes this is the key! It's not "let's build all these great frameworks on top of the system", but "we need a clean, beautiful system that people can use and explore and build on without tearing their hair out in frustration" [1] and some of the components are so messy that the only clear path forward is replacement. This is not a criticism of Squeak. Many parts of Squeak (and Squeak itself was intended as just a "tool to think" about the next evolution) were spikes, not real implementations but experiments. So the usual advice that it's better to clean than to rewrite does not apply. Proofs of concept are meant to be thrown away! Each of these components brings us closer to a clean, beautiful, Smalltalk-inspired system "and users are expected to build on that directly". The Smalltalk-inspired part is crucial also. Until I got that, I was plagued by questions of interoperability and backward-compatibility (which are obviously still nice, but subordinate to evolution). I see Squeak and Pharo's goals aligning more and more and, sure, it is frustrating to duplicate efforts. Although, maybe we'd all be complacent without the friendly rivalry. As the Zen master said, "we'll see..." [2] [1] I can't imagine someone putting FileDirectory and friends through serious use and not praying for something like Filesystem. And a programming environment without built-in support for https?? Zodiac is a fundamental piece of the system. These things are not "buzz", but core components that let us get back to work and play, instead of beating our heads against soul-crushing constructs like "FileDirectory extensionFor: file name" instead of "file extension" [2] A boy is given a horse on his 14th birthday. Everyone in the village says, âOh how wonderful.â But a Zen master who lives in the village says, âWe'll see.â 'The boy falls off the horse and breaks his foot. Everyone in the village says, âOh how awful.â The Zen master says, âWe'll see.â The village is thrown into war and all the young men have to go to war. But, because of the broken foot, the boy stays behind. Everyone says, âOh, how wonderful.â The Zen master says, âWe'll see.â Cheers, Sean -- View this message in context: http://forum.world.st/Pharo-and-the-rest-of-the-ecosystem-tp4582839p4590003… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] A last 1.3 update? (startup related)

by Sean P. DeNigris

Marcus Denker-4 wrote > >>> Issue 4617: More error tolerant startup >> >> Sounds awesome! I lost many images due to broken startup scripts. In >> fact, >> that's what "inspired" me to hack the VM ;-) I know several others have >> had >> similar issues breaking images... >> > 1.4 has a different solution that is in need of documenation ;-) > Holy crap!!! I just experienced my first startup error on 1.4 and boy was it cool!! Instead of freezing the image and forcing me to debug the vm, it gave me a helpful message and opened a debugger on the error. Thank you. Thank you. Thank you. Sean -- View this message in context: http://forum.world.st/A-last-1-3-update-startup-related-tp4576887p4589924.h… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] How to declare Zinc/FS dependency in Metacello Configuration for 1.4

by Sean P. DeNigris

fstephany wrote > > I'm updating some Metacello configurations and I'm wondering how to > declare the dependencies on... FS > I noticed that the configuration for FS in MetacelloRepository points to the original fs project on SqS. It seems it would be better, when working in 1.3, to load the new Pharo version at http://ss3.gemstone.com/ss/Pharo14. This way, you will not need a compatibility layer to load your code in both 1.3 and 1.4 since e.g. the class names have changed (e.g. FSReference -> FileReference). The Pharo 1.4 FS loads in 1.3 with all green tests. HTH, Sean -- View this message in context: http://forum.world.st/How-to-declare-Zinc-FS-dependency-in-Metacello-Config… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] System browser in Pharo 1.4

by Esteban Lorenzano

Hi, Pharo 1.4 does not have OB by default, which was the browser previous pharo versions had. So... it ships with the old historical browsers, who is very basic and lacks a lot of functionality (not just visuals, also Refactorings, etc.). So... for installing Nautilus (It will be the official Pharo 2.0 browser and is currently in development) or OB, you can open the Configurations Browser, in Menu/Tools/Configurations Browser and choose one of them, and install it. cheers, Esteban On Apr 26, 2012, at 3:43 PM, Olivier Auverlot wrote: > Hi, > > This is probably a stupid question. Why in Pharo 1.4, the system browser does not display classes hierarchically in the column instances / class. Each class is displayed at the same level. It's not easy to read. > > How to get the same behavior as in previous versions of Pharo ? > > Best regards > Olivier

April 26, 2012