- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Confused about #isDirectory

by Sean P. DeNigris

Sean P. DeNigris wrote > > The changed snippet is... > Issue 79: primitiveDirectoryLookup returns wrong dir flag http://code.google.com/p/cog/issues/detail?id=79 -- View this message in context: http://forum.world.st/Confused-about-isDirectory-tp4589441p4590357.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] Confused about #isDirectory

by Sean P. DeNigris

Should have been... Sean P. DeNigris wrote > > Removing the call returns the correct value (true) for "'/private/tmp' > asFileReference isDirectory" > -- View this message in context: http://forum.world.st/Confused-about-isDirectory-tp4589441p4590346.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] Confused about #isDirectory

by Sean P. DeNigris

Sean P. DeNigris wrote > > For /private/var, <primitive: 'primitiveDirectoryLookup' module: > 'FilePlugin'> returns false for the dir flag... still investigating... > sqSqueakOSXFileDirectoryInterface-resolvedAliasFiles: calls stringByStandardizingPath on the filePath, which the manual [1] states may "Remove an initial component of â/privateâ from the path if the result still indicates an existing file or directory (checked by consulting the file system).". This returns /var for /private/var, which is a symlink. Removing the call returns the correct value (true) for "". Also, all Tests-Files and Filesystem-Tests pass in 1.4. The changed snippet is (I also fixed the misspelled variable "compoents" to "components"): - (NSString *)resolvedAliasFiles:(NSString *)filePath { NSArray *components = [filePath pathComponents]; // removed stringByStandardizingPath call NSString *thisComponent; NSString *path = [[NSString new] autorelease]; for (thisComponent in components) { [1] https://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foun… -- View this message in context: http://forum.world.st/Confused-about-isDirectory-tp4589441p4590343.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

April 26, 2012

Re: [Pharo-project] Pharo and the rest of the ecosystem

by Casimiro de Almeida Barreto

On 26-04-2012 11:31, Sean P. DeNigris wrote: > > Yes this is the key! It's not "let's build all these great frameworks on top > of the system", but "we need a clean, beautiful system that people can use > and explore and build on without tearing their hair out in frustration" [1] > and some of the components are so messy that the only clear path forward is > replacement. This is not a criticism of Squeak. Many parts of Squeak (and > Squeak itself was intended as just a "tool to think" about the next > evolution) were spikes, not real implementations but experiments. So the > usual advice that it's better to clean than to rewrite does not apply. > Proofs of concept are meant to be thrown away! Yes, that's true: some times replacements are necessary. Thing is that historical smalltalk distros were not devised in modular way. Particularly they didn't have the idea of core system and foundation classes and that lead to a development model that depended on non orthogonal stuff (meaning: at some point it got difficult to insulate causes of trouble and fixing one thing lead to problems in other things). Situation was further complicated by the fact that sometimes problems found in certain areas were "solved" by building new packages to deal with them and these new packages should live side by side with old ones. In terms of maintenance it is pretty like hell. The efforts of generating minimal images (cores) are extremely healthy: to do so it is necessary to eliminate much undesired inter-dependencies. Besides people are minding about the real necessity of having different things to deal with the same stuff (like file and directory management). Another aspect of it is that people are enhancing code where possible or just replacing things that are fubar. I guess that one thing for immediate future of an open source "industry grade" smalltalk is to define foundation classes to deal with core functionality (file and directory management, (inter)networking, multimedia, interfacing with OS, user interface (including seamless use of full screen display and OpenGL), etc, etc, etc. IMO, interfacing with OS and foreign libraries is something that must be seen "with love" because FFI is not well behaved and as VMs were designed for past generation x86-32bit (mono-core single-thread) architectures. There is a lot to do in terms of porting things to the new 64bit multi-core multi-thread hardware architectures. I remember that around 2010 there were hot discussions about having multi-core multi-thread support in VM but now things are quite silent. Also, the last things I saw about 64bit images date from 2010. > > Each of these components brings us closer to a clean, beautiful, > Smalltalk-inspired system "and users are expected to build on that > directly". The Smalltalk-inspired part is crucial also. Until I got that, I > was plagued by questions of interoperability and backward-compatibility > (which are obviously still nice, but subordinate to evolution). > > I see Squeak and Pharo's goals aligning more and more and, sure, it is > frustrating to duplicate efforts. Although, maybe we'd all be complacent > without the friendly rivalry. As the Zen master said, "we'll see..." [2] > > [1] I can't imagine someone putting FileDirectory and friends through > serious use and not praying for something like Filesystem. And a programming > environment without built-in support for https?? Zodiac is a fundamental > piece of the system. These things are not "buzz", but core components that > let us get back to work and play, instead of beating our heads against > soul-crushing constructs like "FileDirectory extensionFor: file name" > instead of "file extension" > > [2] A boy is given a horse on his 14th birthday. Everyone in the village > says, âOh how wonderful.â But a Zen master who lives in the village says, > âWe'll see.â 'The boy falls off the horse and breaks his foot. Everyone in > the village says, âOh how awful.â The Zen master says, âWe'll see.â The > village is thrown into war and all the young men have to go to war. But, > because of the broken foot, the boy stays behind. Everyone says, âOh, how > wonderful.â The Zen master says, âWe'll see.â > > Cheers, > Sean > > -- > View this message in context: http://forum.world.st/Pharo-and-the-rest-of-the-ecosystem-tp4582839p4590003… > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > > Best regards, CdAB

April 26, 2012

Re: [Pharo-project] VNC for pharo 1.3 or 1.4

by Norbert Hartl

Am 26.04.2012 um 17:21 schrieb Hernan Wilkinson: > Hi > does anybody ported the implementation of VNC to pharo 1.3 or 1.4? > If so, could you share it? :-) > I use it daily on pharo 1.3. I load it via metacello. My baseline entry looks like this project: 'RFB' with: [ spec className: 'ConfigurationOfRFB'; file: 'ConfigurationOfRFB'; versionString: #'bleedingEdge'; repository: 'http://www.squeaksource.com/MetacelloRepository' ]; Norbert

April 26, 2012

[Pharo-project] VNC for pharo 1.3 or 1.4

by Hernan Wilkinson

Hi does anybody ported the implementation of VNC to pharo 1.3 or 1.4? If so, could you share it? :-) Thanks! Hernan. -- *HernÃ¡n Wilkinson Agile Software Development, Teaching & Coaching* *Phone: +54 - 011 - 4311 - 8404** Mobile: +54 - 911 - 4470 - 7207 email: hernan.wilkinson(a)10Pines.com site: http://www.10Pines.com <http://www.10pines.com/>* Address: Paraguay 523, Floor 7 N, Buenos Aires, Argentina

April 26, 2012

Re: [Pharo-project] Do we have the new primitive?? [WAS] Re: IdentitySet but using #hash rather than #identityHash ?

by Mariano Martinez Peck

> >>> Hi Eliot/Levente. What is the status of this? Do we have already the new >>> primitive? If true, how can we adapt LargeIdentitySet to use such new >>> primitive? >>> >> >> AFAIK the new primitive is not implemented yet. Adding the primitive to >> the interpreter VM is very easy, but it seems to be a lot more complicated >> (to me) to add it to Cog, because the receiver can be a MethodContext which >> needs special handling. >> I'll rewrite both LargeIdentitySet and LargeIdentityDictionary when the >> primitive is ready. >> > > Thanks Levente. So we should wait Eliot. > I will ping again in a couple of weeks/months ;) > > Ping. So I did it :) Eliot if you tell us what it is needed maybe I can push Esteban or Igor (or me?) to do it ;) Thanks! > > >> >> >> Levente >> >> >> >>> Thanks! >>> >>> >>> >>> >>> >>> >>> >>> >>> Levente >>> >>> >>> thanks >>> >>> On Fri, Dec 16, 2011 at 3:28 PM, Levente Uzonyi < >>> leves(a)elte.hu> wrote: >>> >>> On Fri, 16 Dec 2011, Henrik Sperre Johansen wrote: >>> >>> On 16.12.2011 03:26, Levente Uzonyi wrote: >>> >>> >>> How about my numbers? :) >>> >>> "Preallocate objects, so we won't count gc time." >>> n := 1000000. >>> objects := Array new: n streamContents: [ :stream | >>> n timesRepeat: [ stream nextPut: Object new ] ]. >>> >>> set := IdentitySet new: n. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "4949" >>> >>> set := LargeIdentitySet new. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "331" >>> >>> set := (PluggableSet new: n) >>> hashBlock: [ :object | object identityHash * 4096 + object >>> class >>> identityHash * 64 ]; "Change this to #basicIdentityHash in >>> Pharo" >>> equalBlock: [ :a :b | a == b ]; >>> yourself. >>> Smalltalk garbageCollect. >>> [1 to: n do: [ :i | set add: (objects at: i) ] ] timeToRun. >>> "5511" >>> >>> >>> I also have a LargeIdentityDictionary, which is relatively >>> fast, but not >>> as fast as LargeIdentitySet, because (for some unknown >>> reason) we don't >>> have a primitive that could support it. If we had a primitive >>> like >>> primitive 132 which would return the index of the element if >>> found or 0 if >>> not, then we could have a really fast LargeIdentityDictionary. >>> >>> >>> Levente >>> >>> Hehe yes, if writing a version fully exploiting the limited range, >>> that's >>> probably the approach I would go for as well. >>> (IAssuming it's the version at http://leves.web.elte.hu/** >>> squeak/LargeIdentitySet.st<htt**p://leves.web.elte.hu/squeak/** >>> LargeIdentitySet.st<http://leves.web.elte.hu/squeak/LargeIdentitySet.st> >>> > >>> ) >>> >>> Mariano commented in the version at http://www.squeaksource.com/** >>> FuelExperiments <http://www.squeaksource.com/**FuelExperiments<http://www.squeaksource.com/FuelExperiments>> >>> that it's >>> slow for them, which I guess is due to not adopting #identityHash calls >>> to >>> #basicIdentityHash calls for Pharo: >>> ((0 to: 4095) collect: [:each | each << 22 \\ 4096 ]) asSet size -> 1 >>> So it basically uses 1 bucket instead of 4096... Whoops. :) >>> >>> Uploaded a new version to the MC repository which is adapted for Pharo, >>> on the same machine my numbers were taken from, it does the same test as >>> I >>> used above in 871 ms. (181 with preallocation). >>> >>> >>> Cool. One more thing: in Squeak the method using primitive 132 directly >>> was renamed to #instVarsInclude:, so now #pointsTo: works as expected. If >>> this was also added to Pharo, then the #pointsTo: sends should be changed >>> to #instVarsInclude:, otherwise Array can be reported as included even if >>> it wasn't added. >>> I'll upload my LargeIdentityDictionary implementation to the same place >>> this evening, since it's still 2-3 factor faster than other solutionts >>> and >>> there seem to be demand for it. >>> >>> >>> Levente >>> >>> >>> Cheers, >>> Henry >>> >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.**com <http://marianopeck.wordpress.com> >>> >>> >>> > > > -- > Mariano > http://marianopeck.wordpress.com > > -- Mariano http://marianopeck.wordpress.com

April 26, 2012

Re: [Pharo-project] PetitSmalltalk: now with a self-contained grammar for number literals

by Frank Shearar

On 26 April 2012 16:01, Levente Uzonyi <leves(a)elte.hu> wrote: > On Thu, 26 Apr 2012, Frank Shearar wrote: > >> On 26 April 2012 03:35, Levente Uzonyi <leves(a)elte.hu> wrote: >>> >>> On Wed, 25 Apr 2012, Levente Uzonyi wrote: >>> >>>> On Wed, 25 Apr 2012, Frank Shearar wrote: >>>> >>>>> On 20 April 2012 15:15, Frank Shearar <frank.shearar(a)gmail.com> wrote: >>>>>> >>>>>> >>>>>> On 20 April 2012 03:51, Levente Uzonyi <leves(a)elte.hu> wrote: >>>>>>> >>>>>>> >>>>>>> On Thu, 19 Apr 2012, Frank Shearar wrote: >>>>>>> >>>>>>>> I found a serious bug in parsing numbers with negative exponents. It >>>>>>>> was completey broken, in fact, parsing 1e-1 as 10, not 1 / 10. >>>>>>>> Anyway. >>>>>>>> This version fixes that, and adds a bunch of tests demonstrating >>>>>>>> that >>>>>>>> number parsing will return rationals if it can. >>>>>>>> >>>>>>>> It's significantly slower than Squeak's SqNumberParser: >>>>>>>> >>>>>>>> Time millisecondsToRun: [100000 timesRepeat: [SqNumberParser parse: >>>>>>>> '1234567890']] => 466 >>>>>>>> >>>>>>>> Time millisecondsToRun: [100000 timesRepeat: >>>>>>>> [PPSmalltalkNumberParser >>>>>>>> parse: '1234567890']] => 32082 >>>>>>>> >>>>>>>> I've attached a MessageTally spying on the latter: I've not much >>>>>>>> skill >>>>>>>> in reading these, but nothing leaps out at me as being obviously >>>>>>>> awful. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Didn't check the code, just the tally, and I think that >>>>>>> PPSmalltalkNumberParser(PPSmalltalkNumberGrammar)>>digitsBase: is >>>>>>> begging >>>>>>> for optimization. It's probably also the cause of the high amount of >>>>>>> garbage >>>>>>> which causes significant amount of time spent with garbage >>>>>>> collection. >>>>>>> It's also interesting is that the finalization process does so much >>>>>>> work, >>>>>>> there may be something wrong with your image. >>>>>> >>>>>> >>>>>> >>>>>> Thanks for taking a look, Levente. >>>>>> >>>>>> I'd expect digitsBase: to dominate the running costs, given that we're >>>>>> parsing numbers. >>>>>> >>>>>> I do make a large number of throwaway "immutable" values with a >>>>>> Builder-like pattern... in PPSmalltalkNumberParser >> >>>>>> #makeNumberFrom:base:. That, I would imagine, could explain the >>>>>> garbage? >>>>>> >>>>>> If I may, what do you look for when reading the MessageTally? How do >>>>>> you tell, for instance, that there's excessive garbage production? >>>>>> That the incremental GCs take 7ms? (I'm reading Andreas' comments on >>>>>> http://wiki.squeak.org/squeak/4210 again.) >>>>> >>>>> >>>>> >>>>> Levente, you're quite right: #digitsBase: has now been optimised even >>>>> more, reducing the time taken to run my benchmark >>>>> >>>>> MessageTally spyOn: [Time millisecondsToRun: [100000 timesRepeat: >>>>> [PPSmalltalkNumberParser parse: '1234567890']]] >>>>> >>>>> from ~32 seconds to ~16 seconds. (Memoising was the answer: >>>>> #digitsBase: is effectively a higher-order production and, like OMeta, >>>>> PPCompositeParser doesn't memoise those. A simple class var dictionary >>>>> solves that problem. >>>> >>>> >>>> >>>> Creating a new parser (PPSmalltalkNumberParser) takes quite a while, if >>>> you extract the parser creation, then you'll get closer to the >>>> performance >>>> limit of PetitParser: >>>> >>>> | p | >>>> p := PPSmalltalkNumberParser new. >>>> [ Time millisecondsToRun: [100000 timesRepeat: [p parse: '1234567890']] >>>> ] >>>> timeProfile. "==> 2594" >>>> >>>> There are still places where you create parsers on the fly, like >>>> PPSmalltalkNumberGrammar #>> number. These should be avoided. >>> >>> >>> >>> I hacked the code a bit and got the following results: >>> >>> (1 to: 5) collect: [ :run | >>> >>> Â Â Â Â | p | >>> Â Â Â Â p := PPSmalltalkNumberParser new. >>> Â Â Â Â Smalltalk garbageCollect. >>> Â Â Â Â [ 1 to: 100000 do: [ :i | p parse: '1234567890' ] ] timeToRun ]. >>> >>> #(1198 1209 1225 1221 1204) >>> >>> It's ~6 times slower than SqNumberParser and I don't really think it can >>> get >>> much closer to it. The mcz is available here: >>> http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.62.mcz >> >> >> Ah excellent! Thanks! I see you eagerly initialise the number parsers >> where I'd gone the lazy/memoise route. Any performance implications I >> should be aware of, other than of course that eager initialisation > > > There's only one perdicate parser (instead of two + a choice parser) and a > String is passed as a collection to the PPPredicateParser, so #includes: can > use a fast primitive. > > >> means keeping 36 parsers around when maybe three or four of those will >> see regular use? (decimal, hex, binary, octal) For instance, I guess >> the running cost of Dictionary >> #at:ifAbsentPut: is greater than >> Array >> #at: because the latter just uses a primitive. > > > Lazy initialization of the parsers is also possible, but running the tests > will initialize 35 of them. I wonder if holding the parsers in a class > variable has any side effects. Anyway, I uploaded a new version which has > lazy initialization: > http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.65.mcz Er, yes. That should have been obvious. I feel a bit silly now! In which case there's nothing to decide either way. I'm unhappy using a class-side cache: it means consuming memory forever when you only need (some of) the 35 parsers during parsing. But PPCompositeParser works by initialising ALL instvars to PPDelegateParsers, so it's a bit complicated putting non-parser things in the instvars. It worked when I memoised the original digit parsers (so when I moved DigitParsers -> digitParsers) but broke when I applied the same technique elsewhere. Presumably I was lucky the first time round, and it only worked because of where in the initialisation I initialised the dictionary. If I could browbeat the grammar to use an instvar to hold the digit parsers it would make a lot of sense to switch to lazily instantiating the number parsers: you're likely only to instantiate at most four of them at a time, unless you're being very tricky with your arithmetic. frank > Levente > > >> >> Lastly, I added PetitSmalltalk-fbs.64 to >> http://ss3.gemstone.com/ss/Scratchpad-fbs/, based on your .62. It just >> normalises how signs work. Why I didn't write the exponent sign as a >> string from the beginning (like I do the number's sign), I don't know. >> At any rate, that normalises NumberMaker's API (it always takes >> Strings) and keeps the cheaper x = '-' rather than #($- '-') includes: >> x, and makes the tests all pass :) >> >> And now I'm going to add a feature request for SS3, asking for a >> RESTful interface to the repositories! >> >> frank >> >>> Levente >>> >>> >>>> >>>> >>>> Levente >>>> >>>> >>>>> >>>>> frank >>>>> >>>>>> frank >>>>>> >>>>>>> Levente >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> frank >>>>>>>> >>>>>>>> On 14 September 2011 20:26, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 3 September 2011 19:35, Nicolas Cellier >>>>>>>>> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2011/9/3 Frank Shearar <frank.shearar(a)gmail.com>: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3 September 2011 18:50, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> I think it is a good idea to have the number parser separate, >>>>>>>>>>>> after >>>>>>>>>>>> all it might also make sense to use it separately. >>>>>>>>>>>> >>>>>>>>>>>> It seems that the new Smalltalk grammar is significantly slower. >>>>>>>>>>>> The >>>>>>>>>>>> benchmark PPSmalltalkClassesTests class>>#benchmark: that uses >>>>>>>>>>>> the >>>>>>>>>>>> source code of the collection hierarchy and does not especially >>>>>>>>>>>> target >>>>>>>>>>>> number literals runs 30% slower. >>>>>>>>>>>> >>>>>>>>>>>> Also I see that "Number readFrom: ..." is still used within the >>>>>>>>>>>> grammar. This seems to be a bit strange, no? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Yes: it's a double-parse, which is a bit lame. First, we parse >>>>>>>>>>> the >>>>>>>>>>> literal with PPSmalltalkNumberParser, which ensures that the >>>>>>>>>>> thing >>>>>>>>>>> given to Number class >> #readFrom: is a well-formed token (so, >>>>>>>>>>> in >>>>>>>>>>> particular, Squeak's Number doesn't get to see anything other >>>>>>>>>>> than >>>>>>>>>>> a >>>>>>>>>>> well-formed token). >>>>>>>>>>> >>>>>>>>>>> It sounds like you're happy with the basic concept, so maybe I >>>>>>>>>>> should >>>>>>>>>>> remove the Number class >> #readFrom: stuff, see if I can't >>>>>>>>>>> remove >>>>>>>>>>> the >>>>>>>>>>> performance issues, and resubmit the patch. >>>>>>>>>>> >>>>>>>>>>> frank >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes, a NumberParser is essentially parsing, and this duplication >>>>>>>>>> sounds >>>>>>>>>> useless. >>>>>>>>>> The main feature of interest in NumberParser that I consider a >>>>>>>>>> requirement and should find its equivalence in a PetitNumberParser >>>>>>>>>> is: >>>>>>>>>> - round a decimal representation to nearest Float >>>>>>>>>> It's simple, just convert a Fraction asFloat in a single final >>>>>>>>>> step >>>>>>>>>> to >>>>>>>>>> avoid cumulating round off errors - see >>>>>>>>>> #makeFloatFromMantissa:exponent:base: >>>>>>>>>> >>>>>>>>>> The second feature of interest in NumberParser is the ability to >>>>>>>>>> parser LargeInteger efficiently by avoiding (10 * largeValue + >>>>>>>>>> digitValue) loops, and replacing them with a log(n) cost. >>>>>>>>>> This would be a simple thing to implement in a functional >>>>>>>>>> language. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Hopefully this won't offend your sensibilities too much :). It >>>>>>>>> does, >>>>>>>>> in fact, use 10* loops - I wrote an experimental "front half * rear >>>>>>>>> half" recursion, which was slower in my benchmarks. >>>>>>>>> >>>>>>>>> This version has the grammar and parser doing no string->number >>>>>>>>> conversion at all. PPSmalltalkNumberMaker supplies a number of >>>>>>>>> utility >>>>>>>>> methods designed to stop one from making malformed numbers. It also >>>>>>>>> supplies a builder interface that the parser uses to construct >>>>>>>>> numbers. >>>>>>>>> >>>>>>>>> frank >>>>>>>>> >>>>>>>>>> Nicolas >>>>>>>>>> >>>>>>>>>>>> Lukas >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3 September 2011 17:18, Frank Shearar >>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3 September 2011 15:56, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 3 September 2011 16:51, Frank Shearar >>>>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Lukas, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I haven't :) mainly because I'm unsure where to put it - is >>>>>>>>>>>>>>> there >>>>>>>>>>>>>>> perhaps a PP Inbox, or shall I just post the merged version, >>>>>>>>>>>>>>> or >>>>>>>>>>>>>>> what's >>>>>>>>>>>>>>> your preference? (How about an mcd between my merge and PP's >>>>>>>>>>>>>>> head?) >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Just put the .mcz at some public URL (dropbox, squeak source, >>>>>>>>>>>>>> ...) >>>>>>>>>>>>>> or >>>>>>>>>>>>>> attach it to a mail. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Ah, great - here it is. You'll see I've written the grammar as >>>>>>>>>>>>> a >>>>>>>>>>>>> separate class. That was really more to make what I'd done more >>>>>>>>>>>>> obvious and to minimise the change to PPSmalltalkGrammar, but >>>>>>>>>>>>> perhaps >>>>>>>>>>>>> it's not a bad idea anyway: it's easy to see the number literal >>>>>>>>>>>>> subgrammar. >>>>>>>>>>>>> >>>>>>>>>>>>> frank >>>>>>>>>>>>> >>>>>>>>>>>>>> Lukas >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>> >>>> >>> >> >

April 26, 2012

Re: [Pharo-project] PetitSmalltalk: now with a self-contained grammar for number literals

by Levente Uzonyi

On Thu, 26 Apr 2012, Frank Shearar wrote: > On 26 April 2012 03:35, Levente Uzonyi <leves(a)elte.hu> wrote: >> On Wed, 25 Apr 2012, Levente Uzonyi wrote: >> >>> On Wed, 25 Apr 2012, Frank Shearar wrote: >>> >>>> On 20 April 2012 15:15, Frank Shearar <frank.shearar(a)gmail.com> wrote: >>>>> >>>>> On 20 April 2012 03:51, Levente Uzonyi <leves(a)elte.hu> wrote: >>>>>> >>>>>> On Thu, 19 Apr 2012, Frank Shearar wrote: >>>>>> >>>>>>> I found a serious bug in parsing numbers with negative exponents. It >>>>>>> was completey broken, in fact, parsing 1e-1 as 10, not 1 / 10. Anyway. >>>>>>> This version fixes that, and adds a bunch of tests demonstrating that >>>>>>> number parsing will return rationals if it can. >>>>>>> >>>>>>> It's significantly slower than Squeak's SqNumberParser: >>>>>>> >>>>>>> Time millisecondsToRun: [100000 timesRepeat: [SqNumberParser parse: >>>>>>> '1234567890']] => 466 >>>>>>> >>>>>>> Time millisecondsToRun: [100000 timesRepeat: [PPSmalltalkNumberParser >>>>>>> parse: '1234567890']] => 32082 >>>>>>> >>>>>>> I've attached a MessageTally spying on the latter: I've not much skill >>>>>>> in reading these, but nothing leaps out at me as being obviously >>>>>>> awful. >>>>>> >>>>>> >>>>>> >>>>>> Didn't check the code, just the tally, and I think that >>>>>> PPSmalltalkNumberParser(PPSmalltalkNumberGrammar)>>digitsBase: is >>>>>> begging >>>>>> for optimization. It's probably also the cause of the high amount of >>>>>> garbage >>>>>> which causes significant amount of time spent with garbage collection. >>>>>> It's also interesting is that the finalization process does so much >>>>>> work, >>>>>> there may be something wrong with your image. >>>>> >>>>> >>>>> Thanks for taking a look, Levente. >>>>> >>>>> I'd expect digitsBase: to dominate the running costs, given that we're >>>>> parsing numbers. >>>>> >>>>> I do make a large number of throwaway "immutable" values with a >>>>> Builder-like pattern... in PPSmalltalkNumberParser >> >>>>> #makeNumberFrom:base:. That, I would imagine, could explain the >>>>> garbage? >>>>> >>>>> If I may, what do you look for when reading the MessageTally? How do >>>>> you tell, for instance, that there's excessive garbage production? >>>>> That the incremental GCs take 7ms? (I'm reading Andreas' comments on >>>>> http://wiki.squeak.org/squeak/4210 again.) >>>> >>>> >>>> Levente, you're quite right: #digitsBase: has now been optimised even >>>> more, reducing the time taken to run my benchmark >>>> >>>> MessageTally spyOn: [Time millisecondsToRun: [100000 timesRepeat: >>>> [PPSmalltalkNumberParser parse: '1234567890']]] >>>> >>>> from ~32 seconds to ~16 seconds. (Memoising was the answer: >>>> #digitsBase: is effectively a higher-order production and, like OMeta, >>>> PPCompositeParser doesn't memoise those. A simple class var dictionary >>>> solves that problem. >>> >>> >>> Creating a new parser (PPSmalltalkNumberParser) takes quite a while, if >>> you extract the parser creation, then you'll get closer to the performance >>> limit of PetitParser: >>> >>> | p | >>> p := PPSmalltalkNumberParser new. >>> [ Time millisecondsToRun: [100000 timesRepeat: [p parse: '1234567890']] ] >>> timeProfile. "==> 2594" >>> >>> There are still places where you create parsers on the fly, like >>> PPSmalltalkNumberGrammar #>> number. These should be avoided. >> >> >> I hacked the code a bit and got the following results: >> >> (1 to: 5) collect: [ :run | >> >> Â Â Â Â | p | >> Â Â Â Â p := PPSmalltalkNumberParser new. >> Â Â Â Â Smalltalk garbageCollect. >> Â Â Â Â [ 1 to: 100000 do: [ :i | p parse: '1234567890' ] ] timeToRun ]. >> >> #(1198 1209 1225 1221 1204) >> >> It's ~6 times slower than SqNumberParser and I don't really think it can get >> much closer to it. The mcz is available here: >> http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.62.mcz > > Ah excellent! Thanks! I see you eagerly initialise the number parsers > where I'd gone the lazy/memoise route. Any performance implications I > should be aware of, other than of course that eager initialisation There's only one perdicate parser (instead of two + a choice parser) and a String is passed as a collection to the PPPredicateParser, so #includes: can use a fast primitive. > means keeping 36 parsers around when maybe three or four of those will > see regular use? (decimal, hex, binary, octal) For instance, I guess > the running cost of Dictionary >> #at:ifAbsentPut: is greater than > Array >> #at: because the latter just uses a primitive. Lazy initialization of the parsers is also possible, but running the tests will initialize 35 of them. I wonder if holding the parsers in a class variable has any side effects. Anyway, I uploaded a new version which has lazy initialization: http://leves.web.elte.hu/squeak/PetitSmalltalk-ul.65.mcz Levente > > Lastly, I added PetitSmalltalk-fbs.64 to > http://ss3.gemstone.com/ss/Scratchpad-fbs/, based on your .62. It just > normalises how signs work. Why I didn't write the exponent sign as a > string from the beginning (like I do the number's sign), I don't know. > At any rate, that normalises NumberMaker's API (it always takes > Strings) and keeps the cheaper x = '-' rather than #($- '-') includes: > x, and makes the tests all pass :) > > And now I'm going to add a feature request for SS3, asking for a > RESTful interface to the repositories! > > frank > >> Levente >> >> >>> >>> >>> Levente >>> >>> >>>> >>>> frank >>>> >>>>> frank >>>>> >>>>>> Levente >>>>>> >>>>>> >>>>>>> >>>>>>> frank >>>>>>> >>>>>>> On 14 September 2011 20:26, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> On 3 September 2011 19:35, Nicolas Cellier >>>>>>>> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> 2011/9/3 Frank Shearar <frank.shearar(a)gmail.com>: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 3 September 2011 18:50, Lukas Renggli <renggli(a)gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> I think it is a good idea to have the number parser separate, >>>>>>>>>>> after >>>>>>>>>>> all it might also make sense to use it separately. >>>>>>>>>>> >>>>>>>>>>> It seems that the new Smalltalk grammar is significantly slower. >>>>>>>>>>> The >>>>>>>>>>> benchmark PPSmalltalkClassesTests class>>#benchmark: that uses the >>>>>>>>>>> source code of the collection hierarchy and does not especially >>>>>>>>>>> target >>>>>>>>>>> number literals runs 30% slower. >>>>>>>>>>> >>>>>>>>>>> Also I see that "Number readFrom: ..." is still used within the >>>>>>>>>>> grammar. This seems to be a bit strange, no? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Yes: it's a double-parse, which is a bit lame. First, we parse the >>>>>>>>>> literal with PPSmalltalkNumberParser, which ensures that the thing >>>>>>>>>> given to Number class >> #readFrom: is a well-formed token (so, in >>>>>>>>>> particular, Squeak's Number doesn't get to see anything other than >>>>>>>>>> a >>>>>>>>>> well-formed token). >>>>>>>>>> >>>>>>>>>> It sounds like you're happy with the basic concept, so maybe I >>>>>>>>>> should >>>>>>>>>> remove the Number class >> #readFrom: stuff, see if I can't remove >>>>>>>>>> the >>>>>>>>>> performance issues, and resubmit the patch. >>>>>>>>>> >>>>>>>>>> frank >>>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, a NumberParser is essentially parsing, and this duplication >>>>>>>>> sounds >>>>>>>>> useless. >>>>>>>>> The main feature of interest in NumberParser that I consider a >>>>>>>>> requirement and should find its equivalence in a PetitNumberParser >>>>>>>>> is: >>>>>>>>> - round a decimal representation to nearest Float >>>>>>>>> It's simple, just convert a Fraction asFloat in a single final step >>>>>>>>> to >>>>>>>>> avoid cumulating round off errors - see >>>>>>>>> #makeFloatFromMantissa:exponent:base: >>>>>>>>> >>>>>>>>> The second feature of interest in NumberParser is the ability to >>>>>>>>> parser LargeInteger efficiently by avoiding (10 * largeValue + >>>>>>>>> digitValue) loops, and replacing them with a log(n) cost. >>>>>>>>> This would be a simple thing to implement in a functional language. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Hopefully this won't offend your sensibilities too much :). It does, >>>>>>>> in fact, use 10* loops - I wrote an experimental "front half * rear >>>>>>>> half" recursion, which was slower in my benchmarks. >>>>>>>> >>>>>>>> This version has the grammar and parser doing no string->number >>>>>>>> conversion at all. PPSmalltalkNumberMaker supplies a number of >>>>>>>> utility >>>>>>>> methods designed to stop one from making malformed numbers. It also >>>>>>>> supplies a builder interface that the parser uses to construct >>>>>>>> numbers. >>>>>>>> >>>>>>>> frank >>>>>>>> >>>>>>>>> Nicolas >>>>>>>>> >>>>>>>>>>> Lukas >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 3 September 2011 17:18, Frank Shearar <frank.shearar(a)gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 3 September 2011 15:56, Lukas Renggli <renggli(a)gmail.com> >>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 3 September 2011 16:51, Frank Shearar >>>>>>>>>>>>> <frank.shearar(a)gmail.com> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Lukas, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I haven't :) mainly because I'm unsure where to put it - is >>>>>>>>>>>>>> there >>>>>>>>>>>>>> perhaps a PP Inbox, or shall I just post the merged version, or >>>>>>>>>>>>>> what's >>>>>>>>>>>>>> your preference? (How about an mcd between my merge and PP's >>>>>>>>>>>>>> head?) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Just put the .mcz at some public URL (dropbox, squeak source, >>>>>>>>>>>>> ...) >>>>>>>>>>>>> or >>>>>>>>>>>>> attach it to a mail. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Ah, great - here it is. You'll see I've written the grammar as a >>>>>>>>>>>> separate class. That was really more to make what I'd done more >>>>>>>>>>>> obvious and to minimise the change to PPSmalltalkGrammar, but >>>>>>>>>>>> perhaps >>>>>>>>>>>> it's not a bad idea anyway: it's easy to see the number literal >>>>>>>>>>>> subgrammar. >>>>>>>>>>>> >>>>>>>>>>>> frank >>>>>>>>>>>> >>>>>>>>>>>>> Lukas >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Lukas Renggli >>>>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Lukas Renggli >>>>>>>>>>> www.lukas-renggli.ch >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>> >>> >>> >> > >

April 26, 2012

Re: [Pharo-project] Downloading a file

by Sven Van Caekenberghe

Sean, On 26 Apr 2012, at 02:23, Sean P. DeNigris wrote: > Sven Van Caekenberghe wrote >> >> Now, you are right that there should be some convenience method to make >> that easier. I'll think about it for a bit and let you know if I can come >> up with something. >> > > What about: > download: urlString to: anFSReference > "anFSReference can be either: > - a directory, where the file will be saved under its original name > - otherwise, the file will be saved as the FSReference" > > | target entity | > target := anFSReference isDirectory > ifTrue: [ anFSReference / urlString asUrl path last ] > ifFalse: [ anFSReference ]. > > entity := ZnClient new > streaming: true; > get: urlString; > entity. > target fileStreamDo: [ :stream | > stream binary. > entity writeOn: stream ]. > > This would allow really easy, flexible use like: > "Save in this directory as same name" > ZnClient download: 'http://www.pharo-project.org/images/pharo.png' to: > '/Volumes/Fast/Squeak/Main Working Images/Pharo-1.3' asFile. > > "Save under new name in this exact place" > ZnClient download: 'http://www.pharo-project.org/images/pharo.png' to: > '/Volumes/Fast/Squeak/Main Working Images/Pharo-1.3/new_name.png' asFile. I implemented ZnClient>>#downloadTo: which accepts both a directory or a filename path string. (Zinc-HTTP-SvenVanCaekenberghe.265 and Zinc-Tests-SvenVanCaekenberghe.136). ZnClient new systemPolicy; accept: ZnMimeType text; url: 'http://zn.stfx.eu/zn/numbers.txt'; downloadTo: '/Users/sven/Desktop'. I temporarily rolled back my use of FileSystem due to http://code.google.com/p/pharo/issues/detail?id=5693 and the fact that I had to hack too much to maintain 1.3 compatibility (you can still see the code in a previous version, Zinc-HTTP-SvenVanCaekenberghe.264). Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

April 26, 2012