- Pharo-dev - lists.pharo.org

Re: [Pharo-project] IRC monthly meeting + project description

by Marcus Denker

On Dec 7, 2010, at 2:43 PM, StÃ©phane Ducasse wrote: > Ok so at 20h00 we get our first IRC meeting. See U there. > I have tickets for a concert tonight... lille orchestra playing la Symphonie fantastique So I can not join. (or maybe afterwards) Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

Dec. 7, 2010

Re: [Pharo-project] Bad characters in update package names

by Gary Chambers

After a bit of digging the following was found: http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx Assuming we are not concerned with allowing ascii 0..31 in alternate data streams (an interesting concept I have used, though rather esoteric) then the following should do: DosFileDirectory>> checkName: aFileName fixErrors: fixing "Check if the file name contains any invalid characters" | fName badChars hasBadChars | fName := super checkName: aFileName fixErrors: fixing. badChars := (#( $: $< $> $| $/ $\ $? $* $"), ((0 to: 31) collect: [:n | n asCharacter])) asSet. hasBadChars := fName includesAnyOf: badChars. (hasBadChars and:[fixing not]) ifTrue:[^self error:'Invalid file name']. hasBadChars ifFalse:[^ fName]. ^ fName collect: [:char | (badChars includes: char) ifTrue:[$#] ifFalse:[char]] Regards, Gary ----- Original Message ----- From: "Gary Chambers" <gazzaguru2(a)btinternet.com> To: <Pharo-project(a)lists.gforge.inria.fr> Sent: Tuesday, December 07, 2010 4:28 PM Subject: Re: [Pharo-project] Bad characters in update package names > After further investigation the $: does get converted to $# > in DosFileDirectory>>checkName:fixErrors: > > The real cause of grief is the tab character immediately following the > colon. > > The method modification below should help, though I expect the list of > invalid charaters is not exhaustive yet > and I'll see if I can find a complete set of them for Windows... > > DosFileDirectory>> > checkName: aFileName fixErrors: fixing > "Check if the file name contains any invalid characters" > | fName badChars hasBadChars | > fName := super checkName: aFileName fixErrors: fixing. > badChars := (#( $: $< $> $| $/ $\ $? $* $"), String tab, String lf, String > cr) asSet. > hasBadChars := fName includesAnyOf: badChars. > (hasBadChars and:[fixing not]) ifTrue:[^self error:'Invalid file name']. > hasBadChars ifFalse:[^ fName]. > ^ fName collect: > [:char | (badChars includes: char) > ifTrue:[$#] > ifFalse:[char]] > > > Regards, Gary > > ----- Original Message ----- > From: "StÃ©phane Ducasse" <stephane.ducasse(a)inria.fr> > To: <Pharo-project(a)lists.gforge.inria.fr> > Sent: Tuesday, December 07, 2010 11:55 AM > Subject: Re: [Pharo-project] Bad characters in update package names > > > > On Dec 7, 2010, at 12:46 PM, Gary Chambers wrote: > >> As Guillermo pointed out >> 'Issue 3191: SparseLargeArray-MarcusDenker.2.mcz' > > > this is the name of the package!!!! > Argh jean-baptiste? > > I will fix it now. > > Stef > > >> >> Amongst others... >> Windows will fail to create the local file in package-cache due to the >> colon in the name... >> >> Regards, Gary >> >> >> ----- Original Message ----- From: "StÃ©phane Ducasse" >> <stephane.ducasse(a)inria.fr> >> To: <Pharo-project(a)lists.gforge.inria.fr> >> Sent: Tuesday, December 07, 2010 11:33 AM >> Subject: Re: [Pharo-project] Bad characters in update package names >> >> >> Gary do you have the name of the package? >> And yes we should check that. I know that luc wrote some doc to make sure >> that package names would not contain stupid characters. >> >> On Dec 6, 2010, at 5:02 PM, Gary Chambers wrote: >> >>> Seems didn't get sent... >>> >>> Regards, Gary >>> ----- Original Message ----- >>> From: Gary Chambers >>> To: Pharo Development >>> Sent: Friday, December 03, 2010 4:02 PM >>> Subject: Bad characters in update package names >>> >>> It seems that having $: in the package name for certain updates is >>> causing >>> the save to local package cache to error on Windows (unsurprisingly). >>> >>> Perhaps worth having a sanity check before the package name is accepted >>> since it eventually gets to the filesystem. >>> >>> Regards, Gary >> >> >> > > >

Dec. 7, 2010

Re: [Pharo-project] Bad characters in update package names

by Gary Chambers

After further investigation the $: does get converted to $# in DosFileDirectory>>checkName:fixErrors: The real cause of grief is the tab character immediately following the colon. The method modification below should help, though I expect the list of invalid charaters is not exhaustive yet and I'll see if I can find a complete set of them for Windows... DosFileDirectory>> checkName: aFileName fixErrors: fixing "Check if the file name contains any invalid characters" | fName badChars hasBadChars | fName := super checkName: aFileName fixErrors: fixing. badChars := (#( $: $< $> $| $/ $\ $? $* $"), String tab, String lf, String cr) asSet. hasBadChars := fName includesAnyOf: badChars. (hasBadChars and:[fixing not]) ifTrue:[^self error:'Invalid file name']. hasBadChars ifFalse:[^ fName]. ^ fName collect: [:char | (badChars includes: char) ifTrue:[$#] ifFalse:[char]] Regards, Gary ----- Original Message ----- From: "StÃ©phane Ducasse" <stephane.ducasse(a)inria.fr> To: <Pharo-project(a)lists.gforge.inria.fr> Sent: Tuesday, December 07, 2010 11:55 AM Subject: Re: [Pharo-project] Bad characters in update package names On Dec 7, 2010, at 12:46 PM, Gary Chambers wrote: > As Guillermo pointed out > 'Issue 3191: SparseLargeArray-MarcusDenker.2.mcz' this is the name of the package!!!! Argh jean-baptiste? I will fix it now. Stef > > Amongst others... > Windows will fail to create the local file in package-cache due to the > colon in the name... > > Regards, Gary > > > ----- Original Message ----- From: "StÃ©phane Ducasse" > <stephane.ducasse(a)inria.fr> > To: <Pharo-project(a)lists.gforge.inria.fr> > Sent: Tuesday, December 07, 2010 11:33 AM > Subject: Re: [Pharo-project] Bad characters in update package names > > > Gary do you have the name of the package? > And yes we should check that. I know that luc wrote some doc to make sure > that package names would not contain stupid characters. > > On Dec 6, 2010, at 5:02 PM, Gary Chambers wrote: > >> Seems didn't get sent... >> >> Regards, Gary >> ----- Original Message ----- >> From: Gary Chambers >> To: Pharo Development >> Sent: Friday, December 03, 2010 4:02 PM >> Subject: Bad characters in update package names >> >> It seems that having $: in the package name for certain updates is >> causing >> the save to local package cache to error on Windows (unsurprisingly). >> >> Perhaps worth having a sanity check before the package name is accepted >> since it eventually gets to the filesystem. >> >> Regards, Gary > > >

Dec. 7, 2010

[Pharo-project] [Progress Report] Zinc HTTP Components

by Sven Van Caekenberghe

This is another progress report on the Zinc HTTP Components project. - I finally integrated support for UTF-8 encoding and decoding of text entities (ZnStringEntity, ZnUTF8Encoder). - Internally, all input and output is done through binary streams (but character streams are still supported for debugging). - A number of units tests were added. - Class comments were improved. - Lots of small fixes and new features. The overall structure of the project was improved by moving old code into a separate MC package, renaming categories and renaming some classes. The external documentation was updated http://homepage.mac.com/svc/Zinc-HTTP-Components I would still love more users/testers. The code works well enough for day to day use and the feature set is reasonably complete now. Sven PS: Please check the SqueakSource history for a all the details.

Dec. 7, 2010

Re: [Pharo-project] Why is FileStream writing almost an order of a magnitude slower than reading ?

by Philippe Marschall

On 12/07/2010 04:20 PM, Levente Uzonyi wrote: > On Tue, 7 Dec 2010, Sven Van Caekenberghe wrote: > >> Related to the Matrix CSV input/output optimalization quest, I was >> puzzled why writing seemed so much slower than reading. >> >> Here is a simple example: >> >> [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | >> 100000 timesRepeat: [ stream print: 100 atRandom; space ] ] ] >> timeToRun. >> 1558 >> [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | >> 100000 timesRepeat: [ Integer readFrom: stream. stream peekFor: $ >> ] ] ] timeToRun. >> 183 >> [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | >> 100000 timesRepeat: [ stream nextPut: ($a to: $z) atRandom; space >> ] ] ] timeToRun. >> 1705 >> [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | >> 100000 timesRepeat: [ stream next. stream peekFor: $ ] ] ] timeToRun. >> 47 >> >> Clearly, the writing is close to an order of magnitude slower than >> reading. >> >> This was on Pharo 1.1 with Cog, but I double-checked with Pharo 1.2 >> and Squeak 4.1. >> >> On my machine (Mac Book Pro), this is what another dynamic language does: >> >> > (time (with-output-to-file (out "/tmp/numbers.txt") >> (loop repeat 100000 do (format out "~d " (random 100))))) >> Timing the evaluation of (WITH-OUTPUT-TO-FILE (OUT "/tmp/numbers.txt") >> (LOOP REPEAT 100000 DO (FORMAT OUT "~d " (RANDOM 100)))) >> >> User time = 0.413 >> System time = 0.002 >> Elapsed time = 0.401 >> Allocation = 2502320 bytes >> 0 Page faults >> Calls to %EVAL 1700063 >> NIL >> >>> (time (with-open-file (in "/tmp/numbers.txt") >> (loop repeat 100000 do (read in)))) >> Timing the evaluation of (WITH-OPEN-FILE (IN "/tmp/numbers.txt") (LOOP >> REPEAT 100000 DO (READ IN))) >> >> User time = 0.328 >> System time = 0.001 >> Elapsed time = 0.315 >> Allocation = 2500764 bytes >> 0 Page faults >> Calls to %EVAL 1400056 >> NIL >> >> So Pharo Smalltalk clearly matches the read/parse speed, which is >> great, but fails at simple writing. >> >> Maybe I am doing something wrong here (I know these are >> MultiByteFileSteams), but I fail to see what. Something with >> buffering/flushing ? >> >> Anybody any idea ? > > That's because filestreams are read buffered, but not write buffered. I > implemented a subclass of FileStream (intended as a possible replacement > of StandardFileStream) which is read and write buffered. It gives the > same performance for reading as the current implementation and a > significant boost for writes, so it can be done. But write buffering has > side effects, while read buffering doesn't. Maybe it can be added as a > separate subclass of FileStream if there's need for it, but the > multibyte stuff has to be duplicated in this case (note that it's > already duplicated in MultiByteFileStream and > MultiByteBinaryOrTextStream). I also had an idea to create > MultiByteStream which would be a stream that wraps another stream and > does the conversion stuff using a TextConverter. It'd be a lot of work > to do it and I don't expect more than 30% performance improvement (for > the read performance). No, buffering should not be in a subclass or even the file stream class itself. Buffering should be an other class that wraps file stream. Cheers Philippe

Dec. 7, 2010

Re: [Pharo-project] Why is FileStream writing almost an order of a magnitude slower than reading ?

by Sven Van Caekenberghe

Levente, On 07 Dec 2010, at 16:20, Levente Uzonyi wrote: > That's because filestreams are read buffered, but not write buffered. I implemented a subclass of FileStream (intended as a possible replacement of StandardFileStream) which is read and write buffered. It gives the same performance for reading as the current implementation and a significant boost for writes, so it can be done. But write buffering has side effects, while read buffering doesn't. Maybe it can be added as a separate subclass of FileStream if there's need for it, but the multibyte stuff has to be duplicated in this case (note that it's already duplicated in MultiByteFileStream and MultiByteBinaryOrTextStream). I also had an idea to create MultiByteStream which would be a stream that wraps another stream and does the conversion stuff using a TextConverter. It'd be a lot of work to do it and I don't expect more than 30% performance improvement (for the read performance). Thanks for the explanation, some quick and dirty buffering makes a huge difference: [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :fileStream | 1000 timesRepeat: [ fileStream nextPutAll: (String streamContents: [ :stream | 100 timesRepeat: [ stream print: 100 atRandom; space ] ]) ] ] ] timeToRun. 159 Still, the asymmetry is a bit strange. Can't the side effects be dealt with using #flush ? > There are several stream libraries (for example XTreams) that can easily support write buffering without the need to care about compatibility. Yeah, although the Smalltalk Collection and Stream classes were better than everything else 20, 30 years ago, lots of things have changed and there is lots of competition. The fact that these classes are so nice to use seem to have prevented necessary improvements. I think I might file this as a Pharo issue. Sven

Dec. 7, 2010

Re: [Pharo-project] Comments or no integration: a simple choice foryou

by Alexandre Bergel

Indeed. We could also refine it further by identifying the most used classes for a representative sample of the Pharo ecosystem. It still says StandardFonts, SystemChangeNotifier, FreeTypeSettings, MCPackage, ... have to be commented Alexandre On 7 Dec 2010, at 12:29, Gary Chambers wrote: > Naturally, every method of a class references its own class as a literal... > Also, class-side methods not accounted for. > > Easy enough to change the script to higlight only external references: > > d := Dictionary new. > Smalltalk allClasses, (Smalltalk allClasses collect: [:cls | cls class]) do: [ :cls | > cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) > do: [:assoc | > (assoc value isBehavior and: [assoc value ~~ cls]) > ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. > associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. > associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] > > Regards, Gary > > ----- Original Message ----- From: "Alexandre Bergel" <alexandre(a)bergel.eu> > To: <Pharo-project(a)lists.gforge.inria.fr> > Sent: Tuesday, December 07, 2010 2:53 PM > Subject: Re: [Pharo-project] Comments or no integration: a simple choice foryou > > > A small script that gives the most referenced classes that are not commented: > I run the test in an image with Moose and Spy. > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > "Get the number of references per class" > d := Dictionary new. > Smalltalk allClasses do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) > do: [:assoc | > (assoc value isBehavior) > ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. > associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. > > "Get the classes that are not commented" > associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= > > The first 30 entries: > SortedCollectionTest->307 > HeapTest->304 > IntervalTest->250 > DictionaryTest->192 > BagTest->167 > RxMatcherTest->166 > PluggableTextMorph->149 > MooseModel->144 > UIThemeWatery2->132 > SystemChangeNotifier->132 > PPSmalltalkParserTests->129 > FAMIXClassNavigationTest->128 > PPJavaSyntax->128 > PPSmalltalkGrammarTests->128 > RBRefactoryTestDataApp->123 > NSFileStream->118 > NSRWBinaryOrTextStream->109 > LogicalFont->109 > NSMultiByteBinaryOrTextStream->108 > UIThemeW2K->103 > MCWorkingCopy->101 > BrowserEnvironment->97 > DemoSettingStyle->94 > PPJavaLexiconTests->94 > MCClassDefinition->93 > NSAbstractInflateStream->92 > MCWorkingCopyBrowser->91 > NSCollectionStreamTest->91 > GMMetaObject->91 > ExceptionTester->90 > > > Cheers, > Alexandre > > > On 7 Dec 2010, at 10:48, StÃ©phane Ducasse wrote: > >>> The good idae is to push and promote people to put class comments, but not to remove not commented classes from Dev, bacause otherise, you will need to remove 80% of the classes. >> >> >> So this is simple to start fixing some of them. >> I think that without such objectives we will never make any progress and we should. >> >> so what are the first 5% of packages that are not documented. >> >> Stef >> >>> The next step would be a Pareto classification, as it can be accepted (I hope) that SUnit has more history (and it is documented elsewhere, like PBE book) than other classes which even the name of it or its methods may be insufficient to arrive at for what is the class. >>> >>> We should create IRC meetings, sprints, etc. specially targeted to document these classes. >>> >>> my .019999... >>> >>> -- >>> Cesar Rabak >>> >>> >>> Em 05/12/2010 20:07, Mariano Martinez Peck < marianopeck(a)gmail.com > escreveu: >>> >>> On Sun, Dec 5, 2010 at 9:52 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >>> >>> Hi guys >>> >>> I decided that I will not integrate any code that is not documented in Pharo. >>> >>> I strongly suggest to remove from Pharo-dev packages whose classes are not commented. >>> >>> >>> I think this is not a good idea...look this approx: >>> >>> >>> >>> | pharoCorePackages packagesDev nonCommentedClasses totalClasses | >>> pharoCorePackages := #('MultilingualTests' 'Morphic' 'ToolBuilder-Kernel' 'PackageInfo' 'Polymorph-ToolBuilder' 'Network-Kernel' 'Network-Protocols' 'Compiler' 'System-Finalization' 'Settings-Graphics' 'Collections' 'Network-RemoteDirectory' 'Traits' 'Network-URI' 'Settings-Display' 'Collections-Abstract' 'Settings-Compiler' 'MonticelloMocks' 'ScriptLoader11' 'System-Digital Signatures' 'System-Localization' 'MenuRegistration' 'Polymorph-TaskbarIcons' 'Monticello' 'ToolBuilder-Morphic' 'MCDirtyPackage' 'Collections-Weak' 'Settings-Polymorph' 'Tests' 'HelpSystem-Core-Utilities' 'SUnit' 'Collections-Text' 'Compression' 'Morphic-MorphTreeWidget' 'HelpSystem-Core' 'ToolsTest' 'Gofer-Core' 'Gofer-Tests' 'Collections-Stack' 'FreeType' 'MonticelloConfigurations' 'Polymorph-Geometry' 'Collections-Support' 'SUnitGUI' 'HelpSystem-Core-Model' 'Tests-VM' 'System-Change Notification' 'System' 'Tests-ObjectsAsMethods' 'HostMenus' 'Network-MIME' 'Network-RFC822' 'VB-Regex' 'TrueType' 'Depr >>> ecatedPreferences' 'System-Clipboard' 'System-Settings' 'MorphicTests' 'System-Serial Port' 'NetworkTests' 'Collections-Unordered' 'System-Changes' 'System-Object Storage' 'Collections-Traits' 'System-Platforms' 'Tests-PrimCallController' 'Morphic-TrueType' 'Multilingual' 'Graphics' 'Collections-Strings' 'KernelTests' 'System-FileRegistry' 'Tests-Bugs' 'ToolBuilder-SUnit' 'Tests-Release' 'Morphic-Balloon' 'Settings-Network' 'System-Applications' 'Polymorph-Tools-Diff' 'Settings-Tools' 'ScriptLoader' 'ST80' 'FreeTypeSubPixelAntiAliasing' 'Settings-Kernel' 'Settings-System' 'CompilerTests' 'System-Tools' 'System-Hashing' 'System-Download' 'Collections-Streams' 'Tests-SystemChangeNotification' 'GraphicsTests' 'Polymorph-Widgets' 'Kernel' 'Tools' 'Settings-FreeType' 'FreeTypeTests' 'Settings-Monticello' 'HelpSystem-Tests' 'Polymorph-EventEnhancements' 'System-Support' 'MonticelloGUI' 'System-FilePackage' 'System-Object Events' 'Announcements' 'System-Text' 'Exceptions' 'Network- >>> UUID' 'System-Pools' 'FixUnderscores' 'HelpSystem-Core-UI' 'GraphicsResources' 'Files' 'Gofer' 'Network-MailSending' 'Collections-Arrayed' 'CollectionsTests' 'Balloon' 'Network-Url' 'Collections-Sequenceable' ). >>> packagesDev := (PackageInfo allPackages >>> collect: [:each | each packageName]) >>> difference: pharoCorePackages. >>> nonCommentedClasses := packagesDev >>> inject: 0 >>> into: [:sum :each | sum + (((PackageInfo named: each) classes >>> select: [:aClass | aClass comment asString includesSubString: 'is xxxxxxxxx.']) size)]. >>> Transcript show: 'nonCommentedClasses: ', nonCommentedClasses asString. >>> >>> totalClasses := packagesDev inject: 0 into: [:sum :each | sum + (PackageInfo named: each) classes size ]. >>> Transcript show: 'totalClasses: ', totalClasses asString. >>> >>> >>> >>> Shows: >>> >>> nonCommentedClasses: 1116 >>> totalClasses: 1369 >>> >>> >>> I'm not sure that I will look at code or answer question to code that is not commented. >>> We should change our mindset and it seems that we do not care, so we should take radical decisions: >>> less changes, less progress, more comments. >>> >>> And for once I will not bash the past. Smalltalk was always with methods fully documented. >>> We are just plain lazy and this is a shame. >>> >>> Stef >>> >>> >>> >>> >>> >>> >> >> > > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 7, 2010

Re: [Pharo-project] Native Windows in Pharo

by Tudor Girba

Thanks for the translation. Indeed, this is what I meant as well :) Doru On 7 Dec 2010, at 14:39, StÃ©phane Ducasse wrote: > > On Dec 7, 2010, at 2:13 PM, Geert Claes wrote: > >> >> Hi Andres, Ideas and suggestions are very welcome in the Pharo community and >> they are mainly discussed here rather than in a UserVoice or GetSatisfaction >> environment. Idea voting is an excellent way to engage customers and users >> to improve products. The only thing Stef and Doru are saying (at least >> thats how I read it) is that Pharo doesn't necessarily have a shortage of >> ideas or lack of direction, but simply not enough people to do all the work. > > :) > indeed well rephrased :) > >> So yes, ideas and feature requests are great and please do continue to post >> them here as there are many enthusiastic skilled Smalltalkers that can help, >> advice and guide you. When you find something is missing or can be >> improved, you are more encouraged to contribute towards the solution rather >> than just vote. >> >> Getting back to your original post though, congrats getting your in-house >> project! How about a rich Seaside, AIDAWeb or Iliad browser based app? >> >> Cheers, Geert >> >> >> >> Andres Fortier-2 wrote: >>> >>> Stef, Doru: >>> >>> first of all I was *asking if* such a model exists, not demanding one. >>> Having just entered the Pharo world I wanted to know how you guys handle >>> the feature requests. I see now. >>> >>> On the other hand if you really want to discuss the question of "who has >>> the rights for asking for X" in the open source model I'm open, but just >>> to be sure I will first ask: Is it ok to open such a thread? Or should I >>> first contribute to Pharo before asking? >>> >>> AndrÃ©s >>> >> >> -- >> View this message in context: http://forum.world.st/Native-Windows-in-Pharo-tp3071794p3076336.html >> Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >> > > -- www.tudorgirba.com "Problem solving should be focused on describing the problem in a way that makes the solution obvious."

Dec. 7, 2010

Re: [Pharo-project] Comments or no integration: a simple choice foryou

by Gary Chambers

Naturally, every method of a class references its own class as a literal... Also, class-side methods not accounted for. Easy enough to change the script to higlight only external references: d := Dictionary new. Smalltalk allClasses, (Smalltalk allClasses collect: [:cls | cls class]) do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) do: [:assoc | (assoc value isBehavior and: [assoc value ~~ cls]) ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] Regards, Gary ----- Original Message ----- From: "Alexandre Bergel" <alexandre(a)bergel.eu> To: <Pharo-project(a)lists.gforge.inria.fr> Sent: Tuesday, December 07, 2010 2:53 PM Subject: Re: [Pharo-project] Comments or no integration: a simple choice foryou A small script that gives the most referenced classes that are not commented: I run the test in an image with Moose and Spy. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "Get the number of references per class" d := Dictionary new. Smalltalk allClasses do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) do: [:assoc | (assoc value isBehavior) ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. "Get the classes that are not commented" associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The first 30 entries: SortedCollectionTest->307 HeapTest->304 IntervalTest->250 DictionaryTest->192 BagTest->167 RxMatcherTest->166 PluggableTextMorph->149 MooseModel->144 UIThemeWatery2->132 SystemChangeNotifier->132 PPSmalltalkParserTests->129 FAMIXClassNavigationTest->128 PPJavaSyntax->128 PPSmalltalkGrammarTests->128 RBRefactoryTestDataApp->123 NSFileStream->118 NSRWBinaryOrTextStream->109 LogicalFont->109 NSMultiByteBinaryOrTextStream->108 UIThemeW2K->103 MCWorkingCopy->101 BrowserEnvironment->97 DemoSettingStyle->94 PPJavaLexiconTests->94 MCClassDefinition->93 NSAbstractInflateStream->92 MCWorkingCopyBrowser->91 NSCollectionStreamTest->91 GMMetaObject->91 ExceptionTester->90 Cheers, Alexandre On 7 Dec 2010, at 10:48, StÃ©phane Ducasse wrote: >> The good idae is to push and promote people to put class comments, but >> not to remove not commented classes from Dev, bacause otherise, you will >> need to remove 80% of the classes. > > > So this is simple to start fixing some of them. > I think that without such objectives we will never make any progress and > we should. > > so what are the first 5% of packages that are not documented. > > Stef > >> The next step would be a Pareto classification, as it can be accepted (I >> hope) that SUnit has more history (and it is documented elsewhere, like >> PBE book) than other classes which even the name of it or its methods may >> be insufficient to arrive at for what is the class. >> >> We should create IRC meetings, sprints, etc. specially targeted to >> document these classes. >> >> my .019999... >> >> -- >> Cesar Rabak >> >> >> Em 05/12/2010 20:07, Mariano Martinez Peck < marianopeck(a)gmail.com > >> escreveu: >> >> On Sun, Dec 5, 2010 at 9:52 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >> >> Hi guys >> >> I decided that I will not integrate any code that is not documented in >> Pharo. >> >> I strongly suggest to remove from Pharo-dev packages whose classes are >> not commented. >> >> >> I think this is not a good idea...look this approx: >> >> >> >> | pharoCorePackages packagesDev nonCommentedClasses totalClasses | >> pharoCorePackages := #('MultilingualTests' 'Morphic' 'ToolBuilder-Kernel' >> 'PackageInfo' 'Polymorph-ToolBuilder' 'Network-Kernel' >> 'Network-Protocols' 'Compiler' 'System-Finalization' 'Settings-Graphics' >> 'Collections' 'Network-RemoteDirectory' 'Traits' 'Network-URI' >> 'Settings-Display' 'Collections-Abstract' 'Settings-Compiler' >> 'MonticelloMocks' 'ScriptLoader11' 'System-Digital Signatures' >> 'System-Localization' 'MenuRegistration' 'Polymorph-TaskbarIcons' >> 'Monticello' 'ToolBuilder-Morphic' 'MCDirtyPackage' 'Collections-Weak' >> 'Settings-Polymorph' 'Tests' 'HelpSystem-Core-Utilities' 'SUnit' >> 'Collections-Text' 'Compression' 'Morphic-MorphTreeWidget' >> 'HelpSystem-Core' 'ToolsTest' 'Gofer-Core' 'Gofer-Tests' >> 'Collections-Stack' 'FreeType' 'MonticelloConfigurations' >> 'Polymorph-Geometry' 'Collections-Support' 'SUnitGUI' >> 'HelpSystem-Core-Model' 'Tests-VM' 'System-Change Notification' 'System' >> 'Tests-ObjectsAsMethods' 'HostMenus' 'Network-MIME' 'Network-RFC822' >> 'VB-Regex' 'TrueType' 'Depr >> ecatedPreferences' 'System-Clipboard' 'System-Settings' 'MorphicTests' >> 'System-Serial Port' 'NetworkTests' 'Collections-Unordered' >> 'System-Changes' 'System-Object Storage' 'Collections-Traits' >> 'System-Platforms' 'Tests-PrimCallController' 'Morphic-TrueType' >> 'Multilingual' 'Graphics' 'Collections-Strings' 'KernelTests' >> 'System-FileRegistry' 'Tests-Bugs' 'ToolBuilder-SUnit' 'Tests-Release' >> 'Morphic-Balloon' 'Settings-Network' 'System-Applications' >> 'Polymorph-Tools-Diff' 'Settings-Tools' 'ScriptLoader' 'ST80' >> 'FreeTypeSubPixelAntiAliasing' 'Settings-Kernel' 'Settings-System' >> 'CompilerTests' 'System-Tools' 'System-Hashing' 'System-Download' >> 'Collections-Streams' 'Tests-SystemChangeNotification' 'GraphicsTests' >> 'Polymorph-Widgets' 'Kernel' 'Tools' 'Settings-FreeType' 'FreeTypeTests' >> 'Settings-Monticello' 'HelpSystem-Tests' 'Polymorph-EventEnhancements' >> 'System-Support' 'MonticelloGUI' 'System-FilePackage' 'System-Object >> Events' 'Announcements' 'System-Text' 'Exceptions' 'Network- >> UUID' 'System-Pools' 'FixUnderscores' 'HelpSystem-Core-UI' >> 'GraphicsResources' 'Files' 'Gofer' 'Network-MailSending' >> 'Collections-Arrayed' 'CollectionsTests' 'Balloon' 'Network-Url' >> 'Collections-Sequenceable' ). >> packagesDev := (PackageInfo allPackages >> collect: [:each | each packageName]) >> difference: pharoCorePackages. >> nonCommentedClasses := packagesDev >> inject: 0 >> into: [:sum :each | sum + (((PackageInfo named: each) classes >> select: [:aClass | aClass comment asString includesSubString: 'is >> xxxxxxxxx.']) size)]. >> Transcript show: 'nonCommentedClasses: ', nonCommentedClasses asString. >> >> totalClasses := packagesDev inject: 0 into: [:sum :each | sum + >> (PackageInfo named: each) classes size ]. >> Transcript show: 'totalClasses: ', totalClasses asString. >> >> >> >> Shows: >> >> nonCommentedClasses: 1116 >> totalClasses: 1369 >> >> >> I'm not sure that I will look at code or answer question to code that is >> not commented. >> We should change our mindset and it seems that we do not care, so we >> should take radical decisions: >> less changes, less progress, more comments. >> >> And for once I will not bash the past. Smalltalk was always with methods >> fully documented. >> We are just plain lazy and this is a shame. >> >> Stef >> >> >> >> >> >> > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 7, 2010

Re: [Pharo-project] Why is FileStream writing almost an order of a magnitude slower than reading ?

by Levente Uzonyi

On Tue, 7 Dec 2010, Sven Van Caekenberghe wrote: > Related to the Matrix CSV input/output optimalization quest, I was puzzled why writing seemed so much slower than reading. > > Here is a simple example: > > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream print: 100 atRandom; space ] ] ] timeToRun. > 1558 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ Integer readFrom: stream. stream peekFor: $ ] ] ] timeToRun. > 183 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream nextPut: ($a to: $z) atRandom; space ] ] ] timeToRun. > 1705 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream next. stream peekFor: $ ] ] ] timeToRun. > 47 > > Clearly, the writing is close to an order of magnitude slower than reading. > > This was on Pharo 1.1 with Cog, but I double-checked with Pharo 1.2 and Squeak 4.1. > > On my machine (Mac Book Pro), this is what another dynamic language does: > > > (time (with-output-to-file (out "/tmp/numbers.txt") > (loop repeat 100000 do (format out "~d " (random 100))))) > Timing the evaluation of (WITH-OUTPUT-TO-FILE (OUT "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (FORMAT OUT "~d " (RANDOM 100)))) > > User time = 0.413 > System time = 0.002 > Elapsed time = 0.401 > Allocation = 2502320 bytes > 0 Page faults > Calls to %EVAL 1700063 > NIL > >> (time (with-open-file (in "/tmp/numbers.txt") > (loop repeat 100000 do (read in)))) > Timing the evaluation of (WITH-OPEN-FILE (IN "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (READ IN))) > > User time = 0.328 > System time = 0.001 > Elapsed time = 0.315 > Allocation = 2500764 bytes > 0 Page faults > Calls to %EVAL 1400056 > NIL > > So Pharo Smalltalk clearly matches the read/parse speed, which is great, but fails at simple writing. > > Maybe I am doing something wrong here (I know these are MultiByteFileSteams), but I fail to see what. Something with buffering/flushing ? > > Anybody any idea ? That's because filestreams are read buffered, but not write buffered. I implemented a subclass of FileStream (intended as a possible replacement of StandardFileStream) which is read and write buffered. It gives the same performance for reading as the current implementation and a significant boost for writes, so it can be done. But write buffering has side effects, while read buffering doesn't. Maybe it can be added as a separate subclass of FileStream if there's need for it, but the multibyte stuff has to be duplicated in this case (note that it's already duplicated in MultiByteFileStream and MultiByteBinaryOrTextStream). I also had an idea to create MultiByteStream which would be a stream that wraps another stream and does the conversion stuff using a TextConverter. It'd be a lot of work to do it and I don't expect more than 30% performance improvement (for the read performance). There are several stream libraries (for example XTreams) that can easily support write buffering without the need to care about compatibility. Levente > > Sven > > >

Dec. 7, 2010