December 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Comments or no integration: a simple choice foryou

by Gary Chambers

Naturally, every method of a class references its own class as a literal... Also, class-side methods not accounted for. Easy enough to change the script to higlight only external references: d := Dictionary new. Smalltalk allClasses, (Smalltalk allClasses collect: [:cls | cls class]) do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) do: [:assoc | (assoc value isBehavior and: [assoc value ~~ cls]) ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] Regards, Gary ----- Original Message ----- From: "Alexandre Bergel" <alexandre(a)bergel.eu> To: <Pharo-project(a)lists.gforge.inria.fr> Sent: Tuesday, December 07, 2010 2:53 PM Subject: Re: [Pharo-project] Comments or no integration: a simple choice foryou A small script that gives the most referenced classes that are not commented: I run the test in an image with Moose and Spy. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "Get the number of references per class" d := Dictionary new. Smalltalk allClasses do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) do: [:assoc | (assoc value isBehavior) ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. "Get the classes that are not commented" associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The first 30 entries: SortedCollectionTest->307 HeapTest->304 IntervalTest->250 DictionaryTest->192 BagTest->167 RxMatcherTest->166 PluggableTextMorph->149 MooseModel->144 UIThemeWatery2->132 SystemChangeNotifier->132 PPSmalltalkParserTests->129 FAMIXClassNavigationTest->128 PPJavaSyntax->128 PPSmalltalkGrammarTests->128 RBRefactoryTestDataApp->123 NSFileStream->118 NSRWBinaryOrTextStream->109 LogicalFont->109 NSMultiByteBinaryOrTextStream->108 UIThemeW2K->103 MCWorkingCopy->101 BrowserEnvironment->97 DemoSettingStyle->94 PPJavaLexiconTests->94 MCClassDefinition->93 NSAbstractInflateStream->92 MCWorkingCopyBrowser->91 NSCollectionStreamTest->91 GMMetaObject->91 ExceptionTester->90 Cheers, Alexandre On 7 Dec 2010, at 10:48, StÃ©phane Ducasse wrote: >> The good idae is to push and promote people to put class comments, but >> not to remove not commented classes from Dev, bacause otherise, you will >> need to remove 80% of the classes. > > > So this is simple to start fixing some of them. > I think that without such objectives we will never make any progress and > we should. > > so what are the first 5% of packages that are not documented. > > Stef > >> The next step would be a Pareto classification, as it can be accepted (I >> hope) that SUnit has more history (and it is documented elsewhere, like >> PBE book) than other classes which even the name of it or its methods may >> be insufficient to arrive at for what is the class. >> >> We should create IRC meetings, sprints, etc. specially targeted to >> document these classes. >> >> my .019999... >> >> -- >> Cesar Rabak >> >> >> Em 05/12/2010 20:07, Mariano Martinez Peck < marianopeck(a)gmail.com > >> escreveu: >> >> On Sun, Dec 5, 2010 at 9:52 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >> >> Hi guys >> >> I decided that I will not integrate any code that is not documented in >> Pharo. >> >> I strongly suggest to remove from Pharo-dev packages whose classes are >> not commented. >> >> >> I think this is not a good idea...look this approx: >> >> >> >> | pharoCorePackages packagesDev nonCommentedClasses totalClasses | >> pharoCorePackages := #('MultilingualTests' 'Morphic' 'ToolBuilder-Kernel' >> 'PackageInfo' 'Polymorph-ToolBuilder' 'Network-Kernel' >> 'Network-Protocols' 'Compiler' 'System-Finalization' 'Settings-Graphics' >> 'Collections' 'Network-RemoteDirectory' 'Traits' 'Network-URI' >> 'Settings-Display' 'Collections-Abstract' 'Settings-Compiler' >> 'MonticelloMocks' 'ScriptLoader11' 'System-Digital Signatures' >> 'System-Localization' 'MenuRegistration' 'Polymorph-TaskbarIcons' >> 'Monticello' 'ToolBuilder-Morphic' 'MCDirtyPackage' 'Collections-Weak' >> 'Settings-Polymorph' 'Tests' 'HelpSystem-Core-Utilities' 'SUnit' >> 'Collections-Text' 'Compression' 'Morphic-MorphTreeWidget' >> 'HelpSystem-Core' 'ToolsTest' 'Gofer-Core' 'Gofer-Tests' >> 'Collections-Stack' 'FreeType' 'MonticelloConfigurations' >> 'Polymorph-Geometry' 'Collections-Support' 'SUnitGUI' >> 'HelpSystem-Core-Model' 'Tests-VM' 'System-Change Notification' 'System' >> 'Tests-ObjectsAsMethods' 'HostMenus' 'Network-MIME' 'Network-RFC822' >> 'VB-Regex' 'TrueType' 'Depr >> ecatedPreferences' 'System-Clipboard' 'System-Settings' 'MorphicTests' >> 'System-Serial Port' 'NetworkTests' 'Collections-Unordered' >> 'System-Changes' 'System-Object Storage' 'Collections-Traits' >> 'System-Platforms' 'Tests-PrimCallController' 'Morphic-TrueType' >> 'Multilingual' 'Graphics' 'Collections-Strings' 'KernelTests' >> 'System-FileRegistry' 'Tests-Bugs' 'ToolBuilder-SUnit' 'Tests-Release' >> 'Morphic-Balloon' 'Settings-Network' 'System-Applications' >> 'Polymorph-Tools-Diff' 'Settings-Tools' 'ScriptLoader' 'ST80' >> 'FreeTypeSubPixelAntiAliasing' 'Settings-Kernel' 'Settings-System' >> 'CompilerTests' 'System-Tools' 'System-Hashing' 'System-Download' >> 'Collections-Streams' 'Tests-SystemChangeNotification' 'GraphicsTests' >> 'Polymorph-Widgets' 'Kernel' 'Tools' 'Settings-FreeType' 'FreeTypeTests' >> 'Settings-Monticello' 'HelpSystem-Tests' 'Polymorph-EventEnhancements' >> 'System-Support' 'MonticelloGUI' 'System-FilePackage' 'System-Object >> Events' 'Announcements' 'System-Text' 'Exceptions' 'Network- >> UUID' 'System-Pools' 'FixUnderscores' 'HelpSystem-Core-UI' >> 'GraphicsResources' 'Files' 'Gofer' 'Network-MailSending' >> 'Collections-Arrayed' 'CollectionsTests' 'Balloon' 'Network-Url' >> 'Collections-Sequenceable' ). >> packagesDev := (PackageInfo allPackages >> collect: [:each | each packageName]) >> difference: pharoCorePackages. >> nonCommentedClasses := packagesDev >> inject: 0 >> into: [:sum :each | sum + (((PackageInfo named: each) classes >> select: [:aClass | aClass comment asString includesSubString: 'is >> xxxxxxxxx.']) size)]. >> Transcript show: 'nonCommentedClasses: ', nonCommentedClasses asString. >> >> totalClasses := packagesDev inject: 0 into: [:sum :each | sum + >> (PackageInfo named: each) classes size ]. >> Transcript show: 'totalClasses: ', totalClasses asString. >> >> >> >> Shows: >> >> nonCommentedClasses: 1116 >> totalClasses: 1369 >> >> >> I'm not sure that I will look at code or answer question to code that is >> not commented. >> We should change our mindset and it seems that we do not care, so we >> should take radical decisions: >> less changes, less progress, more comments. >> >> And for once I will not bash the past. Smalltalk was always with methods >> fully documented. >> We are just plain lazy and this is a shame. >> >> Stef >> >> >> >> >> >> > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 7, 2010

Re: [Pharo-project] Why is FileStream writing almost an order of a magnitude slower than reading ?

by Levente Uzonyi

On Tue, 7 Dec 2010, Sven Van Caekenberghe wrote: > Related to the Matrix CSV input/output optimalization quest, I was puzzled why writing seemed so much slower than reading. > > Here is a simple example: > > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream print: 100 atRandom; space ] ] ] timeToRun. > 1558 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ Integer readFrom: stream. stream peekFor: $ ] ] ] timeToRun. > 183 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream nextPut: ($a to: $z) atRandom; space ] ] ] timeToRun. > 1705 > [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | > 100000 timesRepeat: [ stream next. stream peekFor: $ ] ] ] timeToRun. > 47 > > Clearly, the writing is close to an order of magnitude slower than reading. > > This was on Pharo 1.1 with Cog, but I double-checked with Pharo 1.2 and Squeak 4.1. > > On my machine (Mac Book Pro), this is what another dynamic language does: > > > (time (with-output-to-file (out "/tmp/numbers.txt") > (loop repeat 100000 do (format out "~d " (random 100))))) > Timing the evaluation of (WITH-OUTPUT-TO-FILE (OUT "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (FORMAT OUT "~d " (RANDOM 100)))) > > User time = 0.413 > System time = 0.002 > Elapsed time = 0.401 > Allocation = 2502320 bytes > 0 Page faults > Calls to %EVAL 1700063 > NIL > >> (time (with-open-file (in "/tmp/numbers.txt") > (loop repeat 100000 do (read in)))) > Timing the evaluation of (WITH-OPEN-FILE (IN "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (READ IN))) > > User time = 0.328 > System time = 0.001 > Elapsed time = 0.315 > Allocation = 2500764 bytes > 0 Page faults > Calls to %EVAL 1400056 > NIL > > So Pharo Smalltalk clearly matches the read/parse speed, which is great, but fails at simple writing. > > Maybe I am doing something wrong here (I know these are MultiByteFileSteams), but I fail to see what. Something with buffering/flushing ? > > Anybody any idea ? That's because filestreams are read buffered, but not write buffered. I implemented a subclass of FileStream (intended as a possible replacement of StandardFileStream) which is read and write buffered. It gives the same performance for reading as the current implementation and a significant boost for writes, so it can be done. But write buffering has side effects, while read buffering doesn't. Maybe it can be added as a separate subclass of FileStream if there's need for it, but the multibyte stuff has to be duplicated in this case (note that it's already duplicated in MultiByteFileStream and MultiByteBinaryOrTextStream). I also had an idea to create MultiByteStream which would be a stream that wraps another stream and does the conversion stuff using a TextConverter. It'd be a lot of work to do it and I don't expect more than 30% performance improvement (for the read performance). There are several stream libraries (for example XTreams) that can easily support write buffering without the need to care about compatibility. Levente > > Sven > > >

Dec. 7, 2010

Re: [Pharo-project] Comments or no integration: a simple choice for you

by Alexandre Bergel

A small script that gives the most referenced classes that are not commented: I run the test in an image with Moose and Spy. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= "Get the number of references per class" d := Dictionary new. Smalltalk allClasses do: [ :cls | cls methods do: [:cm | (cm literals select: [:l | l isKindOf: Association ]) do: [:assoc | (assoc value isBehavior) ifTrue: [d at: assoc value put: (d at: assoc value ifAbsentPut: [0] ) + 1]]]]. associations := d associations asSortedCollection: [:a1 :a2 | a2 value < a1 value]. "Get the classes that are not commented" associations select: [:ass | ass key comment isNil or: [ass key comment isEmpty or: [ass key comment asString includesSubString: 'xxxxxx']]] -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= The first 30 entries: SortedCollectionTest->307 HeapTest->304 IntervalTest->250 DictionaryTest->192 BagTest->167 RxMatcherTest->166 PluggableTextMorph->149 MooseModel->144 UIThemeWatery2->132 SystemChangeNotifier->132 PPSmalltalkParserTests->129 FAMIXClassNavigationTest->128 PPJavaSyntax->128 PPSmalltalkGrammarTests->128 RBRefactoryTestDataApp->123 NSFileStream->118 NSRWBinaryOrTextStream->109 LogicalFont->109 NSMultiByteBinaryOrTextStream->108 UIThemeW2K->103 MCWorkingCopy->101 BrowserEnvironment->97 DemoSettingStyle->94 PPJavaLexiconTests->94 MCClassDefinition->93 NSAbstractInflateStream->92 MCWorkingCopyBrowser->91 NSCollectionStreamTest->91 GMMetaObject->91 ExceptionTester->90 Cheers, Alexandre On 7 Dec 2010, at 10:48, StÃ©phane Ducasse wrote: >> The good idae is to push and promote people to put class comments, but not to remove not commented classes from Dev, bacause otherise, you will need to remove 80% of the classes. > > > So this is simple to start fixing some of them. > I think that without such objectives we will never make any progress and we should. > > so what are the first 5% of packages that are not documented. > > Stef > >> The next step would be a Pareto classification, as it can be accepted (I hope) that SUnit has more history (and it is documented elsewhere, like PBE book) than other classes which even the name of it or its methods may be insufficient to arrive at for what is the class. >> >> We should create IRC meetings, sprints, etc. specially targeted to document these classes. >> >> my .019999... >> >> -- >> Cesar Rabak >> >> >> Em 05/12/2010 20:07, Mariano Martinez Peck < marianopeck(a)gmail.com > escreveu: >> >> On Sun, Dec 5, 2010 at 9:52 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >> >> Hi guys >> >> I decided that I will not integrate any code that is not documented in Pharo. >> >> I strongly suggest to remove from Pharo-dev packages whose classes are not commented. >> >> >> I think this is not a good idea...look this approx: >> >> >> >> | pharoCorePackages packagesDev nonCommentedClasses totalClasses | >> pharoCorePackages := #('MultilingualTests' 'Morphic' 'ToolBuilder-Kernel' 'PackageInfo' 'Polymorph-ToolBuilder' 'Network-Kernel' 'Network-Protocols' 'Compiler' 'System-Finalization' 'Settings-Graphics' 'Collections' 'Network-RemoteDirectory' 'Traits' 'Network-URI' 'Settings-Display' 'Collections-Abstract' 'Settings-Compiler' 'MonticelloMocks' 'ScriptLoader11' 'System-Digital Signatures' 'System-Localization' 'MenuRegistration' 'Polymorph-TaskbarIcons' 'Monticello' 'ToolBuilder-Morphic' 'MCDirtyPackage' 'Collections-Weak' 'Settings-Polymorph' 'Tests' 'HelpSystem-Core-Utilities' 'SUnit' 'Collections-Text' 'Compression' 'Morphic-MorphTreeWidget' 'HelpSystem-Core' 'ToolsTest' 'Gofer-Core' 'Gofer-Tests' 'Collections-Stack' 'FreeType' 'MonticelloConfigurations' 'Polymorph-Geometry' 'Collections-Support' 'SUnitGUI' 'HelpSystem-Core-Model' 'Tests-VM' 'System-Change Notification' 'System' 'Tests-ObjectsAsMethods' 'HostMenus' 'Network-MIME' 'Network-RFC822' 'VB-Regex' 'TrueType' 'Depr >> ecatedPreferences' 'System-Clipboard' 'System-Settings' 'MorphicTests' 'System-Serial Port' 'NetworkTests' 'Collections-Unordered' 'System-Changes' 'System-Object Storage' 'Collections-Traits' 'System-Platforms' 'Tests-PrimCallController' 'Morphic-TrueType' 'Multilingual' 'Graphics' 'Collections-Strings' 'KernelTests' 'System-FileRegistry' 'Tests-Bugs' 'ToolBuilder-SUnit' 'Tests-Release' 'Morphic-Balloon' 'Settings-Network' 'System-Applications' 'Polymorph-Tools-Diff' 'Settings-Tools' 'ScriptLoader' 'ST80' 'FreeTypeSubPixelAntiAliasing' 'Settings-Kernel' 'Settings-System' 'CompilerTests' 'System-Tools' 'System-Hashing' 'System-Download' 'Collections-Streams' 'Tests-SystemChangeNotification' 'GraphicsTests' 'Polymorph-Widgets' 'Kernel' 'Tools' 'Settings-FreeType' 'FreeTypeTests' 'Settings-Monticello' 'HelpSystem-Tests' 'Polymorph-EventEnhancements' 'System-Support' 'MonticelloGUI' 'System-FilePackage' 'System-Object Events' 'Announcements' 'System-Text' 'Exceptions' 'Network- >> UUID' 'System-Pools' 'FixUnderscores' 'HelpSystem-Core-UI' 'GraphicsResources' 'Files' 'Gofer' 'Network-MailSending' 'Collections-Arrayed' 'CollectionsTests' 'Balloon' 'Network-Url' 'Collections-Sequenceable' ). >> packagesDev := (PackageInfo allPackages >> collect: [:each | each packageName]) >> difference: pharoCorePackages. >> nonCommentedClasses := packagesDev >> inject: 0 >> into: [:sum :each | sum + (((PackageInfo named: each) classes >> select: [:aClass | aClass comment asString includesSubString: 'is xxxxxxxxx.']) size)]. >> Transcript show: 'nonCommentedClasses: ', nonCommentedClasses asString. >> >> totalClasses := packagesDev inject: 0 into: [:sum :each | sum + (PackageInfo named: each) classes size ]. >> Transcript show: 'totalClasses: ', totalClasses asString. >> >> >> >> Shows: >> >> nonCommentedClasses: 1116 >> totalClasses: 1369 >> >> >> I'm not sure that I will look at code or answer question to code that is not commented. >> We should change our mindset and it seems that we do not care, so we should take radical decisions: >> less changes, less progress, more comments. >> >> And for once I will not bash the past. Smalltalk was always with methods fully documented. >> We are just plain lazy and this is a shame. >> >> Stef >> >> >> >> >> >> > > -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 7, 2010

Re: [Pharo-project] Comments or no integration: a simple choice for you

by StÃ©phane Ducasse

> The good idae is to push and promote people to put class comments, but not to remove not commented classes from Dev, bacause otherise, you will need to remove 80% of the classes. So this is simple to start fixing some of them. I think that without such objectives we will never make any progress and we should. so what are the first 5% of packages that are not documented. Stef > The next step would be a Pareto classification, as it can be accepted (I hope) that SUnit has more history (and it is documented elsewhere, like PBE book) than other classes which even the name of it or its methods may be insufficient to arrive at for what is the class. > > We should create IRC meetings, sprints, etc. specially targeted to document these classes. > > my .019999... > > -- > Cesar Rabak > > > Em 05/12/2010 20:07, Mariano Martinez Peck < marianopeck(a)gmail.com > escreveu: > > On Sun, Dec 5, 2010 at 9:52 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > > Hi guys > > I decided that I will not integrate any code that is not documented in Pharo. > > I strongly suggest to remove from Pharo-dev packages whose classes are not commented. > > > I think this is not a good idea...look this approx: > > > > | pharoCorePackages packagesDev nonCommentedClasses totalClasses | > pharoCorePackages := #('MultilingualTests' 'Morphic' 'ToolBuilder-Kernel' 'PackageInfo' 'Polymorph-ToolBuilder' 'Network-Kernel' 'Network-Protocols' 'Compiler' 'System-Finalization' 'Settings-Graphics' 'Collections' 'Network-RemoteDirectory' 'Traits' 'Network-URI' 'Settings-Display' 'Collections-Abstract' 'Settings-Compiler' 'MonticelloMocks' 'ScriptLoader11' 'System-Digital Signatures' 'System-Localization' 'MenuRegistration' 'Polymorph-TaskbarIcons' 'Monticello' 'ToolBuilder-Morphic' 'MCDirtyPackage' 'Collections-Weak' 'Settings-Polymorph' 'Tests' 'HelpSystem-Core-Utilities' 'SUnit' 'Collections-Text' 'Compression' 'Morphic-MorphTreeWidget' 'HelpSystem-Core' 'ToolsTest' 'Gofer-Core' 'Gofer-Tests' 'Collections-Stack' 'FreeType' 'MonticelloConfigurations' 'Polymorph-Geometry' 'Collections-Support' 'SUnitGUI' 'HelpSystem-Core-Model' 'Tests-VM' 'System-Change Notification' 'System' 'Tests-ObjectsAsMethods' 'HostMenus' 'Network-MIME' 'Network-RFC822' 'VB-Regex' 'TrueType' 'Depr > ecatedPreferences' 'System-Clipboard' 'System-Settings' 'MorphicTests' 'System-Serial Port' 'NetworkTests' 'Collections-Unordered' 'System-Changes' 'System-Object Storage' 'Collections-Traits' 'System-Platforms' 'Tests-PrimCallController' 'Morphic-TrueType' 'Multilingual' 'Graphics' 'Collections-Strings' 'KernelTests' 'System-FileRegistry' 'Tests-Bugs' 'ToolBuilder-SUnit' 'Tests-Release' 'Morphic-Balloon' 'Settings-Network' 'System-Applications' 'Polymorph-Tools-Diff' 'Settings-Tools' 'ScriptLoader' 'ST80' 'FreeTypeSubPixelAntiAliasing' 'Settings-Kernel' 'Settings-System' 'CompilerTests' 'System-Tools' 'System-Hashing' 'System-Download' 'Collections-Streams' 'Tests-SystemChangeNotification' 'GraphicsTests' 'Polymorph-Widgets' 'Kernel' 'Tools' 'Settings-FreeType' 'FreeTypeTests' 'Settings-Monticello' 'HelpSystem-Tests' 'Polymorph-EventEnhancements' 'System-Support' 'MonticelloGUI' 'System-FilePackage' 'System-Object Events' 'Announcements' 'System-Text' 'Exceptions' 'Network- > UUID' 'System-Pools' 'FixUnderscores' 'HelpSystem-Core-UI' 'GraphicsResources' 'Files' 'Gofer' 'Network-MailSending' 'Collections-Arrayed' 'CollectionsTests' 'Balloon' 'Network-Url' 'Collections-Sequenceable' ). > packagesDev := (PackageInfo allPackages > collect: [:each | each packageName]) > difference: pharoCorePackages. > nonCommentedClasses := packagesDev > inject: 0 > into: [:sum :each | sum + (((PackageInfo named: each) classes > select: [:aClass | aClass comment asString includesSubString: 'is xxxxxxxxx.']) size)]. > Transcript show: 'nonCommentedClasses: ', nonCommentedClasses asString. > > totalClasses := packagesDev inject: 0 into: [:sum :each | sum + (PackageInfo named: each) classes size ]. > Transcript show: 'totalClasses: ', totalClasses asString. > > > > Shows: > > nonCommentedClasses: 1116 > totalClasses: 1369 > > > I'm not sure that I will look at code or answer question to code that is not commented. > We should change our mindset and it seems that we do not care, so we should take radical decisions: > less changes, less progress, more comments. > > And for once I will not bash the past. Smalltalk was always with methods fully documented. > We are just plain lazy and this is a shame. > > Stef > > > > > >

Dec. 7, 2010

Re: [Pharo-project] IRC monthly meeting + project description

by StÃ©phane Ducasse

Ok so at 20h00 we get our first IRC meeting. See U there. > > I suggest that we start also a monthly IRC chat > the first/second tuesday of the month. I propose tuesday 7 at 18h00 or 20h00 Paris time so that > other people can join. > > Le me know what you think about it. > > Stef

Dec. 7, 2010

Re: [Pharo-project] Native Windows in Pharo

by StÃ©phane Ducasse

On Dec 7, 2010, at 2:13 PM, Geert Claes wrote: > > Hi Andres, Ideas and suggestions are very welcome in the Pharo community and > they are mainly discussed here rather than in a UserVoice or GetSatisfaction > environment. Idea voting is an excellent way to engage customers and users > to improve products. The only thing Stef and Doru are saying (at least > thats how I read it) is that Pharo doesn't necessarily have a shortage of > ideas or lack of direction, but simply not enough people to do all the work. :) indeed well rephrased :) > So yes, ideas and feature requests are great and please do continue to post > them here as there are many enthusiastic skilled Smalltalkers that can help, > advice and guide you. When you find something is missing or can be > improved, you are more encouraged to contribute towards the solution rather > than just vote. > > Getting back to your original post though, congrats getting your in-house > project! How about a rich Seaside, AIDAWeb or Iliad browser based app? > > Cheers, Geert > > > > Andres Fortier-2 wrote: >> >> Stef, Doru: >> >> first of all I was *asking if* such a model exists, not demanding one. >> Having just entered the Pharo world I wanted to know how you guys handle >> the feature requests. I see now. >> >> On the other hand if you really want to discuss the question of "who has >> the rights for asking for X" in the open source model I'm open, but just >> to be sure I will first ask: Is it ok to open such a thread? Or should I >> first contribute to Pharo before asking? >> >> AndrÃ©s >> > > -- > View this message in context: http://forum.world.st/Native-Windows-in-Pharo-tp3071794p3076336.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. >

Dec. 7, 2010

[Pharo-project] Why is FileStream writing almost an order of a magnitude slower than reading ?

by Sven Van Caekenberghe

Related to the Matrix CSV input/output optimalization quest, I was puzzled why writing seemed so much slower than reading. Here is a simple example: [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | 100000 timesRepeat: [ stream print: 100 atRandom; space ] ] ] timeToRun. 1558 [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | 100000 timesRepeat: [ Integer readFrom: stream. stream peekFor: $ ] ] ] timeToRun. 183 [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | 100000 timesRepeat: [ stream nextPut: ($a to: $z) atRandom; space ] ] ] timeToRun. 1705 [ FileStream fileNamed: '/tmp/numbers.txt' do: [ :stream | 100000 timesRepeat: [ stream next. stream peekFor: $ ] ] ] timeToRun. 47 Clearly, the writing is close to an order of magnitude slower than reading. This was on Pharo 1.1 with Cog, but I double-checked with Pharo 1.2 and Squeak 4.1. On my machine (Mac Book Pro), this is what another dynamic language does: > (time (with-output-to-file (out "/tmp/numbers.txt") (loop repeat 100000 do (format out "~d " (random 100))))) Timing the evaluation of (WITH-OUTPUT-TO-FILE (OUT "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (FORMAT OUT "~d " (RANDOM 100)))) User time = 0.413 System time = 0.002 Elapsed time = 0.401 Allocation = 2502320 bytes 0 Page faults Calls to %EVAL 1700063 NIL > (time (with-open-file (in "/tmp/numbers.txt") (loop repeat 100000 do (read in)))) Timing the evaluation of (WITH-OPEN-FILE (IN "/tmp/numbers.txt") (LOOP REPEAT 100000 DO (READ IN))) User time = 0.328 System time = 0.001 Elapsed time = 0.315 Allocation = 2500764 bytes 0 Page faults Calls to %EVAL 1400056 NIL So Pharo Smalltalk clearly matches the read/parse speed, which is great, but fails at simple writing. Maybe I am doing something wrong here (I know these are MultiByteFileSteams), but I fail to see what. Something with buffering/flushing ? Anybody any idea ? Sven

Dec. 7, 2010

Re: [Pharo-project] Fwd: [Vm-dev] Image freeze because handleTimerEvent and Seaside process gone?!

by David T. Lewis

The symptoms that I see are intermittent, and it's hard for me to say what the root cause might be. It might be worthwhile to see if you can reproduce my results. If so, it might get you closer to a reproducible test case. What I did was take a Pharo core 1.1, 1.1.1, and 1.2 image, and in each image, I loaded these two packages (ignoring the MVC warnings in CommandShell): http://squeaksource.com/OSProcess/OSProcess-dtl.59.mcz http://squeaksource.com/OSProcess/CommandShell-dtl.49.mcz On Pharo 1.1, I get intermittent failures and errors in the CommandShell and OSProcess tests, but the problems seem to be resolved in 1.1.1 and 1.2. These tests are timing and machine dependent to some extent, so I am not sure if you will see the same symptoms. HTH, Dave On Tue, Dec 07, 2010 at 12:38:54PM +0100, Adrian Lienhard wrote: > The changes between 1.1 and 1.1.1 are the issues in [1]. None seems related... did I miss something? > > One change that I don't understand, although it probably is unrelated, is in [2]: > > LargePositiveInteger removeSelector: #=! > LargePositiveInteger removeSelector: #bitAnd:! > LargePositiveInteger removeSelector: #bitOr:! > LargePositiveInteger removeSelector: #bitShift:! > LargePositiveInteger removeSelector: #bitXor:! > LargePositiveInteger removeSelector: #'~='! > > Why would one want to remove these primitive calls from large integers? > > Cheers, > Adrian > > [1] http://code.google.com/p/pharo/issues/list?can=1&q=Milestone%3D1.1.1&colspe… > [2] http://code.google.com/p/pharo/issues/attachmentText?id=2912&aid=-244293168… > > > On Dec 7, 2010, at 09:52 , Mariano Martinez Peck wrote: > > > ---------- Forwarded message ---------- > > From: David T. Lewis <lewis(a)mail.msen.com> > > Date: Tue, Dec 7, 2010 at 2:06 AM > > Subject: Re: [Vm-dev] Image freeze because handleTimerEvent and Seaside > > process gone?! > > To: Squeak Virtual Machine Development Discussion < > > vm-dev(a)lists.squeakfoundation.org>, adi(a)netstyle.ch > > > > > > > > On Mon, Dec 06, 2010 at 12:33:59PM -0800, Andreas Raab wrote: > >> > >> At a guess, I'd say it's either one of two issues: > >> > >> 1) Your STOP/CONT handling. This sounds suspicious and it could affect > >> the timer handling. I'm assuming that the issue happens after receiving > >> the CONT signal, no? If you can, you might want to a) make sure that you > >> only get the STOP signal when the VM is in ioRelinquish() and not (for > >> example) currently executing the delay process and b) consider to dump > >> the call stacks whenever the VM gets the CONT signal to see what the > >> status is. > >> > >> 2) Some set of incomplete process/delay/semaphore changes in Pharo. One > >> of the problems with processes and delays is that this part of the > >> system reacts very badly to random "cleaning". I.e., changing "foo == > >> nil" to "foo isNil" can have dramatic effects (since it introduces a > >> suspension point) with just the kind of weird issue you're seeing. > > > > Actually #2 does seem like a likely culprit. I found a Pharo 1.1 image > > and loaded the CommandShell and OSProcess test suites. The CommandShell > > tests put a heavy load on process switching, and are rather timing > > dependent. On Pharo 1.1 I get intermittent and non-reproducible errors > > and test failures, and I can't get a clean run of the test suite. The > > errors seem to be different each time. > > > > On Pharo 1.1.1 and 1.2 I can get clean runs of the CommandShell/OSProcess > > tests, so I think there must be some issues in Pharo 1.1. If you are > > using PharoCore 1.1 now and have the option of moving to Pharo 1.1.1 > > or 1.2, I suspect you may see the problems go away. > > > > Dave > > > > > >> > >> With regards to these processes not being printed, that's a side effect > >> of how printAllStacks gathers the processes - it will not print > >> suspended processes which explains why the UI process doesn't print and > >> most likely handleTimerEvent is suspended in a debugger. > >> > >> Depending on how important this issue is you can also try to dissect the > >> object memory itself. If you call writeImageFile (or is it > >> writeImageFileIO?) from gdb it will dump the .image file and you can use > >> the simulator to look at it more closely. Most likely you'll be able to > >> find the processes and look at their stacks. > >> > >> Cheers, > >> - Andreas > >> > >> On 12/6/2010 2:55 AM, Adrian Lienhard wrote: > >>> > >>> Hi all, > >>> > >>> We've been experiencing an "interesting" problem: the image freezes and > >>> does not response to HTTP requests anymore after it has been running for > >>> days. > >>> > >>> Here some basic information about our setup: > >>> > >>> Squeak VM 4.0.3-2202 compiled with gcc 4.3.2 > >>> PharoCore 1.1 > >>> OS Debian Lenny amd64 (CPUs are 4 Intel Xeon E5530 2.40GHz) > >>> > >>> - We have never seen the problem with the Squeak VM 3.9-9 and Squeak 3.9 > >>> on the identical machine and with the same application source (modulo > > some > >>> adaptations to make it run on Pharo). > >>> - We run the VM with -mmap 512m -vm-sound-null -vm-display-null, and the > >>> UI process is suspended (Project uiProcess suspend) > >>> - VM does not hog the CPU and memory usage is normal > >>> - The meantime between failure is several weeks and we haven't managed to > >>> reproduce the problem > >>> - The application mainly serves HTTP requests. When the image does not > >>> receive requests for some time we send it a STOP signal, when a request > >>> comes in it is sent a CONT signal. > >>> - lsof shows > >>> TCP *:9093 (LISTEN) > >>> TCP server:9093->server:46930 (CLOSE_WAIT) > >>> > >>> Below is a GDB backtrace and the Smalltalk stacks from an image that was > >>> frozen (the VM had been running for almost 100 hours): > >>> > >>> ============================================================= > >>> (gdb) bt > >>> #0 0x08072020 in ?? () > >>> #1<signal handler called> > >>> #2 0xb766f5e0 in malloc () from /lib/libc.so.6 > >>> #3<function called from gdb> > >>> #4 0xb76c50c8 in select () from /lib/libc.so.6 > >>> #5 0x08071063 in aioPoll () > >>> #6 0xb778bb8d in ?? () from > > /usr/lib/squeak/4.0.3-2202//so.vm-display-null > >>> #7 0x000003e8 in ?? () > >>> #8 0x997b5a34 in ?? () > >>> #9 0xbfe7cb28 in ?? () > >>> #10 0x08074575 in ioRelinquishProcessorForMicroseconds () > >>> Backtrace stopped: frame did not save the PC > >>> > >>> (gdb) call printCallStack() > >>> -1719969228>idleProcess > >>> -1719969320>startUp > >>> -1740134028 BlockClosure>newProcess > >>> $3 = -1755344892 > >>> > >>> (gdb) call (int) printAllStacks() > >>> Process > >>> -1719969228>idleProcess > >>> -1719969320>startUp > >>> -1740134028 BlockClosure>newProcess > >>> > >>> Process > >>> -1740113860>finalizationProcess > >>> -1740113952>restartFinalizationProcess > >>> -1740113532 BlockClosure>newProcess > >>> > >>> Process > >>> -1740134424 SmalltalkImage>lowSpaceWatcher > >>> -1740134516 SmalltalkImage>installLowSpaceWatcher > >>> -1740134300 BlockClosure>newProcess > >>> > >>> Process > >>> -1719451488 Delay>wait > >>> -1719451580 BlockClosure>ifCurtailed: > >>> -1719451704 Delay>wait > >>> -1719451796 InputEventPollingFetcher>waitForInput > >>> -1740126940 InputEventFetcher>eventLoop > >>> -1740127032 InputEventFetcher>installEventLoop > >>> -1740126816 BlockClosure>newProcess > >>> > >>> Process > >>> -1719557780 UnixOSProcessAccessor>grimReaperProcess > >>> -1740113624 BlockClosure>repeat > >>> -1740113716 UnixOSProcessAccessor>grimReaperProcess > >>> -1740117340 BlockClosure>newProcess > >>> > >>> [omitted many newlines between output above] > >>> ============================================================= > >>> > >>> What is striking from the above process listing is that two processes are > >>> missing: the handleTimerEvent process and the Seaside process (that is, > >>> the TCP listener loop). How comes these processes vanished? > >>> > >>> This may be related to Pharo or to the Squeak VM. > >>> > >>> Has anybody else seen this problem? Any idea how to debug/fix this issue > >>> is very much appreciated! > >>> > >>> Cheers, > >>> Adrian > >>> > >>> > >>> CCed to pharo-dev since this may be related to Pharo; please respond on > >>> the squeak-vm list > >>> > >>> > >>> >

Dec. 7, 2010

Re: [Pharo-project] Native Windows in Pharo

by Igor Stasenko

On 7 December 2010 14:13, Geert Claes <geert.wl.claes(a)gmail.com> wrote: > > Hi Andres, Ideas and suggestions are very welcome in the Pharo community and > they are mainly discussed here rather than in a UserVoice or GetSatisfaction > environment. Â Idea voting is an excellent way to engage customers and users > to improve products. Â The only thing Stef and Doru are saying (at least > thats how I read it) is that Pharo doesn't necessarily have a shortage of > ideas or lack of direction, but simply not enough people to do all the work. > So yes, ideas and feature requests are great and please do continue to post > them here as there are many enthusiastic skilled Smalltalkers that can help, > advice and guide you. Â When you find something is missing or can be > improved, you are more encouraged to contribute towards the solution rather > than just vote. > That's exactly what i said about this before. We simply lacking manhours to do everything we want. I want to do a lot of cool stuff. But it requires a lot of time. > Getting back to your original post though, congrats getting your in-house > project! Â How about a rich Seaside, AIDAWeb or Iliad browser based app? > > Cheers, Geert > > > > Andres Fortier-2 wrote: >> >> Stef, Doru: >> >> first of all I was *asking if* such a model exists, not demanding one. >> Having just entered the Pharo world I wanted to know how you guys handle >> the feature requests. I see now. >> >> On the other hand if you really want to discuss the question of "who has >> the rights for asking for X" in the open source model I'm open, but just >> to be sure I will first ask: Is it ok to open such a thread? Or should I >> first contribute to Pharo before asking? >> >> AndrÃ©s >> > > -- > View this message in context: http://forum.world.st/Native-Windows-in-Pharo-tp3071794p3076336.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > > -- Best regards, Igor Stasenko AKA sig.

Dec. 7, 2010

Re: [Pharo-project] Native Windows in Pharo

by Geert Claes

Hi Andres, Ideas and suggestions are very welcome in the Pharo community and they are mainly discussed here rather than in a UserVoice or GetSatisfaction environment. Idea voting is an excellent way to engage customers and users to improve products. The only thing Stef and Doru are saying (at least thats how I read it) is that Pharo doesn't necessarily have a shortage of ideas or lack of direction, but simply not enough people to do all the work. So yes, ideas and feature requests are great and please do continue to post them here as there are many enthusiastic skilled Smalltalkers that can help, advice and guide you. When you find something is missing or can be improved, you are more encouraged to contribute towards the solution rather than just vote. Getting back to your original post though, congrats getting your in-house project! How about a rich Seaside, AIDAWeb or Iliad browser based app? Cheers, Geert Andres Fortier-2 wrote: > > Stef, Doru: > > first of all I was *asking if* such a model exists, not demanding one. > Having just entered the Pharo world I wanted to know how you guys handle > the feature requests. I see now. > > On the other hand if you really want to discuss the question of "who has > the rights for asking for X" in the open source model I'm open, but just > to be sure I will first ask: Is it ok to open such a thread? Or should I > first contribute to Pharo before asking? > > AndrÃ©s > -- View this message in context: http://forum.world.st/Native-Windows-in-Pharo-tp3071794p3076336.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 7, 2010