October 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] external semaphoresâ¦again

by Norbert Hartl

Am 11.10.2013 um 10:53 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: > > On 11 Oct 2013, at 10:24, Norbert Hartl <norbert(a)hartl.name> wrote: > >> I can report that the behavior is different now. There were two new vm releases this week in ppa. The first one didn't work but the second changed something. My application was never running that long. It is more than a day now having an actual external objects table size of 623 which wasn't ever reached before. So I would say that there is chance that this particular problem is gone. I monitor this further and I think that this wasn't the only problem. But then it is another problem. > > Yeah, but not knowing your application load, 623, which would be about 200 sockets (3 semaphores per sockets), is still a lot to be active at the same time. Can you in some way invoke a full GC externally, like using ZnReadEvalPrintDelegate and see if it eventually drops due to finalization ? It should, at least that is what I see. > Yes, that's what I meant. There is always only one outgoing connection at a time. Every 15 seconds one request is issued. So you see why expect more to find. I'm travelling right now and will have a deeper look after being back Norbert >> Thanks to all of you who've helped solving this. If it comes to the VM being the source of problems it is always extra annoying because it is way harder to change something there. >> >> Norbert >> >> >> Am 08.10.2013 um 11:27 schrieb Igor Stasenko <siguctua(a)gmail.com>: >> >>> >>> >>> >>> On 7 October 2013 18:36, Norbert Hartl <norbert(a)hartl.name> wrote: >>> >>> Am 07.10.2013 um 16:36 schrieb Igor Stasenko <siguctua(a)gmail.com>: >>> >>>> 1 thing. >>>> >>>> can you tell me what given expression yields for your VM/image: >>>> >>>> Smalltalk vm maxExternalSemaphores >>>> >>>> (if it gives you number less than 10000000 then i think i know what is your problem :) >>> It is 10000000 >>> >>> What would be the problem if it would be smaller? >>> >>> >>> that just means your VM don't have external object size cap. >>> I changed the implementation to not have hard limit (the arbitrary large number >>> is there just to be "compatible" with previous implementation). >>> >>> This means, that you can actually change in your image the check and completely ignore limits >>> and just keep growing if it necessary. >>> >>> Now, since you using VM which don't have a limit, but problem still persists, >>> it seems like it somewhere else.. :/ >>>> i just found that after one merge, my changes get lost >>>> we're just plugged them back in, and it should be back again with newer VMs.. >>>> but the problem could be more than just semaphores.. if merge broken this, it may break >>>> many other things, so we need time to check >>> I try to look at it some more time. I'm using the pharo-vm from the launchpad build. Are the changes supposed to be in this one? >>> >>> Norbert >>> >>> Launchpad? You mean ppa? I can't say i remember all the details how changes to VM source >>> gets into ppa distro, and how fast they get there. @Damien, can you enlighten us? >>> >>> >>> Well, the VM which i downloaded recently using zero-conf script, having limit back to 256. Just some merge mistake, which now is fixed.. means that couple builds will use limit-based implementation.. but then >>> it will be back to my implementaiton. >>>> >>>> >>>> >>>> >>>> >>>> On 7 October 2013 12:31, Norbert Hartl <norbert(a)hartl.name> wrote: >>>> >>>> Am 07.10.2013 um 11:28 schrieb Henrik Johansen <henrik.s.johansen(a)veloxit.no>: >>>> >>>>> >>>>> On Oct 7, 2013, at 11:16 , Norbert Hartl <norbert(a)hartl.name> wrote: >>>>> >>>>>> As I need an image that runs longer than 24 hours I'm looking at some stuff and wonder. Can anybody explain me the rationale for a code like this >>>>>> >>>>>> maxExternalSemaphores: aSize >>>>>> "This method should never be called as result of normal program >>>>>> execution. If it is however, handle it differently: >>>>>> - In development, signal an error to promt user to set a bigger size >>>>>> at startup immediately. >>>>>> - In production, accept the cost of potentially unhandled interrupts, >>>>>> but log the action for later review. >>>>>> >>>>>> See comment in maxExternalObjectsSilently: why this behaviour is >>>>>> desirable, " >>>>>> "Can't find a place where development/production is decided. >>>>>> Suggest Smalltalk image inProduction, but use an overridable temp >>>>>> meanwhile. " >>>>>> | inProduction | >>>>>> self maxExternalSemaphores >>>>>> ifNil: [^ 0]. >>>>>> inProduction := false. >>>>>> ^ inProduction >>>>>> ifTrue: [self maxExternalSemaphoresSilently: aSize. >>>>>> self crTrace: 'WARNING: Had to increase size of semaphore signal handling table due to many external objects concurrently in use'; >>>>>> crTrace: 'You should increase this size at startup using #maxExternalObjectsSilently:'; >>>>>> crTrace: 'Current table size: ' , self maxExternalSemaphores printString] >>>>>> ifFalse: ["Smalltalk image" >>>>>> self error: 'Not enough space for external objects, set a larger size at startup!' >>>>>> "Smalltalk image"] >>>>>> >>>>>> I have reported this once but got no feedback so I like to have a few opinions. >>>>>> >>>>>> The report is here: https://pharo.fogbugz.com/f/cases/10839/ >>>>>> >>>>>> Norbert >>>>> >>>>> The rationale is that inProduction would be some global setting, not yet in place when the code was writtenâ¦ >>>>> Excessive simultaneous Semaphore usage is something that should be caught during development, in which case it's better to get an active notification, than having it logged somewhere. >>>> >>>> Agreed. But didn't work in my case because it needed roughly 20 hours and an instable remote backend to trigger the problem. And somehow I forgot to install my logger as Transcript so there is no warning message. I saw only dead images in the morning. >>>> This not satisfactory but on the other hand this type of problems are hard to solve anyway. My feeling tells me there is more to discover. Sockets resources get unregistered at finalization time but this didn't work either. I would have said that the unlikely situation that no garbage collection ran could be the case. But it can't because in ExternalSemaphoreTable>>#freedSlotsIn:ratherThanIncreaseSizeTo: there is explicit garbage collection. >>>> >>>>> If I've understood correctly, it's moot on newer Pharo VM's, where there's no limit on the semtable size, but for legacy code a startup item setting size using maxExternalObjectsSilently: (as suggested in the Warning text), is still a more proper fix than setting inProduction to true and crossing your fingers hoping no signals will be lost during table growth. >>>> >>>> Ah, I didn't know about the risk of loosing signals while resizing the table. Thanks for that. Don't get me wrong I wasn't proposing to set inProduction in effect. I don't think that automatically growing resource management is a proper way to design a system. There is always a range of resources you need for your use case. Not setting an upper bound for this just covers leaking behavior. >>>> >>>> Norbert >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> Best regards, >>>> Igor Stasenko. >>> >>> >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko. > >

Oct. 11, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30472 Home: https://github.com/pharo-project/pharo-core

Oct. 11, 2013

[pharo-project/pharo-core] f2ceb5: 30472

by GitHub

Branch: refs/heads/3.0 Home: https://github.com/pharo-project/pharo-core Commit: f2ceb56e1b987f4e54e27e11c6e53bcdbaa02143 https://github.com/pharo-project/pharo-core/commit/f2ceb56e1b987f4e54e27e11… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2013-10-11 (Fri, 11 Oct 2013) Changed paths: A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/README.md A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/class/class initialization/initialize.st A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/definition.st A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/instance/accessing/fontContents.st A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/instance/accessing/install.st A FreeType-Fonts-SourceCode.package/OpenSansRegular.class/instance/accessing/originalFileName.st A FreeType-Fonts-SourceCode.package/SourceCodeFonts.class/class/accessing/codeFontName.st A FreeType-Fonts-SourceCode.package/SourceCodeFonts.class/class/accessing/defaultFontName.st M FreeType-Fonts-SourceCode.package/SourceCodeFonts.class/class/accessing/setSourceCodeFonts_.st R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/README.md R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/class/class initialization/initialize.st R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/definition.st R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/instance/accessing/fontContents.st R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/instance/accessing/install.st R FreeType-Fonts-SourceCode.package/SourceSansProRegular.class/instance/accessing/originalFileName.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - scripts/script127.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - updates/update30472.st M ScriptLoader30.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st M System-Support.package/SmalltalkImage.class/instance/sources%2C change log/licenseString.st Log Message: ----------- 30472 11839 New L&F for Pharo30 (first iteration, second try) https://pharo.fogbugz.com/f/cases/11839 http://files.pharo.org/image/30/30472.zip

Oct. 11, 2013

Re: [Pharo-dev] Fwd: Small text composition bench

by Igor Stasenko

On 11 October 2013 07:34, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>wrote: > Hi Igor > > it would be good to see how the new text model perform. > > Thanks.. Rough equivalent of it in new text model would be: ======= morph := (TxTextEditorMorph openInWindowWithString: '=Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." Section 1.10.32 of "de Finibus Bonorum et Malorum", written by Cicero in 45 BC =Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." Section 1.10.32 of "de Finibus Bonorum et Malorum", written by Cicero in 45 BC =Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum." Section 1.10.32 of "de Finibus Bonorum et Malorum", written by Cicero in 45 BC ') submorphs last submorphs last. layoutView := morph instVarNamed: #layoutViewMorph. [100 timesRepeat: [ layoutView resetLayout ]] timeToRun 0:00:00:00.006 but that comparison don't have much sense, because the layout & rendering is different. First, because in reality, the layout is not calculated each time when view is rendered, but only when the view changed or text changed. Second, the layout is calculated only for portion of text (big enough to correctly display the text in view), and thus, depending mostly just on view extent, in contrast to old model, which depends mostly on text size itself. The rendering uses precalculated layout and highly depends on what font/resolution etc is used... So, for more fair comparison, i'd better compare the rendering speed, because at the end it is up (or down) to the "how fast you can deliver text on screen". -- Best regards, Igor Stasenko.

Oct. 11, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30471 Home: https://github.com/pharo-project/pharo-core

Oct. 11, 2013

[pharo-project/pharo-core] 408a0d: 30471

by GitHub

Branch: refs/heads/3.0 Home: https://github.com/pharo-project/pharo-core Commit: 408a0d38dbf6238b1cf003a6c7cbe01ffd3b0fdd https://github.com/pharo-project/pharo-core/commit/408a0d38dbf6238b1cf003a6… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2013-10-11 (Fri, 11 Oct 2013) Changed paths: A FileSystem-Core-Kernel.package/Path.class/class/as yet unclassified/cleanUp.st A Fuel-Clusters.package/FLCompiledMethodCluster.class/class/cleanup/cleanUp.st M Kernel-Methods.package/ContextPart.class/class/simulation/initializeTryNamedPrimitiveTemplateMethod.st A Keymapping-Core.package/KMRepository.class/class/cleanup/cleanUp.st A Refactoring-Core-Refactorings.package/RBRefactoring.class/class/as yet unclassified/cleanUp.st A Refactoring-Critics.package/RBTransformationRule.class/class/as yet unclassified/cleanUp.st A Refactoring-Tests-Core-Data.package/RBTransformationRuleTest.class/class/as yet unclassified/cleanUp.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - scripts/script126.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - updates/update30471.st M ScriptLoader30.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A System-Support.package/SystemNavigation.class/instance/query/allMethods.st A Text-Edition.package/TextSelectionColor.class/class/as yet unclassified/cleanUp.st Log Message: ----------- 30471 11847 Some classes hold on to blocks statically: make sure #cleanUp resets those caches https://pharo.fogbugz.com/f/cases/11847 http://files.pharo.org/image/30/30471.zip

Oct. 11, 2013

[regression reporter]regression occurred

by no-reply＠ci.inria.fr

https://ci.inria.fr/pharo/job/Pharo-3.0-Update-Step-2.1-Validation/label=ma… 5 regressions found. Refactoring.Tests.Critics.RBSmalllintTest.testAsOrderedCollectionNotNeeded Refactoring.Tests.Critics.RBSmalllintTest.testAssignmentInIfTrue Refactoring.Tests.Critics.RBSmalllintTest.testAtIfAbsent Refactoring.Tests.Critics.RBSmalllintTest.testCodeCruftLeftInMethods Refactoring.Tests.Critics.RBSmalllintTest.testMinMax

Oct. 11, 2013

[regression reporter]regression occurred

by no-reply＠ci.inria.fr

https://ci.inria.fr/pharo/job/Pharo-3.0-Update-Step-2.1-Validation/label=wi… 6 regressions found. Refactoring.Tests.Critics.RBSmalllintTest.testAsOrderedCollectionNotNeeded Refactoring.Tests.Critics.RBSmalllintTest.testAssignmentInIfTrue Refactoring.Tests.Critics.RBSmalllintTest.testAtIfAbsent Refactoring.Tests.Critics.RBSmalllintTest.testCodeCruftLeftInMethods Refactoring.Tests.Critics.RBSmalllintTest.testMinMax Zinc.Tests.ZnServerTests.testEntityTooLarge

Oct. 11, 2013

[regression reporter]regression occurred

by no-reply＠ci.inria.fr

https://ci.inria.fr/pharo/job/Pharo-3.0-Update-Step-2.1-Validation/label=li… 5 regressions found. Refactoring.Tests.Critics.RBSmalllintTest.testAsOrderedCollectionNotNeeded Refactoring.Tests.Critics.RBSmalllintTest.testAssignmentInIfTrue Refactoring.Tests.Critics.RBSmalllintTest.testAtIfAbsent Refactoring.Tests.Critics.RBSmalllintTest.testCodeCruftLeftInMethods Refactoring.Tests.Critics.RBSmalllintTest.testMinMax

Oct. 11, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30470 Home: https://github.com/pharo-project/pharo-core

Oct. 11, 2013