- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] external semaphoresâ¦again

by phil＠highoctane.be

I do have a version #20619 with a VM from get.pharo.org (built Oct 2, jenkins #14731) which gives me 256... Is it bad Mr Stasenko ? (maybe no wonder I had socket issues when scraping a lot of things) Phil On Tue, Oct 8, 2013 at 11:27 AM, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 7 October 2013 18:36, Norbert Hartl <norbert(a)hartl.name> wrote: > >> >> Am 07.10.2013 um 16:36 schrieb Igor Stasenko <siguctua(a)gmail.com>: >> >> 1 thing. >> >> can you tell me what given expression yields for your VM/image: >> >> Smalltalk vm maxExternalSemaphores >> >> (if it gives you number less than 10000000 then i think i know what is >> your problem :) >> >> It is 10000000 >> >> What would be the problem if it would be smaller? >> >> > that just means your VM don't have external object size cap. > I changed the implementation to not have hard limit (the arbitrary large > number > is there just to be "compatible" with previous implementation). > > This means, that you can actually change in your image the check and > completely ignore limits > and just keep growing if it necessary. > > Now, since you using VM which don't have a limit, but problem still > persists, > it seems like it somewhere else.. :/ > > i just found that after one merge, my changes get lost > we're just plugged them back in, and it should be back again with newer > VMs.. > but the problem could be more than just semaphores.. if merge broken this, > it may break > many other things, so we need time to check > > I try to look at it some more time. I'm using the pharo-vm from the > launchpad build. Are the changes supposed to be in this one? > > Norbert > > Launchpad? You mean ppa? I can't say i remember all the details how > changes to VM source > gets into ppa distro, and how fast they get there. @Damien, can you > enlighten us? > > > Well, the VM which i downloaded recently using zero-conf script, having > limit back to 256. Just some merge mistake, which now is fixed.. means that > couple builds will use limit-based implementation.. but then > it will be back to my implementaiton. > > > > > > > On 7 October 2013 12:31, Norbert Hartl <norbert(a)hartl.name> wrote: > >> >> Am 07.10.2013 um 11:28 schrieb Henrik Johansen < >> henrik.s.johansen(a)veloxit.no>: >> >> >> On Oct 7, 2013, at 11:16 , Norbert Hartl <norbert(a)hartl.name> wrote: >> >> As I need an image that runs longer than 24 hours I'm looking at some >> stuff and wonder. Can anybody explain me the rationale for a code like this >> >> maxExternalSemaphores: aSize >> "This method should never be called as result of normal program >> execution. If it is however, handle it differently: >> - In development, signal an error to promt user to set a bigger size >> at startup immediately. >> - In production, accept the cost of potentially unhandled interrupts, >> but log the action for later review. >> See comment in maxExternalObjectsSilently: why this behaviour is >> desirable, " >> "Can't find a place where development/production is decided. >> Suggest Smalltalk image inProduction, but use an overridable temp >> meanwhile. " >> | inProduction | >> self maxExternalSemaphores >> ifNil: [^ 0]. >> inProduction := false. >> ^ inProduction >> ifTrue: [self maxExternalSemaphoresSilently: aSize. >> self crTrace: 'WARNING: Had to increase size of semaphore signal >> handling table due to many external objects concurrently in use'; >> crTrace: 'You should increase this size at startup using >> #maxExternalObjectsSilently:'; >> crTrace: 'Current table size: ' , self maxExternalSemaphores >> printString] >> ifFalse: ["Smalltalk image" >> self error: 'Not enough space for external objects, set a larger size >> at startup!' >> "Smalltalk image"] >> >> I have reported this once but got no feedback so I like to have a few >> opinions. >> >> The report is here: https://pharo.fogbugz.com/f/cases/10839/ >> >> Norbert >> >> >> The rationale is that inProduction would be some global setting, not yet >> in place when the code was writtenâ¦ >> Excessive simultaneous Semaphore usage is something that should be caught >> during development, in which case it's better to get an active >> notification, than having it logged somewhere. >> >> >> Agreed. But didn't work in my case because it needed roughly 20 hours and >> an instable remote backend to trigger the problem. And somehow I forgot to >> install my logger as Transcript so there is no warning message. I saw only >> dead images in the morning. >> This not satisfactory but on the other hand this type of problems are >> hard to solve anyway. My feeling tells me there is more to discover. >> Sockets resources get unregistered at finalization time but this didn't >> work either. I would have said that the unlikely situation that no garbage >> collection ran could be the case. But it can't because in >> ExternalSemaphoreTable>>#freedSlotsIn:ratherThanIncreaseSizeTo: there is >> explicit garbage collection. >> >> If I've understood correctly, it's moot on newer Pharo VM's, where >> there's no limit on the semtable size, but for legacy code a startup item >> setting size using maxExternalObjectsSilently: (as suggested in the Warning >> text), is still a more proper fix than setting inProduction to true and >> crossing your fingers hoping no signals will be lost during table growth. >> >> >> Ah, I didn't know about the risk of loosing signals while resizing the >> table. Thanks for that. Don't get me wrong I wasn't proposing to set >> inProduction in effect. I don't think that automatically growing resource >> management is a proper way to design a system. There is always a range of >> resources you need for your use case. Not setting an upper bound for this >> just covers leaking behavior. >> >> Norbert >> >> >> > > > -- > Best regards, > Igor Stasenko. > > > > > > -- > Best regards, > Igor Stasenko. >

Oct. 8, 2013

Re: [Pharo-dev] Please try to reproduce this bug

by phil＠highoctane.be

BTW, I had issues w/ ppa this week end during a Hackathon. I wanted my teammate to install pharo on his Wheezy 7 Debian and it failed. Now, we had to enable multiarch to get things running. Pretty out of reach for basic users. As I had to put stuff on a Wheezy 7 server down the line, the get.pharo.orgthing worked better as I had no admin rights. Phil On Tue, Oct 8, 2013 at 9:18 AM, Damien Cassou <damien.cassou(a)gmail.com>wrote: > On Mon, Oct 7, 2013 at 11:21 AM, Norbert Hartl <norbert(a)hartl.name> wrote: > > I have a lot of issues that this or like this. I get the sources problem > as well as images I work for some time tell me that my working directory > doesn't exist. I need to save and reopen the image to be able to do > anything. > > > do you use the Ubuntu PPA? > > -- > Damien Cassou > http://damiencassou.seasidehosting.st > > "Success is the ability to go from one failure to another without > losing enthusiasm." > Winston Churchill > >

Oct. 8, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30462 Home: https://github.com/pharo-project/pharo-core

Oct. 8, 2013

[pharo-project/pharo-core] 603f9e: 30462

by GitHub

Branch: refs/heads/3.0 Home: https://github.com/pharo-project/pharo-core Commit: 603f9ef63f3e8bcf227e4548defed140208becdb https://github.com/pharo-project/pharo-core/commit/603f9ef63f3e8bcf227e4548… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2013-10-08 (Tue, 08 Oct 2013) Changed paths: M FileSystem-Memory.package/MemoryFileSystemDirectory.class/instance/enumeration/fileEntriesDo_.st M Ring-Core-Kernel.package/RGBehaviorDefinition.class/instance/testing/includesProtocol_.st M Ring-Core-Kernel.package/RGClassDescriptionDefinition.class/instance/testing/isSameRevisionAs_.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - scripts/script117.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - updates/update30462.st M ScriptLoader30.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st M Spec-Inspector.package/EyeDictionaryInspector.class/instance/accessing/sortedKeys.st M System-Support.package/SystemDictionary.class/instance/class and trait names/classAndTraitNames.st M Tests-Traits.package/TraitPureBehaviorTest.class/instance/testing-applying trait composition/testChangeSuperclass.st M Tools-Search.package/SearchFacade.class/class/package search/packageSearchIn_.st M Tools-Search.package/SearchFacade.class/class/protocol search/protocolSearchIn_.st Log Message: ----------- 30462 11812 replace some #sorted with #sort https://pharo.fogbugz.com/f/cases/11812 11813 Do not bother to sort #classAndTraitNames https://pharo.fogbugz.com/f/cases/11813 11814 remove empty package NewClassOrganizer and overlapping Tabs-Support https://pharo.fogbugz.com/f/cases/11814 11815 includesProtocol: should use #includes: https://pharo.fogbugz.com/f/cases/11815

Oct. 8, 2013

Re: [Pharo-dev] external semaphoresâ¦again

by Igor Stasenko

On 7 October 2013 18:36, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 07.10.2013 um 16:36 schrieb Igor Stasenko <siguctua(a)gmail.com>: > > 1 thing. > > can you tell me what given expression yields for your VM/image: > > Smalltalk vm maxExternalSemaphores > > (if it gives you number less than 10000000 then i think i know what is > your problem :) > > It is 10000000 > > What would be the problem if it would be smaller? > > that just means your VM don't have external object size cap. I changed the implementation to not have hard limit (the arbitrary large number is there just to be "compatible" with previous implementation). This means, that you can actually change in your image the check and completely ignore limits and just keep growing if it necessary. Now, since you using VM which don't have a limit, but problem still persists, it seems like it somewhere else.. :/ i just found that after one merge, my changes get lost we're just plugged them back in, and it should be back again with newer VMs.. but the problem could be more than just semaphores.. if merge broken this, it may break many other things, so we need time to check I try to look at it some more time. I'm using the pharo-vm from the launchpad build. Are the changes supposed to be in this one? Norbert Launchpad? You mean ppa? I can't say i remember all the details how changes to VM source gets into ppa distro, and how fast they get there. @Damien, can you enlighten us? Well, the VM which i downloaded recently using zero-conf script, having limit back to 256. Just some merge mistake, which now is fixed.. means that couple builds will use limit-based implementation.. but then it will be back to my implementaiton. On 7 October 2013 12:31, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 07.10.2013 um 11:28 schrieb Henrik Johansen < > henrik.s.johansen(a)veloxit.no>: > > > On Oct 7, 2013, at 11:16 , Norbert Hartl <norbert(a)hartl.name> wrote: > > As I need an image that runs longer than 24 hours I'm looking at some > stuff and wonder. Can anybody explain me the rationale for a code like this > > maxExternalSemaphores: aSize > "This method should never be called as result of normal program > execution. If it is however, handle it differently: > - In development, signal an error to promt user to set a bigger size > at startup immediately. > - In production, accept the cost of potentially unhandled interrupts, > but log the action for later review. > See comment in maxExternalObjectsSilently: why this behaviour is > desirable, " > "Can't find a place where development/production is decided. > Suggest Smalltalk image inProduction, but use an overridable temp > meanwhile. " > | inProduction | > self maxExternalSemaphores > ifNil: [^ 0]. > inProduction := false. > ^ inProduction > ifTrue: [self maxExternalSemaphoresSilently: aSize. > self crTrace: 'WARNING: Had to increase size of semaphore signal > handling table due to many external objects concurrently in use'; > crTrace: 'You should increase this size at startup using > #maxExternalObjectsSilently:'; > crTrace: 'Current table size: ' , self maxExternalSemaphores printString] > ifFalse: ["Smalltalk image" > self error: 'Not enough space for external objects, set a larger size at > startup!' > "Smalltalk image"] > > I have reported this once but got no feedback so I like to have a few > opinions. > > The report is here: https://pharo.fogbugz.com/f/cases/10839/ > > Norbert > > > The rationale is that inProduction would be some global setting, not yet > in place when the code was writtenâ¦ > Excessive simultaneous Semaphore usage is something that should be caught > during development, in which case it's better to get an active > notification, than having it logged somewhere. > > > Agreed. But didn't work in my case because it needed roughly 20 hours and > an instable remote backend to trigger the problem. And somehow I forgot to > install my logger as Transcript so there is no warning message. I saw only > dead images in the morning. > This not satisfactory but on the other hand this type of problems are hard > to solve anyway. My feeling tells me there is more to discover. Sockets > resources get unregistered at finalization time but this didn't work > either. I would have said that the unlikely situation that no garbage > collection ran could be the case. But it can't because in > ExternalSemaphoreTable>>#freedSlotsIn:ratherThanIncreaseSizeTo: there is > explicit garbage collection. > > If I've understood correctly, it's moot on newer Pharo VM's, where there's > no limit on the semtable size, but for legacy code a startup item setting > size using maxExternalObjectsSilently: (as suggested in the Warning text), > is still a more proper fix than setting inProduction to true and crossing > your fingers hoping no signals will be lost during table growth. > > > Ah, I didn't know about the risk of loosing signals while resizing the > table. Thanks for that. Don't get me wrong I wasn't proposing to set > inProduction in effect. I don't think that automatically growing resource > management is a proper way to design a system. There is always a range of > resources you need for your use case. Not setting an upper bound for this > just covers leaking behavior. > > Norbert > > > -- Best regards, Igor Stasenko. -- Best regards, Igor Stasenko.

Oct. 8, 2013

[regression reporter]regression occurred

by no-reply＠ci.inria.fr

https://ci.inria.fr/pharo/job/Pharo-3.0-Update-Step-2.1-Validation/label=ma… 1 regressions found. Tests.Traits.TraitPureBehaviorTest.testChangeSuperclass

Oct. 8, 2013

[regression reporter]regression occurred

by no-reply＠ci.inria.fr

https://ci.inria.fr/pharo/job/Pharo-3.0-Update-Step-2.1-Validation/label=li… 1 regressions found. Tests.Traits.TraitPureBehaviorTest.testChangeSuperclass

Oct. 8, 2013

Re: [Pharo-dev] external semaphoresâ¦again

by Igor Stasenko

On 7 October 2013 18:01, Henrik Johansen <henrik.s.johansen(a)veloxit.no>wrote: > > On Oct 7, 2013, at 5:40 , Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > > On 7 October 2013 17:25, Henrik Johansen <henrik.s.johansen(a)veloxit.no>wrote: > >> >> On Oct 7, 2013, at 4:36 , Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> > 1 thing. >> > >> > can you tell me what given expression yields for your VM/image: >> > >> > Smalltalk vm maxExternalSemaphores >> > >> > (if it gives you number less than 10000000 then i think i know what is >> your problem :) >> > >> >> Uhm, I didn't read your implementation, but the proper thing to do for a >> VM which doesn't care about the external objects table (splObjs at: 39) >> growing, is returning nil from maxExternalSemaphores, like it will for >> pre-Cog VM's. >> > > or.. release VM + image, where image aware of current VM capabilities, and > refuses to run on older ones, and get rid of all this stuff altogether. > > i changed the low-level implementation (C level), but not primitive level. > sure thing more cleanup & polishing can be done, > but the changes i made never get reviewed and properly tested (though, > there was no problems reported since year from when it was introduced, > except from last unfortunate rollback to Eliot's original implementation). > > > I know, which is why I thought you might like the feedback :P > AFAICT, it would be nice to couple the changes you made with changing > parameterAt: 49 back to returning nil, as well as returning an error for > parameterAt: 49 put: that the section is not in use/obsolete, instead of > returning bogus values. > > Though, sounds a bit strange how parameterAt: 49 can return a huge number > without any changes to the related code (which would then still be reading > from a 16bit field :P) > > > > >> Otherwise, you'll get a domain error when you eventually (yes,yes, it'll >> take awhile) run out, and maxExternalSemaphoresSilently: tries to set the >> new max to a value that won't fit in the 16bit header field where it >> expects to put itâ¦ >> >> yes.. theoretically.. but i have gut feeling that your image/setup will > experience a lot of other different problems (memory consumption/amount of > available OS handles etc), before you get to that point. And at that point, > you will need to modify VM/OS/image anyways (or figure how to work around > limits in other way).. > > > > Yap, I guess you could say changing to a nil return would be more a change > to play by established conventions when reporting capabilities/needs of the > VM than anything with practical impact. > > i agree. i will create a bug entry for it then.. > > Cheers, > Henry > -- Best regards, Igor Stasenko.

Oct. 8, 2013

Re: [Pharo-dev] Commit right on github.com/pharo-project/pharo-vm/

by Camillo Bruni

can you remake the pull request on the development branch? I changed the branches and added travis in the meantime. sorry for the inconvenience :( On 2013-10-08, at 10:19, Luc Fabresse <luc.fabresse(a)gmail.com> wrote: > Hi all, > > I made a pull request on github a week ago on the pharo-vm repo. > https://github.com/pharo-project/pharo-vm/pulls > > Can someone give me commit rights there? > > Bug entry is there: https://pharo.fogbugz.com/f/cases/11698/Serial-Plugin-primitives-ByName > > Cheers, > > #Luc

Oct. 8, 2013

[Issue Tracker] 129 issues tagged 3.0

by Marcus Denker

Hello, There are 129 issues that need to be fixed before we can even think about releasing Pharo3: https://pharo.fogbugz.com/f/filters/37/3-0-Work-Needed There are two options: 1) fix all of them. (unrealistic) 2) decided if all of them are real show stoppers, lower it and fix this hopefully smaller number actively. I know people don't like this option 2), but I do not see any realistic alternative. Marcus

Oct. 8, 2013