September 2011 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] [Vm-dev] jenkins cog - slow monticello on osx

by Igor Stasenko

On 22 September 2011 19:32, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Hi, > > I looked a bit into the slowness of the OS X Jenkins Cog VM with Monticello. The problem exists and it is reproducible. > > I performed the following experiments using the following script: > > Time millisecondsToRun: > Â Â Â Â Â Â Â Â [ 1 to: 100 do: [:x | > Â Â Â Â Â Â Â Â Â Â Â Â | window | > Â Â Â Â Â Â Â Â Â Â Â Â window := MCWorkingCopyBrowser new show. > Â Â Â Â Â Â Â Â Â Â Â Â window delete ] ] > > I ran it on the same image using the following vms (on Mac OS X Lion): > - Jenkins VM from 31.08.2011 > - Cog VM r2489 > > I ran the scenario twice with two different setups: > > 1. I placed the image in an empty folder: > > Cog VM 2389: 4808 ms > Jenkins VM: 7345 ms > > 2. I places the image in a folder that already had a large package-cache with 7400 mcz files: > Cog VM: 4867 ms > Jenkins VM: Â Â Â 50754 ms > > > So: > - even if there is no package cache, the Jenkins VM seems to perform 1.5 times slower > - when there is a package-cache, it becomes hardly unusable. > Thanks, Tudor for sticking with it :) so, your measurements telling that its actually could be related more to filesystem interaction(s) rather than to network. we need to ask an experts (Esteban and John) , what are difference between cocoa and carbon versions in file handling, that may cause such big slowdown. > Cheers, > Doru > > > On 8 Sep 2011, at 12:10, Tudor Girba wrote: > >> Hi, >> >> I did not forget about this, but I did not have enough time yet. >> >> Cheers, >> Doru >> >> >> On 26 Aug 2011, at 14:40, Igor Stasenko wrote: >> >>> >>> On 26 August 2011 13:30, Tudor Girba <tudor(a)tudorgirba.com> wrote: >>>> Hi, >>>> >>>> Since a couple of weeks I am working with the Cog VM built on Jenkins: >>>> https://ci.lille.inria.fr/pharo/view/Cog/job/Cog-Mac-Cocoa/ >>>> >>>> Here is a little feedback. It worked quite well, the only problem is that Monticello is really slow in the Moose image. This means that even opening the Monticello Browser takes several seconds. >>>> >>>> I thought that this might be induced by something in the Moose image, but if I try with the Elliot's VM (2489), it works blazingly fast. >>>> >>> >>> Thanks for the Â information, Tudor. >>> Apparently the difference between Eliot's VM and Cog-Mac-Cocoa that >>> they are using different frameworks >>> (Carbon vs Cocoa) >>> and i don't know too much details about the difference there. >>> John or Esteban may have better insights how to fix that. >>> >>>> Cheers, >>>> Doru >>>> >>>> >>>> -- >>>> www.tudorgirba.com >>>> >>>> "The coherence of a trip is given by the clearness of the goal." >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Igor Stasenko AKA sig. >> >> -- >> www.tudorgirba.com >> >> "Yesterday is a fact. >> Tomorrow is a possibility. >> Today is a challenge." >> >> >> > > -- > www.tudorgirba.com > > "Sometimes the best solution is not the best solution." > > -- Best regards, Igor Stasenko.

Sept. 22, 2011

Re: [Pharo-project] Integrating Changes in 1.4 that require a new VM

by Igor Stasenko

On 22 September 2011 19:16, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > (apologies for the duplicate reply; someone needs to sort out their > threading for the benefit of the community ;) ) > > On Thu, Sep 22, 2011 at 2:36 AM, Marcus Denker <marcus.denker(a)inria.fr> > wrote: >> >> Hi, >> >> There are two changesets waiting for integrating in 1.4 that have serious >> consequences: >> >> - Ephemerons. The VM level changes are in the Cog VMs build on Jenkins, >> but have not >> Â been integrated in the VMMaker codebase. >> >> Â Â Â Â http://code.google.com/p/pharo/issues/detail?id=4265 > > I would *really* like to back out these changes. Â The Ephemeron > implementation is very much a prototype, requiring a hack to determine > whether an object is an ephemeron (the presence of a Â marker class in the > first inst var) that I'm not at all happy with. Â There is a neater > implementation available via using an unused instSpec which IMO has > significant advantages (much simpler & faster, instSpec is valid at all > times, including during compaction, less overhead, doesn't require a marker > class), and is the route I'm taking with the new GC/object-representation > I'm working on now. Â Note that other than determining whether an object is > an ephemeron (instSpec/format vs inst var test) the rest of Igor's code > remains the same. Â I'd like to avoid too much VM forking. Â Would you all > consider putting these changes on hold for now? > If so, I'll make the effort to produce prototype changes (in the area of > ClassBuilder and class definition; no VM code necessary as yet) to allow > defining Ephemerons via the int spec route by next week at the latest. > i agree that in my implementation this is a weak point. But its hard to do anything without making changes to object format to identify these special objects. The main story behind this is can we afford to change the internals of VM without being beaten hard by "backwards compatibility" party? :) Ephemerons are versatile way to get notifications of objects which are about to die, and there are certain parts in language which is hard (or even impossible) to implement without ephemerons. I got stuck with it earlier, when realized that we cannot afford to have weak subscriptions in announcement framework for blocks (which is most convenient and most easy way to define subscriptions) without having ephemerons. And of course, by having ephemerons we can completely review the weak finalization scheme and make it much simpler, and faster. I think we should do something in this regard, even at cost of backward compatibility. Because as to me it blocks us from moving forward. I wanted to remind to people, that it took me around a day to implement ephemerons in VM. And then few more days to actually make a correct implementation and write tests to cover it. Unfortunately, we yet don't have a well established process, which could make VM + language side changes to go in sync, when its required, and go much faster and don't fear to introduce/change functionality. One of the reasons for having a continuous integration setup for VM was exactly for that: having new VMs every day (comparing to having new VMs every year). >> >> - Finalization code checks for #hasNewFinalization >> Â This is true in the current VMs build in Jenkins, but in older VMs this >> is not in. >> >> Â Â Â Â http://code.google.com/p/pharo/issues/detail?id=4483 >> >> There are two options: >> >> Â Â Â Â a) integrate in >> Â Â Â Â b) not integrate it >> >> a) means that the image runs on older VMs, too. >> b) means we accept that we can never improve anything for real. >> >> There will be more changes coming... e.g. imagine we have a Vector >> Graphics Canvas >> as some point next year... what will we do? use it or not use it to stay >> compatible? >> >> Â Â Â Â Marcus >> >> >> -- >> Marcus Denker -- http://marcusdenker.de >> >> > > > > -- > best, > Eliot > -- Best regards, Igor Stasenko.

Sept. 22, 2011

[Pharo-project] new Cog VMs available [please read]

by Eliot Miranda

...in http://www.mirandabanda.org/files/Cog/VM/VM.r2493/. These VMs are not much changed from the 2487 VMs, BUT the 2487 VMs introduced a bug while providing faster doesNotUnderstand: processing. These 2493 VMs contain code to help identify that bug. If you're already using a 2487 VM or newer /please/ upgrade to these 2493 VMs asap. I want to see C stack backtraces from these VMs whenever they crash. Typically the VMs produce a crash.dmp file somewhere in the file system (may be / on Mac OS, may be in the directory containing the VM or the current directory on other OSs, but it will be called crash.dmp, and on Mac OS info will be printed to the console or to a crash report window). The C backtrace part of things looks like this on windows: Stack backtrace: [004193EA] ??? + 4297706 in CogCode [00442337] ??? + 4465463 in CogCode [00442685] ??? + 4466309 in CogCode [00583018] ??? + 5779480 in CogCode [0040124B] ??? + 4198987 in CogCode [00401298] ??? + 4199064 in CogCode [7C816FE7] RegisterWaitForInputIdle + 73 in kernel32.dll [7C913BA7] RtlDosApplyFileIsolationRedirection_Ustr + 1824 in ntdll.dll this on Mac C stack backtrace: 0 nsvm 0x00038689 reportStackState + 105 1 nsvm 0x0003893e sigsegv + 110 2 libsystem_c.dylib 0x9b2e259b _sigtramp + 43 3 ??? 0xffffffff 0x0 + 4294967295 4 nsvm 0x0008b2a5 sufficientSpaceAfterGC + 69 5 nsvm 0x0008d11d primitiveNewWithArg + 221 6 ??? 0x068b9899 0x0 + 109811865 7 nsvm 0x00098d10 initStackPagesAndInterpret + 512 8 nsvm 0x0002c530 EventLoopEventHandler + 16 Etc. So please feel free to email me the crash.dmp's and/or the C backtraces of any crashes you have with 2493 VMs. Hopefully I'll track down this regression quickly. For the curious, what's the bug? I don't know yet. The symptom is that very rarely the inline cache linking machinery for MNU PICs relinks the call of an MNU PIC to the address 0x00000013, which causes a crash when the call is next executed, long after the bug bit. These VMs contain code that will raise an error when the relinking attempt is made, which should reveal the bug. Here's the 2493 readme: CogVM binaries as per VMMaker.oscog-eem.125/r2493. Add callsite link/relocate checks to catch the call 0x00000013 MNU callsite relinking bug. Reduce the size of the simStack to something proportional to LargeContextSize. In the Newspeak VM, don't cd to the image's directory on win32. Fix off-by-one error in Win32OSProcessPlugin>primitiveGetCurrentWorkingDirectory so it doesn't include an erroneous trailing null. Fix the 1Gb allocation bug. -- best, Eliot

Sept. 22, 2011

Re: [Pharo-project] [Vm-dev] jenkins cog - slow monticello on osx

by Tudor Girba

Hi, I looked a bit into the slowness of the OS X Jenkins Cog VM with Monticello. The problem exists and it is reproducible. I performed the following experiments using the following script: Time millisecondsToRun: [ 1 to: 100 do: [:x | | window | window := MCWorkingCopyBrowser new show. window delete ] ] I ran it on the same image using the following vms (on Mac OS X Lion): - Jenkins VM from 31.08.2011 - Cog VM r2489 I ran the scenario twice with two different setups: 1. I placed the image in an empty folder: Cog VM 2389: 4808 ms Jenkins VM: 7345 ms 2. I places the image in a folder that already had a large package-cache with 7400 mcz files: Cog VM: 4867 ms Jenkins VM: 50754 ms So: - even if there is no package cache, the Jenkins VM seems to perform 1.5 times slower - when there is a package-cache, it becomes hardly unusable. Cheers, Doru On 8 Sep 2011, at 12:10, Tudor Girba wrote: > Hi, > > I did not forget about this, but I did not have enough time yet. > > Cheers, > Doru > > > On 26 Aug 2011, at 14:40, Igor Stasenko wrote: > >> >> On 26 August 2011 13:30, Tudor Girba <tudor(a)tudorgirba.com> wrote: >>> Hi, >>> >>> Since a couple of weeks I am working with the Cog VM built on Jenkins: >>> https://ci.lille.inria.fr/pharo/view/Cog/job/Cog-Mac-Cocoa/ >>> >>> Here is a little feedback. It worked quite well, the only problem is that Monticello is really slow in the Moose image. This means that even opening the Monticello Browser takes several seconds. >>> >>> I thought that this might be induced by something in the Moose image, but if I try with the Elliot's VM (2489), it works blazingly fast. >>> >> >> Thanks for the information, Tudor. >> Apparently the difference between Eliot's VM and Cog-Mac-Cocoa that >> they are using different frameworks >> (Carbon vs Cocoa) >> and i don't know too much details about the difference there. >> John or Esteban may have better insights how to fix that. >> >>> Cheers, >>> Doru >>> >>> >>> -- >>> www.tudorgirba.com >>> >>> "The coherence of a trip is given by the clearness of the goal." >>> >>> >>> >>> >>> >>> >> >> >> >> -- >> Best regards, >> Igor Stasenko AKA sig. > > -- > www.tudorgirba.com > > "Yesterday is a fact. > Tomorrow is a possibility. > Today is a challenge." > > > -- www.tudorgirba.com "Sometimes the best solution is not the best solution."

Sept. 22, 2011

Re: [Pharo-project] Integrating Changes in 1.4 that require a new VM

by Eliot Miranda

(apologies for the duplicate reply; someone needs to sort out their threading for the benefit of the community ;) ) On Thu, Sep 22, 2011 at 2:36 AM, Marcus Denker <marcus.denker(a)inria.fr>wrote: > Hi, > > There are two changesets waiting for integrating in 1.4 that have serious > consequences: > > - Ephemerons. The VM level changes are in the Cog VMs build on Jenkins, but > have not > been integrated in the VMMaker codebase. > > http://code.google.com/p/pharo/issues/detail?id=4265 I would *really* like to back out these changes. The Ephemeron implementation is very much a prototype, requiring a hack to determine whether an object is an ephemeron (the presence of a marker class in the first inst var) that I'm not at all happy with. There is a neater implementation available via using an unused instSpec which IMO has significant advantages (much simpler & faster, instSpec is valid at all times, including during compaction, less overhead, doesn't require a marker class), and is the route I'm taking with the new GC/object-representation I'm working on now. Note that other than determining whether an object is an ephemeron (instSpec/format vs inst var test) the rest of Igor's code remains the same. I'd like to avoid too much VM forking. Would you all consider putting these changes on hold for now? If so, I'll make the effort to produce prototype changes (in the area of ClassBuilder and class definition; no VM code necessary as yet) to allow defining Ephemerons via the int spec route by next week at the latest. > > > - Finalization code checks for #hasNewFinalization > This is true in the current VMs build in Jenkins, but in older VMs this is > not in. > > http://code.google.com/p/pharo/issues/detail?id=4483 > > There are two options: > > a) integrate in > b) not integrate it > > a) means that the image runs on older VMs, too. > b) means we accept that we can never improve anything for real. > > There will be more changes coming... e.g. imagine we have a Vector Graphics > Canvas > as some point next year... what will we do? use it or not use it to stay > compatible? > > Marcus > > > -- > Marcus Denker -- http://marcusdenker.de > > > -- best, Eliot

Sept. 22, 2011

Re: [Pharo-project] Integrating Changes in 1.4 that require a new VM

by Eliot Miranda

On Thu, Sep 22, 2011 at 6:17 AM, Dimitry Golubovsky <golubovsky(a)gmail.com>wrote: > Hi, > > Markus Denker wrote: > > > ------------------------------------------------------------------------------------------------------------------------------ > > There are two changesets waiting for integrating in 1.4 that have > serious consequences: > > - Ephemerons. The VM level changes are in the Cog VMs build on > Jenkins, but have not > been integrated in the VMMaker codebase. > > http://code.google.com/p/pharo/issues/detail?id=4265 I would *really* like to back out these changes. The Ephemeron implementation is very much a prototype, requiring a hack to determine whether an object is an ephemeron (the presence of a marker class in the first inst var) that I'm not at all happy with. There is a neater implementation available via using an unused instSpec which IMO has significant advantages (much simpler & faster, instSpec is valid at all times, including during compaction, less overhead, doesn't require a marker class), and is the route I'm taking with the new GC/object-representation I'm working on now. Note that other than determining whether an object is an ephemeron (instSpec/format vs inst var test) the rest of Igor's code remains the same. I'd like to avoid too much VM forking. Would you all consider putting these changes on hold for now? If so, I'll make the effort to produce prototype changes (in the area of ClassBuilder and class definition; no VM code necessary as yet) to allow defining Ephemerons via the int spec route by next week at the latest. > > > - Finalization code checks for #hasNewFinalization > This is true in the current VMs build in Jenkins, but in older VMs > this is not in. > > http://code.google.com/p/pharo/issues/detail?id=4483 > > > ------------------------------------------------------------------------------------------------------------------------------ > > Since my Android-related branch has not been integrated into the > mainline: where should I merge this from to be in sync? Is this > back-compatible with Pharo 1.3? > > Thanks. > > -- > Dimitry Golubovsky > > Anywhere on the Web > > -- best, Eliot

Sept. 22, 2011

Re: [Pharo-project] Clarifying and optimizing Integer>>asFloat

by Nicolas Cellier

2011/9/22 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: > Hello nicolas > > > On Sep 22, 2011, at 12:17 AM, Nicolas Cellier wrote: > >> It's too long for a mail, but all explained here: >> http://smallissimo.blogspot.com/2011/09/clarifying-and-optimizing.html >> >> Optimizing such kernel message is important - for example when reading >> a file of floats... >> Which version do you prefer ? >> Shall comments be that long ? > > YESSSSS :) > We love good comments! > And comments should be close to the code else they rot. > There is no problem having long nd precise comments that empower the reader. > Yes, but if you implement any RFC, you generally put a reference because that is too much duplicated information... So there must be a threshold when a link becomes more clever than inlined comments. I'm glad if I'm below this threshold. > >> Any other opinion ? > > One important question is what is the impact of the hardware dirven solution on bit identical computation > on different machine. I would do the following: > Â Â Â Â use the smalltalk - bit identical idea all the times > Â Â Â Â provide one methods for people knowing that they want to get faster but potentially different floats on different platform. > > Does it make sense? Bit identical computations rely on the rounding mode which is set in hardware flags. To my knowledge, all Smalltalk dialects use the default settings and never change it. So the case of asFloat is not different from * + - / and other mathematical functions. They all rely on this setting. I see I introduced a doubt in the blog because I took a perspective of possible future extensions. But my intention was to resolve it; case of bad English style ? Now if some platforms don't adhere to IEEE754, then I think that we must first try to find a compatibility layer at VM level It's not difficult to emulate floating point arithmetic in Smalltalk - see http://smallissimo.blogspot.com/2011/08/arbitraryprecisionfloat.html (which however does not handle overflow, gradual underflow and NaN). However it's hard to do it with decent speeds... Nicolas > > Stef > > >> >> Nicolas >> > > >

Sept. 22, 2011

Re: [Pharo-project] Switching windows using CMD+right/left

by Luc Fabresse

2011/9/22 Douaille Erwan <douailleerwan(a)gmail.com> > Le 22/09/2011 15:50, Benjamin a Ã©crit : > > He guys, >> >> I really would like to know if someone is using the shortcut >> cmd+right/left to switch between windows into Pharo. >> >> Because right now on Mac, this shortcut is used to jump to the begin/end >> of the line, when fn+right/left jump to the begin/end of the file. >> It means that per day, I lost 15 minutes (and my nerves ^^) because in >> eclipse I just jump at the end of the file instead the end of the line ( and >> then I have to scroll back... frustrating), and when I code in pharo, I use >> cmd, and then it swap the window. And the worst part is that cmd+right >> followed by cmd+left do NOT bring you back to the original window, so I am >> lost and it frustrates me a lot ^^ >> >> >> >> So I would like to propose to change the shortcut binding to be a bit >> consistent between pharo and mac (could windows/linux user tell me how it >> works on those platforms ?), and use ctrl+tab to switch between inner >> windows (like in a web browser or cmd+` like in Mac OSX >> >> >> >> >> Please tell me your experience about this, cause >> http://www.dailymotion.com/**video/x4m18_queen-i-m-going-** >> slightly-mad_news<http://www.dailymotion.com/video/x4m18_queen-i-m-going-slightly-mad_news>;) >> >> >> Ben > > > > I'm linux user, and cmd/alt + tab is used for switch beetwen windows > application, so it's already use. > For the moment i'm only beeginner, so i don't really use all keyboard > shortcut like alt + right ;) > > Erwan > > Hi Ben and Erwan, I am on mac I have the same experience as Ben. I was thinking I am more or less the only one annyed by that. Ben, I think that CTRL+tab is not used under linux. Isn't ALT+tab to cylce between windows? Luc

Sept. 22, 2011

Re: [Pharo-project] Switching windows using CMD+right/left

by Douaille Erwan

Le 22/09/2011 15:50, Benjamin a Ã©crit : > He guys, > > I really would like to know if someone is using the shortcut > cmd+right/left to switch between windows into Pharo. > > Because right now on Mac, this shortcut is used to jump to the > begin/end of the line, when fn+right/left jump to the begin/end of the > file. > It means that per day, I lost 15 minutes (and my nerves ^^) because in > eclipse I just jump at the end of the file instead the end of the line > ( and then I have to scroll back... frustrating), and when I code in > pharo, I use cmd, and then it swap the window. And the worst part is > that cmd+right followed by cmd+left do NOT bring you back to the > original window, so I am lost and it frustrates me a lot ^^ > > > > So I would like to propose to change the shortcut binding to be a bit > consistent between pharo and mac (could windows/linux user tell me how > it works on those platforms ?), and use ctrl+tab to switch between > inner windows (like in a web browser or cmd+` like in Mac OSX > > > > > Please tell me your experience about this, cause > http://www.dailymotion.com/video/x4m18_queen-i-m-going-slightly-mad_news ;) > > > Ben I'm linux user, and cmd/alt + tab is used for switch beetwen windows application, so it's already use. For the moment i'm only beeginner, so i don't really use all keyboard shortcut like alt + right ;) Erwan

Sept. 22, 2011

[Pharo-project] Switching windows using CMD+right/left

by Benjamin

He guys, I really would like to know if someone is using the shortcut cmd+right/left to switch between windows into Pharo. Because right now on Mac, this shortcut is used to jump to the begin/end of the line, when fn+right/left jump to the begin/end of the file. It means that per day, I lost 15 minutes (and my nerves ^^) because in eclipse I just jump at the end of the file instead the end of the line ( and then I have to scroll back... frustrating), and when I code in pharo, I use cmd, and then it swap the window. And the worst part is that cmd+right followed by cmd+left do NOT bring you back to the original window, so I am lost and it frustrates me a lot ^^ So I would like to propose to change the shortcut binding to be a bit consistent between pharo and mac (could windows/linux user tell me how it works on those platforms ?), and use ctrl+tab to switch between inner windows (like in a web browser or cmd+` like in Mac OSX Please tell me your experience about this, cause http://www.dailymotion.com/video/x4m18_queen-i-m-going-slightly-mad_news ;) Ben

Sept. 22, 2011