- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Green tests

by ducasse

> On 2 Apr 2019, at 17:40, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: > > > > On Tue, Apr 2, 2019 at 5:31 PM ducasse <stepharo(a)netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: > Cool and thanks! > I love this green bar and I will try to keep it green. > Do you know if the test writing by pierre should be pushed in Pharo because I have the impression that they will be red first > > Short answer: no :). > Long answer: > I've made a review and requested changes. The issue is related to the need of the new VM. > The main problem is that the only way to test it properly is to open a file, closing it and ask your system if the file is still open (which is OS dependent e.g., calling lsof on unix/osx, or the equivalent on windows). > The second better test we can do is to try to open as many files as the limit and check that we cannot open more files or that we do not segfault :) but it's not super fun when debugging. > Maybe we can just close/ignore that PR? Just to verify we are talking about: https://github.com/pharo-project/pharo/pull/3100 <https://github.com/pharo-project/pharo/pull/3100>Then I will close it. Stef > >> Hi all, >> >> In a couple of minutes I've made some (not complicated) fixes to make Rubric tests green again: >> https://github.com/pharo-project/pharo/pull/3129 <https://github.com/pharo-project/pharo/pull/3129> > > I will check them. > We got some tests for maxlength and started with the stupid point. > > Yeah, we have to fix some ugly stuff on it (the centered ghost text, the max length...), but there is "no rush" :). >> >> This is of course not a super Rubric enhancement (for which I've opened another issue requesting some missing feature https://github.com/pharo-project/pharo/issues/3128 <https://github.com/pharo-project/pharo/issues/3128>) but this allows us to move on with a better infrastructure :). >> >> There are still some some tests that fail from time to time, related to system announcers and finalization. See https://github.com/pharo-project/pharo/issues/2471 <https://github.com/pharo-project/pharo/issues/2471>. >> Though I've spent half a day yesterday and half today trying to track it down, I don't yet have a clear way of reproducing it. I've read the entire code that touches this part of the system and I really bet this is a race condition :). >> So any second pair of eyes here would really be helpful! > > Iâm inherently race condition incompatible and just avoid concurrent programming because Iâm too dull on this. > > Even funnier is that I think the race condition happens during the stage loading BaselineOfPharo. It's a super huge step, and we need better debugging tools for that kind of scenarios... > > >> Guille > > > > -- > > Guille Polito > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> > > Web: http://guillep.github.io <http://guillep.github.io/> > Phone: +33 06 52 70 66 13

April 2, 2019

Re: [Pharo-dev] Green tests

by ducasse

And this was not just for fun that tests were broken with rubric. First there was no tests when Alain fixed rubric Second some features were bad so not copied and pasted to Rubric. Life is complex. And we try our best. But I did not want to say to Alain: âtoo bad you work a lot but now we do not careâ because we care a lot about other contributions. Stef > On 2 Apr 2019, at 17:43, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > > > On Tue, Apr 2, 2019 at 5:36 PM Marcus Denker <marcus.denker(a)inria.fr <mailto:marcus.denker@inria.fr>> wrote: > > > > On 2 Apr 2019, at 17:30, ducasse <stepharo(a)netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: > > > > Cool and thanks! > > I love this green bar and I will try to keep it green. > > Do you know if the test writing by pierre should be pushed in Pharo because I have the impression that they will be red first > > > If we add failing tests, then *everyone* is impacted. Instead of âah, greenâ we will again have to manually check > *every* *single* Pull request. And we, again, will forget to do it. And thus break even more. > > I really think keeping the tests green is very, very much better for everyone. > > Yes Marcus, but there is another "hidden" point in my mail: my fix was really simple, anybody could have done it. > So > - just complaining does not fix the build. > - removing breaking tests will not guarantee us a better quality either > > > Marcus > > > > > -- > > Guille Polito > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> > > Web: http://guillep.github.io <http://guillep.github.io/> > Phone: +33 06 52 70 66 13

April 2, 2019

Re: [Pharo-dev] New stable VM for Pharo8?

by ducasse

Ok thanks ronie. This is why the headless VM is on P8 roadmap. I think that Esteban will start to work on it in May. But he can confirm his agenda. Stef > On 2 Apr 2019, at 17:46, Ronie Salgado <roniesalg(a)gmail.com> wrote: > > Hi, > > I thought that plugin was a plugin, i.e., optional but it looks like it is not. :( > The plugin itself is optional, but it depends on code being exposed from the VM windowing system in order to blit a final rendered rectangle on top the VM window. Since Apple is deprecating OpenGL, then supporting Metal (or another Apple supported alternative) is a necessity. I am more of a fan of the no VM window solution. > > My biggest problem with Metal is on testing with older versions of OS X, and with older versions of the hardware itself. It works perfectly on my Macbook air Early 2015 (The VM, and Woden). But it does not work for example on Alexandre Bergel older desktop machine (black windows with Woden). Tomorrow I will be debugging Woden on Alexandre machine, and I will take some time on trying to debug this VM problem, if I can reproduce it. > > The way of debugging a Metal application is by running it with the XCode debugger, enabling GPU debugging, and capturing some frames in the debugger that which allows to inspect the draw calls and the resources that are used for generating the frame. > > Best regards, > Ronie > > El mar., 2 abr. 2019 a las 12:28, ducasse (<stepharo(a)netcourrier.com <mailto:stepharo@netcourrier.com>>) escribiÃ³: > Thanks ronie for the explanation. > I thought that plugin was a plugin, i.e., optional but it looks like it is not. :( > When I read your mail, it looks like we have to problem with a plugin we do not use. Even better :( > > May be Pharo should have a branch so that we can promote a Pharo stable VM. > I feel uncomfortable to have to remember which commits is the stable one for us because > how can we progress there? I do not see how we can jump over commits. > > And my concern is how can we improve the time you and Pablo and Guille are spending > handling such case. This is ok to look for a bug and fix, now we will have to put in place a process to handle > regressions and unsure that we can blessed a VM a stable for real. > > We should discuss it during PharoDays consortium meeting and the next consortium online meeting. > > Stef > >> On 2 Apr 2019, at 16:56, Ronie Salgado <roniesalg(a)gmail.com <mailto:roniesalg@gmail.com>> wrote: >> >> Hi, >> >> For a job I have to adapt old Squeak code (B3DAcceleratorPlugin) to newer graphics APIs (Metal and D3D11). The VM window has to use the same rendering interface as the plugin for rendering. Recently for trying to solve the issue with the older machines I implemented support selecting the older renderers with a command line option, and I added code for trying to detect whether the current Metal implementation is valid, or not. These command line options are the followings (can be obtained by using -help): >> >> -metal Use metal if the Metal detection code works. >> -opengl Use OpenGL. The previous default. >> -core-graphics Use the older CoreGraphics based backend. >> -headless Null without rendering >> >> This command line option has to be the first command line option passed to the VM program. This is because the normal command line parsing is done after creating the VM window. As a workaround I am parsing the command line option on the main() function as the first command line argument. >> >> On my Mac machine I have Mojave installed. If someone could give pointers on how to setup a VM with the older version, or a way to emulate the older version I could be able to debug the problem. >> >> Perhaps as a workaround, for Pharo we should disable the Metal support (Remove the -DUSE_METAL from the makefile), and maybe go for the CoreGraphics backend, which seems to be better tested and I think does not give conflict problems when trying to use OpenGL from the image side. In Pharo we do not use the B3DAcceleratorPlugin, which in my opinion is a complete mess. >> >> Best regards, >> Ronie >> >> >> El mar., 2 abr. 2019 a las 10:58, Guillermo Polito (<guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>>) escribiÃ³: >> >> >> On Tue, Apr 2, 2019 at 2:11 PM ducasse <stepharo(a)netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: >> Thanks guille and pablo for you dedication for Pharo, I love you :) >> >> Now just that I understand, does it mean that the latest open smalltalk vm does not run on anything else that Mojave and High Sierra? >> >> I don't know exactly on what versions it is not working because I have Windows10 and Mojave. >> But I know that it's not working for Pablo on OSX Sierra (which dates from 2016...). >> >> AFAIU, Ronie has updated the VM's code to render on Metal, which made that at some point the VM was not compiling for latest versions of OSX. >> In these last weeks Ronie has also updated the code to make it compile in older machines, but still in those versions that not support Metal the main window just renders a black screen. >> >> I don't know what are the following steps regarding this, I put Ronie in cc, he may clarify a bit the situation. >> In the meanwhile, using the build I say in the email above may suffice us all I think ^^. >> >> >> Stef (back writing research proposalâ¦..) >> >>> On 2 Apr 2019, at 13:40, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: >>> >>> Hi all, Esteban, >>> >>> In the last sprint with Pablo and Pierre we have detected several misbehaviours with files. >>> The issue was particular reproducible when trying to run iceberg tests: the VM kind of ran out of slots for opening files. The strangest thing is that the leaks seemed to happen when deleting directories, so we had a look at the changes done to iterate directories, get file attributes and so on. >>> >>> While on the FileReference code everything seemed ok, we have observerd with `lsof` that still several directories remained open even after deleting them. >>> >>> Latest VM fixes this issue but contains another one related to Metal rendering, which prevents rendering in OSX versions < than High Sierra. >>> >>> Tracking the issue a bit, this seems to be a bug in the FileAttributes plugin that was fixed by this PR by alistair (https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2… <https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2…>). >>> >>> We have tested that particular commit (a838346b1a67712cc28298534dafbd0c26ea34fb <https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2…>) and this one contains Alistair's fixes but not yet the new rendering using Metal. We have been testing that VM since yesterday and it seems stable, what do you think about marking it stable for Pharo8? >>> >>> Regarding Pharo7, Pharo7's stable VM is the same as Pharo8's, so it contains the same issue. However, since on the image side the code does not use the problematic primitives, we do not think we need to change the VM for Pharo7 (yet). However, there is seemingly a fix for directories containing blanks, maybe Alistair can give us some input here? :) >>> >>> Ideas? >>> >>> Thanks, >>> Guille and Pablo >> >> >> >> -- >> >> Guille Polito >> Research Engineer >> >> Centre de Recherche en Informatique, Signal et Automatique de Lille >> CRIStAL - UMR 9189 >> French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> >> >> Web: http://guillep.github.io <http://guillep.github.io/> >> Phone: +33 06 52 70 66 13 >

April 2, 2019

Re: [Pharo-dev] New stable VM for Pharo8?

by Ronie Salgado

Hi, > I thought that plugin was a plugin, i.e., optional but it looks like it is > not. :( The plugin itself is optional, but it depends on code being exposed from the VM windowing system in order to blit a final rendered rectangle on top the VM window. Since Apple is deprecating OpenGL, then supporting Metal (or another Apple supported alternative) is a necessity. I am more of a fan of the no VM window solution. My biggest problem with Metal is on testing with older versions of OS X, and with older versions of the hardware itself. It works perfectly on my Macbook air Early 2015 (The VM, and Woden). But it does not work for example on Alexandre Bergel older desktop machine (black windows with Woden). Tomorrow I will be debugging Woden on Alexandre machine, and I will take some time on trying to debug this VM problem, if I can reproduce it. The way of debugging a Metal application is by running it with the XCode debugger, enabling GPU debugging, and capturing some frames in the debugger that which allows to inspect the draw calls and the resources that are used for generating the frame. Best regards, Ronie El mar., 2 abr. 2019 a las 12:28, ducasse (<stepharo(a)netcourrier.com>) escribiÃ³: > Thanks ronie for the explanation. > I thought that plugin was a plugin, i.e., optional but it looks like it is > not. :( > When I read your mail, it looks like we have to problem with a plugin we > do not use. Even better :( > > May be Pharo should have a branch so that we can promote a Pharo stable > VM. > I feel uncomfortable to have to remember which commits is the stable one > for us because > how can we progress there? I do not see how we can jump over commits. > > And my concern is how can we improve the time you and Pablo and Guille are > spending > handling such case. This is ok to look for a bug and fix, now we will have > to put in place a process to handle > regressions and unsure that we can blessed a VM a stable for real. > > We should discuss it during PharoDays consortium meeting and the next > consortium online meeting. > > Stef > > On 2 Apr 2019, at 16:56, Ronie Salgado <roniesalg(a)gmail.com> wrote: > > Hi, > > For a job I have to adapt old Squeak code (B3DAcceleratorPlugin) to newer > graphics APIs (Metal and D3D11). The VM window has to use the same > rendering interface as the plugin for rendering. Recently for trying to > solve the issue with the older machines I implemented support selecting the > older renderers with a command line option, and I added code for trying to > detect whether the current Metal implementation is valid, or not. These > command line options are the followings (can be obtained by using -help): > > -metal Use metal if the Metal detection code works. > -opengl Use OpenGL. The previous default. > -core-graphics Use the older CoreGraphics based backend. > -headless Null without rendering > > This command line option has to be the first command line option passed to > the VM program. This is because the normal command line parsing is done > after creating the VM window. As a workaround I am parsing the command line > option on the main() function as the first command line argument. > > On my Mac machine I have Mojave installed. If someone could give pointers > on how to setup a VM with the older version, or a way to emulate the older > version I could be able to debug the problem. > > Perhaps as a workaround, for Pharo we should disable the Metal support > (Remove the -DUSE_METAL from the makefile), and maybe go for the > CoreGraphics backend, which seems to be better tested and I think does not > give conflict problems when trying to use OpenGL from the image side. In > Pharo we do not use the B3DAcceleratorPlugin, which in my opinion is a > complete mess. > > Best regards, > Ronie > > > El mar., 2 abr. 2019 a las 10:58, Guillermo Polito (< > guillermopolito(a)gmail.com>) escribiÃ³: > >> >> >> On Tue, Apr 2, 2019 at 2:11 PM ducasse <stepharo(a)netcourrier.com> wrote: >> >>> Thanks guille and pablo for you dedication for Pharo, I love you :) >>> >>> Now just that I understand, does it mean that the latest open smalltalk >>> vm does not run on anything else that Mojave and High Sierra? >>> >> >> I don't know exactly on what versions it is not working because I have >> Windows10 and Mojave. >> But I know that it's not working for Pablo on OSX Sierra (which dates >> from 2016...). >> >> AFAIU, Ronie has updated the VM's code to render on Metal, which made >> that at some point the VM was not compiling for latest versions of OSX. >> In these last weeks Ronie has also updated the code to make it compile in >> older machines, but still in those versions that not support Metal the main >> window just renders a black screen. >> >> I don't know what are the following steps regarding this, I put Ronie in >> cc, he may clarify a bit the situation. >> In the meanwhile, using the build I say in the email above may suffice us >> all I think ^^. >> >> >>> >>> Stef (back writing research proposalâ¦..) >>> >>> On 2 Apr 2019, at 13:40, Guillermo Polito <guillermopolito(a)gmail.com> >>> wrote: >>> >>> Hi all, Esteban, >>> >>> In the last sprint with Pablo and Pierre we have detected several >>> misbehaviours with files. >>> The issue was particular reproducible when trying to run iceberg tests: >>> the VM kind of ran out of slots for opening files. The strangest thing is >>> that the leaks seemed to happen when deleting directories, so we had a look >>> at the changes done to iterate directories, get file attributes and so on. >>> >>> While on the FileReference code everything seemed ok, we have observerd >>> with `lsof` that still several directories remained open even after >>> deleting them. >>> >>> Latest VM fixes this issue but contains another one related to Metal >>> rendering, which prevents rendering in OSX versions < than High Sierra. >>> >>> Tracking the issue a bit, this seems to be a bug in the FileAttributes >>> plugin that was fixed by this PR by alistair ( >>> https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2… >>> ). >>> >>> We have tested that particular commit ( >>> a838346b1a67712cc28298534dafbd0c26ea34fb >>> <https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2…>) >>> and this one contains Alistair's fixes but not yet the new rendering using >>> Metal. We have been testing that VM since yesterday and it seems stable, >>> what do you think about marking it stable for Pharo8? >>> >>> Regarding Pharo7, Pharo7's stable VM is the same as Pharo8's, so it >>> contains the same issue. However, since on the image side the code does not >>> use the problematic primitives, we do not think we need to change the VM >>> for Pharo7 (yet). However, there is seemingly a fix for directories >>> containing blanks, maybe Alistair can give us some input here? :) >>> >>> Ideas? >>> >>> Thanks, >>> Guille and Pablo >>> >>> >>> >> >> -- >> >> Guille Polito >> Research Engineer >> >> >> Centre de Recherche en Informatique, Signal et Automatique de Lille >> CRIStAL - UMR 9189 >> French National Center for Scientific Research - *http://www.cnrs.fr >> <http://www.cnrs.fr/>* >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 >> > >

April 2, 2019

Re: [Pharo-dev] Green tests

by Guillermo Polito

On Tue, Apr 2, 2019 at 5:36 PM Marcus Denker <marcus.denker(a)inria.fr> wrote: > > > > On 2 Apr 2019, at 17:30, ducasse <stepharo(a)netcourrier.com> wrote: > > > > Cool and thanks! > > I love this green bar and I will try to keep it green. > > Do you know if the test writing by pierre should be pushed in Pharo > because I have the impression that they will be red first > > > If we add failing tests, then *everyone* is impacted. Instead of âah, > greenâ we will again have to manually check > *every* *single* Pull request. And we, again, will forget to do it. And > thus break even more. > > I really think keeping the tests green is very, very much better for > everyone. > Yes Marcus, but there is another "hidden" point in my mail: my fix was really simple, anybody could have done it. So - just complaining does not fix the build. - removing breaking tests will not guarantee us a better quality either > > Marcus > > > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

April 2, 2019

Re: [Pharo-dev] Green tests

by Guillermo Polito

On Tue, Apr 2, 2019 at 5:31 PM ducasse <stepharo(a)netcourrier.com> wrote: > Cool and thanks! > I love this green bar and I will try to keep it green. > Do you know if the test writing by pierre should be pushed in Pharo > because I have the impression that they will be red first > Short answer: no :). Long answer: I've made a review and requested changes. The issue is related to the need of the new VM. The main problem is that the only way to test it properly is to open a file, closing it and ask your system if the file is still open (which is OS dependent e.g., calling lsof on unix/osx, or the equivalent on windows). The second better test we can do is to try to open as many files as the limit and check that we cannot open more files or that we do not segfault :) but it's not super fun when debugging. Maybe we can just close/ignore that PR? > > Hi all, > > In a couple of minutes I've made some (not complicated) fixes to make > Rubric tests green again: > https://github.com/pharo-project/pharo/pull/3129 > > > I will check them. > We got some tests for maxlength and started with the stupid point. > Yeah, we have to fix some ugly stuff on it (the centered ghost text, the max length...), but there is "no rush" :). > > This is of course not a super Rubric enhancement (for which I've opened > another issue requesting some missing feature > https://github.com/pharo-project/pharo/issues/3128) but this allows us to > move on with a better infrastructure :). > > There are still some some tests that fail from time to time, related to > system announcers and finalization. See > https://github.com/pharo-project/pharo/issues/2471. > Though I've spent half a day yesterday and half today trying to track it > down, I don't yet have a clear way of reproducing it. I've read the entire > code that touches this part of the system and I really bet this is a race > condition :). > So any second pair of eyes here would really be helpful! > > > Iâm inherently race condition incompatible and just avoid concurrent > programming because Iâm too dull on this. > Even funnier is that I think the race condition happens during the stage loading BaselineOfPharo. It's a super huge step, and we need better debugging tools for that kind of scenarios... > > Guille > > > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

April 2, 2019

Re: [Pharo-dev] Green tests

by Marcus Denker

> On 2 Apr 2019, at 17:30, ducasse <stepharo(a)netcourrier.com> wrote: > > Cool and thanks! > I love this green bar and I will try to keep it green. > Do you know if the test writing by pierre should be pushed in Pharo because I have the impression that they will be red first > If we add failing tests, then *everyone* is impacted. Instead of âah, greenâ we will again have to manually check *every* *single* Pull request. And we, again, will forget to do it. And thus break even more. I really think keeping the tests green is very, very much better for everyone. Marcus

April 2, 2019

Re: [Pharo-dev] Performance Issues

by ducasse

Yes we want to use it to build a regression database :) Stef > On 2 Apr 2019, at 17:19, Martin Dias <tinchodias(a)gmail.com> wrote: > > Great! > SMark is a very useful package for benchmarking. > > On Tue, Apr 2, 2019 at 12:06 PM Tudor Girba <tudor(a)tudorgirba.com> wrote: >> >> Great work! >> >> Thanks, >> Doru >> >>> On Apr 2, 2019, at 4:22 PM, tesonep(a)gmail.com wrote: >>> >>> Hi, this will be a long mail. I have organized it in different sections. >>> >>> Resume: >>> ====== >>> >>> - There are performance issues in Pharo 7. >>> - I have made benchmarks. >>> - With Guille we made an improvement in saving source code in methods. >>> - More improvements to come. >>> - Automation and CI in the future. >>> >>> >>> Background >>> ========= >>> >>> Since the release of Pharo 7 (and before it), it was notorious that >>> there exists a performance regression in the normal execution of the >>> image. By checking the usual operations, we have seen ( and many have >>> also detected) that there was an issue with the loading, compilation, >>> and unloading of code. Also with the creation of classes, traits and >>> the use of slots. >>> >>> Benchmarks >>> ========= >>> >>> Although we were sure that there is a performance regression, we have >>> to show it in a way we can test it and measure it. If we cannot >>> measure it or repeat its execution it is worthless. >>> For doing so, I have created an initial set of micro-benchmarks to >>> cover normal operations of the image. >>> >>> The set of benchmarks is available here: >>> https://github.com/tesonep/pharo-benchmarks >>> >>> These benchmarks are developed on top of SMark, only adding a command >>> line tool and the ability to generate a CSV file. >>> >>> The idea is to run the benchmarks in different versions of Pharo an >>> assert that we are not breaking anything. >>> >>> The first results were basically a nightmare, some operations take >>> almost 20 times more in Pharo 7. Especially, the ones that are >>> compiling methods. >>> >>> In the attached document, there is the detail of all the benchmarks, >>> the different results and the analysis of the improvements and >>> regressions (Positive percentages are regressions (more time), >>> negative are improvements (less time)). >>> >>> I have checked the results in OSX with 64 bits images. But as the >>> problem is in pure Smalltalk implementations the problems are (and the >>> solutions) cross platforms. >>> >>> First Improvement >>> ============== >>> >>> Having the benchmarks, it was easy to start looking for the problems. >>> Thanks to the help of Guille we have detected some problems in the >>> implementation of SourceFile. Objects of this class have the >>> responsibility to handle the save of the source code when a method is >>> modified. >>> >>> Improving this implementation we have gotten to results similar to >>> Pharo 6 in the compilation of methods. >>> >>> Comparing a stock Pharo8 image with the one with the fix, we have the >>> following improvements: >>> >>> AddingAnInstanceVariableToAClassWithASubclass -3.96% >>> AddingAnInstanceVariableToAClassWithoutASubclassAndInstances -15.91% >>> AddingAnInstanceVariableToAClassWithoutSubclasses -28.50% >>> ClassAndSubclassCreation -25.67% >>> ClassUsingATrait -90.05% >>> SimpleClassCreation -26.55% >>> TraitCreation -92.95% >>> TraitCreationUsingOtherTrait -91.68% >>> CompileAMethodAndModifyIt -32.92% >>> CompileAMethodInAClassUsingATrait -83.28% >>> CompileAMethodInATraitNotUsed -85.12% >>> CompileAMethodInATraitUsed -88.16% >>> CompileANewMethod -40.73% >>> CompileANewMethodUsingInstanceVariable -33.93% >>> MethodAccessingAGlobal -47.57% >>> >>> Again there are more details in the attached file. >>> >>> Also, we have ported this fix to Pharo 7. >>> >>> Next Steps >>> ======== >>> >>> - Making it a part of the CI infrastructure: making it run in each PR >>> and build to detect regressions in the changes. >>> >>> - Adding more micro and macro benchmarks. I have a list of things to >>> test, but I am open to more: >>> >>> # Micro >>> >>> - Slot Implementation >>> - Process handling >>> - Refactorings >>> - Files (open / write / read) >>> >>> # Macro >>> >>> - Loading: Moose / Seaside >>> - Recompile All >>> - Condense Sources >>> - Startup >>> - Shutdown >>> >>> We also know that there are platform related issues (especially >>> Windows), so the idea it will be the same, build a benchmark, measure >>> it, improve it. >>> >>> The idea is to have a more robust way of detecting and handling the >>> performance of Pharo. >>> >>> Of course, I am open to all your comments. >>> >>> Cheers, >>> Pablo >>> <Report.xls> >> >> -- >> www.feenk.com >> >> "If you can't say why something is relevant, >> it probably isn't." >> >> >

April 2, 2019

Re: [Pharo-dev] Green tests

by ducasse

Cool and thanks! I love this green bar and I will try to keep it green. Do you know if the test writing by pierre should be pushed in Pharo because I have the impression that they will be red first > Hi all, > > In a couple of minutes I've made some (not complicated) fixes to make Rubric tests green again: > https://github.com/pharo-project/pharo/pull/3129 <https://github.com/pharo-project/pharo/pull/3129> I will check them. We got some tests for maxlength and started with the stupid point. > > This is of course not a super Rubric enhancement (for which I've opened another issue requesting some missing feature https://github.com/pharo-project/pharo/issues/3128 <https://github.com/pharo-project/pharo/issues/3128>) but this allows us to move on with a better infrastructure :). > > There are still some some tests that fail from time to time, related to system announcers and finalization. See https://github.com/pharo-project/pharo/issues/2471 <https://github.com/pharo-project/pharo/issues/2471>. > Though I've spent half a day yesterday and half today trying to track it down, I don't yet have a clear way of reproducing it. I've read the entire code that touches this part of the system and I really bet this is a race condition :). > So any second pair of eyes here would really be helpful! Iâm inherently race condition incompatible and just avoid concurrent programming because Iâm too dull on this. > Guille

April 2, 2019

Re: [Pharo-dev] New stable VM for Pharo8?

by ducasse

Thanks ronie for the explanation. I thought that plugin was a plugin, i.e., optional but it looks like it is not. :( When I read your mail, it looks like we have to problem with a plugin we do not use. Even better :( May be Pharo should have a branch so that we can promote a Pharo stable VM. I feel uncomfortable to have to remember which commits is the stable one for us because how can we progress there? I do not see how we can jump over commits. And my concern is how can we improve the time you and Pablo and Guille are spending handling such case. This is ok to look for a bug and fix, now we will have to put in place a process to handle regressions and unsure that we can blessed a VM a stable for real. We should discuss it during PharoDays consortium meeting and the next consortium online meeting. Stef > On 2 Apr 2019, at 16:56, Ronie Salgado <roniesalg(a)gmail.com> wrote: > > Hi, > > For a job I have to adapt old Squeak code (B3DAcceleratorPlugin) to newer graphics APIs (Metal and D3D11). The VM window has to use the same rendering interface as the plugin for rendering. Recently for trying to solve the issue with the older machines I implemented support selecting the older renderers with a command line option, and I added code for trying to detect whether the current Metal implementation is valid, or not. These command line options are the followings (can be obtained by using -help): > > -metal Use metal if the Metal detection code works. > -opengl Use OpenGL. The previous default. > -core-graphics Use the older CoreGraphics based backend. > -headless Null without rendering > > This command line option has to be the first command line option passed to the VM program. This is because the normal command line parsing is done after creating the VM window. As a workaround I am parsing the command line option on the main() function as the first command line argument. > > On my Mac machine I have Mojave installed. If someone could give pointers on how to setup a VM with the older version, or a way to emulate the older version I could be able to debug the problem. > > Perhaps as a workaround, for Pharo we should disable the Metal support (Remove the -DUSE_METAL from the makefile), and maybe go for the CoreGraphics backend, which seems to be better tested and I think does not give conflict problems when trying to use OpenGL from the image side. In Pharo we do not use the B3DAcceleratorPlugin, which in my opinion is a complete mess. > > Best regards, > Ronie > > > El mar., 2 abr. 2019 a las 10:58, Guillermo Polito (<guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>>) escribiÃ³: > > > On Tue, Apr 2, 2019 at 2:11 PM ducasse <stepharo(a)netcourrier.com <mailto:stepharo@netcourrier.com>> wrote: > Thanks guille and pablo for you dedication for Pharo, I love you :) > > Now just that I understand, does it mean that the latest open smalltalk vm does not run on anything else that Mojave and High Sierra? > > I don't know exactly on what versions it is not working because I have Windows10 and Mojave. > But I know that it's not working for Pablo on OSX Sierra (which dates from 2016...). > > AFAIU, Ronie has updated the VM's code to render on Metal, which made that at some point the VM was not compiling for latest versions of OSX. > In these last weeks Ronie has also updated the code to make it compile in older machines, but still in those versions that not support Metal the main window just renders a black screen. > > I don't know what are the following steps regarding this, I put Ronie in cc, he may clarify a bit the situation. > In the meanwhile, using the build I say in the email above may suffice us all I think ^^. > > > Stef (back writing research proposalâ¦..) > >> On 2 Apr 2019, at 13:40, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: >> >> Hi all, Esteban, >> >> In the last sprint with Pablo and Pierre we have detected several misbehaviours with files. >> The issue was particular reproducible when trying to run iceberg tests: the VM kind of ran out of slots for opening files. The strangest thing is that the leaks seemed to happen when deleting directories, so we had a look at the changes done to iterate directories, get file attributes and so on. >> >> While on the FileReference code everything seemed ok, we have observerd with `lsof` that still several directories remained open even after deleting them. >> >> Latest VM fixes this issue but contains another one related to Metal rendering, which prevents rendering in OSX versions < than High Sierra. >> >> Tracking the issue a bit, this seems to be a bug in the FileAttributes plugin that was fixed by this PR by alistair (https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2… <https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2…>). >> >> We have tested that particular commit (a838346b1a67712cc28298534dafbd0c26ea34fb <https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/a838346b1a67712cc2…>) and this one contains Alistair's fixes but not yet the new rendering using Metal. We have been testing that VM since yesterday and it seems stable, what do you think about marking it stable for Pharo8? >> >> Regarding Pharo7, Pharo7's stable VM is the same as Pharo8's, so it contains the same issue. However, since on the image side the code does not use the problematic primitives, we do not think we need to change the VM for Pharo7 (yet). However, there is seemingly a fix for directories containing blanks, maybe Alistair can give us some input here? :) >> >> Ideas? >> >> Thanks, >> Guille and Pablo > > > > -- > > Guille Polito > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> > > Web: http://guillep.github.io <http://guillep.github.io/> > Phone: +33 06 52 70 66 13

April 2, 2019