April 2019 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Deep Recursion break/protection

by Sven Van Caekenberghe

> On 25 Apr 2019, at 15:27, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi Sven, > > On Thu, Apr 25, 2019 at 10:04 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> This way, you should run out of heap before the stack gets too large, since you allocate and hold on to an array in each frame. > > I don't understand the purpose of such expression, since I'm not > writing recursive code, where I pay special attention or save the > image before running it, but I'm trying to find a way around > accidental deep recursions. I know, it was an experiment. > However I ran these examples in Dolphin and Pharo 7, and in Dolphin it > triggers an memory exhaustion exception right away, and in Pharo it > grows in memory use, slower than with the previous example, but with > the same behavior (unresponsive image once the memory fills the > available OS memory). Too bad, this is a VM issue anyway. > Regards, > > Esteban A. Maringolo >

April 25, 2019

Re: [Pharo-users] Deep Recursion break/protection

by Esteban Maringolo

Hi Sven, On Thu, Apr 25, 2019 at 10:04 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > This way, you should run out of heap before the stack gets too large, since you allocate and hold on to an array in each frame. I don't understand the purpose of such expression, since I'm not writing recursive code, where I pay special attention or save the image before running it, but I'm trying to find a way around accidental deep recursions. However I ran these examples in Dolphin and Pharo 7, and in Dolphin it triggers an memory exhaustion exception right away, and in Pharo it grows in memory use, slower than with the previous example, but with the same behavior (unresponsive image once the memory fills the available OS memory). Regards, Esteban A. Maringolo

April 25, 2019

Re: [Pharo-users] Deep Recursion break/protection

by Sven Van Caekenberghe

What if you did something like: RecursiveObject>>#recurse | data | data := Array new: 1e5. self recurse. ^ data This way, you should run out of heap before the stack gets too large, since you allocate and hold on to an array in each frame. > On 25 Apr 2019, at 14:56, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > I downloaded a fresh 7.0.3 image and VM, created the above class and > method, and called it. > > The image is running in Ubuntu within a VirtualBox with 4GB RAM. > > Such exception was not triggered in my image when it actually filled > all the available OS RAM, and I had to kill it using the operative > system process manager. > > If I call it and press Alt+. right away it halts the execution (if > running on the main thread). > In Dolphin, because it is not JITted, it is instantaneous after a long > chain of calls. > > Crashing the image by such a simple to make mistake should be avoided. > Fortunately we have other tools such as Epicea to recover changes, but > even still. > > > Regards, > > Esteban A. Maringolo > > On Thu, Apr 25, 2019 at 9:47 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> We also have an OutOfMemory exception, we even have tests provoking such a situation (check usage). >> >> However, I believe such tight loops end up in JIT machine code. >> >> Checking stack overflow on each stack manipulation is costly. >> >> But I totally agree that we should try to catch these situations even if there is a cost. >> >>> On 25 Apr 2019, at 14:39, Esteban Maringolo <emaringolo(a)gmail.com> wrote: >>> >>> I just checked in VisualWorks, where I also develop Seaside and also >>> make these mistakes, and these recursions launch a "Process Monitor >>> Emergency: No space left" process monitor, with the option of killing >>> any running Smalltalk process. >>> >>> Regards, >>> >>> Esteban A. Maringolo >>> >>> On Thu, Apr 25, 2019 at 9:30 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: >>>> >>>> Hi, >>>> >>>> Is there a way to add deep recursion protection to the system? >>>> >>>> eg. >>>> RecursiveObject>>#recurse >>>> self recurse >>>> >>>> So calling `RecursiveObject new recurse` in Dolphin raises an >>>> exception right away... >>>> >>>> Throws the following error with this stack: >>>> ProcessorScheduler>>stackOverflow: >>>> [] in ProcessorScheduler>>vmi:list:no:with: >>>> BlockClosure>>ifCurtailed: >>>> ProcessorScheduler>>vmi:list:no:with: >>>> RecursiveObject>>recurse >>>> RecursiveObject>>recurse >>>> >>>> In Pharo it goes forever until it hits the memory limit (3.6GiB), at >>>> which point doing an Alt+. is useless and you have to kill the VM. >>>> >>>> The example is pretty simple, but when doing Seaside rendering, it is >>>> easy to miss some return, causing the receiver to render recursively, >>>> turning your image useless. >>>> >>>> e.g. >>>> MyComponent>>renderContentOn: html >>>> html render: self someSubComponent >>>> >>>> MyComponent>>someSubComponent >>>> "Here I forget returning the subcomponent" >>>> someSubComponent ifNil: [someSubcomponent := OtherComponent new]. >>>> >>>> When you render MyComponent... boom, because >>>> #MyComponent>>#renderContentOn: will be called recursively. >>>> >>>> Regards, >>>> >>>> Esteban A. Maringolo >>> >> >> >

April 25, 2019

Re: [Pharo-users] Deep Recursion break/protection

by Esteban Maringolo

I downloaded a fresh 7.0.3 image and VM, created the above class and method, and called it. The image is running in Ubuntu within a VirtualBox with 4GB RAM. Such exception was not triggered in my image when it actually filled all the available OS RAM, and I had to kill it using the operative system process manager. If I call it and press Alt+. right away it halts the execution (if running on the main thread). In Dolphin, because it is not JITted, it is instantaneous after a long chain of calls. Crashing the image by such a simple to make mistake should be avoided. Fortunately we have other tools such as Epicea to recover changes, but even still. Regards, Esteban A. Maringolo On Thu, Apr 25, 2019 at 9:47 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > We also have an OutOfMemory exception, we even have tests provoking such a situation (check usage). > > However, I believe such tight loops end up in JIT machine code. > > Checking stack overflow on each stack manipulation is costly. > > But I totally agree that we should try to catch these situations even if there is a cost. > > > On 25 Apr 2019, at 14:39, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > I just checked in VisualWorks, where I also develop Seaside and also > > make these mistakes, and these recursions launch a "Process Monitor > > Emergency: No space left" process monitor, with the option of killing > > any running Smalltalk process. > > > > Regards, > > > > Esteban A. Maringolo > > > > On Thu, Apr 25, 2019 at 9:30 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > >> > >> Hi, > >> > >> Is there a way to add deep recursion protection to the system? > >> > >> eg. > >> RecursiveObject>>#recurse > >> self recurse > >> > >> So calling `RecursiveObject new recurse` in Dolphin raises an > >> exception right away... > >> > >> Throws the following error with this stack: > >> ProcessorScheduler>>stackOverflow: > >> [] in ProcessorScheduler>>vmi:list:no:with: > >> BlockClosure>>ifCurtailed: > >> ProcessorScheduler>>vmi:list:no:with: > >> RecursiveObject>>recurse > >> RecursiveObject>>recurse > >> > >> In Pharo it goes forever until it hits the memory limit (3.6GiB), at > >> which point doing an Alt+. is useless and you have to kill the VM. > >> > >> The example is pretty simple, but when doing Seaside rendering, it is > >> easy to miss some return, causing the receiver to render recursively, > >> turning your image useless. > >> > >> e.g. > >> MyComponent>>renderContentOn: html > >> html render: self someSubComponent > >> > >> MyComponent>>someSubComponent > >> "Here I forget returning the subcomponent" > >> someSubComponent ifNil: [someSubcomponent := OtherComponent new]. > >> > >> When you render MyComponent... boom, because > >> #MyComponent>>#renderContentOn: will be called recursively. > >> > >> Regards, > >> > >> Esteban A. Maringolo > > > >

April 25, 2019

Re: [Pharo-users] Deep Recursion break/protection

by Sven Van Caekenberghe

We also have an OutOfMemory exception, we even have tests provoking such a situation (check usage). However, I believe such tight loops end up in JIT machine code. Checking stack overflow on each stack manipulation is costly. But I totally agree that we should try to catch these situations even if there is a cost. > On 25 Apr 2019, at 14:39, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > I just checked in VisualWorks, where I also develop Seaside and also > make these mistakes, and these recursions launch a "Process Monitor > Emergency: No space left" process monitor, with the option of killing > any running Smalltalk process. > > Regards, > > Esteban A. Maringolo > > On Thu, Apr 25, 2019 at 9:30 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: >> >> Hi, >> >> Is there a way to add deep recursion protection to the system? >> >> eg. >> RecursiveObject>>#recurse >> self recurse >> >> So calling `RecursiveObject new recurse` in Dolphin raises an >> exception right away... >> >> Throws the following error with this stack: >> ProcessorScheduler>>stackOverflow: >> [] in ProcessorScheduler>>vmi:list:no:with: >> BlockClosure>>ifCurtailed: >> ProcessorScheduler>>vmi:list:no:with: >> RecursiveObject>>recurse >> RecursiveObject>>recurse >> >> In Pharo it goes forever until it hits the memory limit (3.6GiB), at >> which point doing an Alt+. is useless and you have to kill the VM. >> >> The example is pretty simple, but when doing Seaside rendering, it is >> easy to miss some return, causing the receiver to render recursively, >> turning your image useless. >> >> e.g. >> MyComponent>>renderContentOn: html >> html render: self someSubComponent >> >> MyComponent>>someSubComponent >> "Here I forget returning the subcomponent" >> someSubComponent ifNil: [someSubcomponent := OtherComponent new]. >> >> When you render MyComponent... boom, because >> #MyComponent>>#renderContentOn: will be called recursively. >> >> Regards, >> >> Esteban A. Maringolo >

April 25, 2019

Re: [Pharo-users] Deep Recursion break/protection

by Esteban Maringolo

I just checked in VisualWorks, where I also develop Seaside and also make these mistakes, and these recursions launch a "Process Monitor Emergency: No space left" process monitor, with the option of killing any running Smalltalk process. Regards, Esteban A. Maringolo On Thu, Apr 25, 2019 at 9:30 AM Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi, > > Is there a way to add deep recursion protection to the system? > > eg. > RecursiveObject>>#recurse > self recurse > > So calling `RecursiveObject new recurse` in Dolphin raises an > exception right away... > > Throws the following error with this stack: > ProcessorScheduler>>stackOverflow: > [] in ProcessorScheduler>>vmi:list:no:with: > BlockClosure>>ifCurtailed: > ProcessorScheduler>>vmi:list:no:with: > RecursiveObject>>recurse > RecursiveObject>>recurse > > In Pharo it goes forever until it hits the memory limit (3.6GiB), at > which point doing an Alt+. is useless and you have to kill the VM. > > The example is pretty simple, but when doing Seaside rendering, it is > easy to miss some return, causing the receiver to render recursively, > turning your image useless. > > e.g. > MyComponent>>renderContentOn: html > html render: self someSubComponent > > MyComponent>>someSubComponent > "Here I forget returning the subcomponent" > someSubComponent ifNil: [someSubcomponent := OtherComponent new]. > > When you render MyComponent... boom, because > #MyComponent>>#renderContentOn: will be called recursively. > > Regards, > > Esteban A. Maringolo

April 25, 2019

Deep Recursion break/protection

by Esteban Maringolo

Hi, Is there a way to add deep recursion protection to the system? eg. RecursiveObject>>#recurse self recurse So calling `RecursiveObject new recurse` in Dolphin raises an exception right away... Throws the following error with this stack: ProcessorScheduler>>stackOverflow: [] in ProcessorScheduler>>vmi:list:no:with: BlockClosure>>ifCurtailed: ProcessorScheduler>>vmi:list:no:with: RecursiveObject>>recurse RecursiveObject>>recurse In Pharo it goes forever until it hits the memory limit (3.6GiB), at which point doing an Alt+. is useless and you have to kill the VM. The example is pretty simple, but when doing Seaside rendering, it is easy to miss some return, causing the receiver to render recursively, turning your image useless. e.g. MyComponent>>renderContentOn: html html render: self someSubComponent MyComponent>>someSubComponent "Here I forget returning the subcomponent" someSubComponent ifNil: [someSubcomponent := OtherComponent new]. When you render MyComponent... boom, because #MyComponent>>#renderContentOn: will be called recursively. Regards, Esteban A. Maringolo

April 25, 2019

Re: [Pharo-users] Baseline subclasses

by Norbert Hartl

Dale, looking at my code I really like the idea solving my white label problem with polymorphism. It makes things much easier IMHO. So ended up doing something like having something like this BaselineOfCoreProduct â¦ spec group: âwhiteLabelâ with: #( âBaselineOfWhiteLabelProductâ ) â¦ and doing Metacello new baseline: #CoreProduct load: âwhiteLabelâ; baseline: #WhiteLabelProduct; load This way each baseline is still in its own package. The project is not for public consumption so we have to do that in our deployment process nowhere else. Norbert > Am 25.04.2019 um 09:51 schrieb Norbert Hartl <norbert(a)hartl.name>: > > > >> Am 25.04.2019 um 01:13 schrieb Dale Henrichs <dale.henrichs(a)gemtalksystems.com>: >> >> >> On 4/24/19 3:08 PM, Norbert Hartl wrote: >>> I donât have an idea how to achieve this with a group. It is about changing the number of dependencies of one package. >> >> There's load order and there's grouping... >> >> If you are using #linear loads for a BaselineOf then hard dependencies (Metacello requires:) dictate load order and load order is important. >> >> If you are using #atomic loads, then load order for packages is not important ... for #atomic loads you only need to specify which set of packages you want to have loaded together ... so if you have a baseline where you don't use #requires: anywhere and simply define different groups for each set of package to want to have loaded together (one for productA and another for productB) ... you should be able to get what you want ... >> >> If this is a public baseline, then you have to warn users of your project that the only legally loadable entities are the set of groups that you've created, then things should work well ... ... I'd think:) >> >>> >>> So no way of making Metacello load a package with one name and load a baseline with another name? >> >> Not sure what you are thinking about here ... i think I need a concrete example of what you mean hereâ¦ >> > Sorry, I wasnât clear. I was talking about the possibility to have a basline loaded where the package name and the name of the baseline are not the same. So letâs say I have the two baselines in the BaselineOfCoreProduct. Then I would need to do something like > > Metacello new > package: âBaselineOfCoreProductâ; > baseline: #BaselineOfWhiteLabelProduct; > load > > That would be sufficient. Having all the baselines in one package would be a smaller problem then. > > Norbert

April 25, 2019

Re: [Pharo-users] Baseline subclasses

by Norbert Hartl

> Am 25.04.2019 um 01:13 schrieb Dale Henrichs <dale.henrichs(a)gemtalksystems.com>: > > > On 4/24/19 3:08 PM, Norbert Hartl wrote: >> I donât have an idea how to achieve this with a group. It is about changing the number of dependencies of one package. > > There's load order and there's grouping... > > If you are using #linear loads for a BaselineOf then hard dependencies (Metacello requires:) dictate load order and load order is important. > > If you are using #atomic loads, then load order for packages is not important ... for #atomic loads you only need to specify which set of packages you want to have loaded together ... so if you have a baseline where you don't use #requires: anywhere and simply define different groups for each set of package to want to have loaded together (one for productA and another for productB) ... you should be able to get what you want ... > > If this is a public baseline, then you have to warn users of your project that the only legally loadable entities are the set of groups that you've created, then things should work well ... ... I'd think:) > >> >> So no way of making Metacello load a package with one name and load a baseline with another name? > > Not sure what you are thinking about here ... i think I need a concrete example of what you mean hereâ¦ > Sorry, I wasnât clear. I was talking about the possibility to have a basline loaded where the package name and the name of the baseline are not the same. So letâs say I have the two baselines in the BaselineOfCoreProduct. Then I would need to do something like Metacello new package: âBaselineOfCoreProductâ; baseline: #BaselineOfWhiteLabelProduct; load That would be sufficient. Having all the baselines in one package would be a smaller problem then. Norbert

April 25, 2019

Re: [Pharo-users] how to model this a better way

by Roelof Wobben

April 25, 2019