Pharo-dev
By thread
pharo-dev@lists.pharo.org
By month
Messages by month
- ----- 2026 -----
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- 4 participants
- 144627 messages
Re: [Pharo-project] Protecting against stack overflow ?
by Eliot Miranda
On Tue, Dec 11, 2012 at 3:35 PM, Eliot Miranda <eliot.miranda(a)gmail.com>wrote:
>
>
>
> On Tue, Dec 11, 2012 at 10:26 AM, Igor Stasenko <siguctua(a)gmail.com>wrote:
>
>> hehe..
>> it looks like we can do it:
>>
>> - when memory hits the waterline mark, VM should do the following:
>> - for every currently scheduled process , push an artificial stack
>> frame on top , which is
>> StackOverflow signal.
>>
>
> If the VM supports the LowSpaceSignal mechanism as it used to be then this
> can be done in the image with a high-priority process. And tehrefore it
> can e.g. be avoided for processes such as the Delay process, where IMO it
> is inappropriate.
>
And if it is done in the image it is easy to e.g. avoid pushing two such
frames on the same process because the process has made no progress since
the last occurrence, etc.
>
>>
>> Then the first thing what this guy should do, when he will get
>> control, is to analyze
>> how deep is the stack of given process and if it surpassed the allowed
>> threshold,
>> and if there's no handler, then decide whether to kill the process or not.
>>
>> Also, since it is exception, a user code can intercept this exception and
>> either ignore it (effectively forcing system to keep running), or
>> otherwise
>> terminate process, but in an educated manner, since he might have a
>> better knowledge,
>> what can eat so much resources in his code.. as well, as do a gracious
>> failure
>> handling, e.g., correctly clean associated system resources etc.
>>
>>
>> --
>> Best regards,
>> Igor Stasenko.
>>
>>
>
>
> --
> best,
> Eliot
>
>
--
best,
Eliot
Dec. 11, 2012
Re: [Pharo-project] Protecting against stack overflow ?
by Eliot Miranda
On Tue, Dec 11, 2012 at 2:44 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote:
> Hi Eliot,
>
> On 11 Dec 2012, at 02:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote:
>
> > 2012/12/9 Sven Van Caekenberghe <sven(a)stfx.eu>
> > Hi,
> >
> > Would it be possible to build some kind of protection against stack
> overflow ?
> >
> > note that there already is some mechanism. The low space mechanism is
> supposed to protect against stack overflow but with today's memories and
> the difficulty of testing it, this mechanism often a) takes way too long to
> kick in, and b) either leaves the system in a very sluggish state or simply
> fails to kick in and the system crashes with an out-of-memory error.
> >
> > Part of the problem is in designing a mechanism which is in keeping with
> the reflective nature of the system. What I mean is that we now have two
> different VMs, the Interpreter and the STack/Cog VMs. First one wants a
> check which is cheap. That means a check which isn't run on every send.
> In the interpreter a natural time to check for recursion would be on e.g.
> fullGC, checking for repetition in the current process's context chain.
> The the Stack/Cog VMs the natural time is on evacuating a stack page when
> the stack zone is full. But these are two different mechanisms, both of
> which are deep in the VM, neither of which can be conveniently intercepted
> from Smalltalk, unlike e.g. doesNotUnderstand:.
> >
> > Personally I like the idea of reviving the low space mechanism and
> maintaining it properly. For example, if the low space mechanism causes
> the low space semaphore to fire often, because the limit was set only a
> little way above the current memory size, and not, as it is now, a little
> below the absolute limit, then when it fires, the low space process could
> inspect the current process's stack (e.g. via accelerated via a primitive
> for performance) and see how deep it is, and possibly whether there is
> recursion (e.g. a bag of context methods would soon reveal high-frequency
> methods, and then those contexts could be inspected for repetition, but
> KISS says just have a per-process limit on the maximum depth of stack).
>
> Thanks for the reply.
>
> Yes, getting a low space warning sooner and then deal with that
> intelligently at the image level sounds like a good solution. It is
> probably more portable, requires miminal VM collaboration, allows for many
> behaviours as a reaction and would catch other problems than just stack
> overflow.
>
> Is this already possible today ?
>
Yes. The primitive is primitive 125.
SmalltalkImage methods for memory space
primSignalAtBytesLeft: numBytes
"Tell the interpreter the low-space threshold in bytes. When the free
space falls below this threshold, the interpreter will signal the low-space
semaphore, if one has been registered. Disable low-space interrupts if the
argument is zero. Fail if numBytes is not an Integer."
<primitive: 125>
self primitiveFailed
There should be an installLowSpaceWatcher to access this, and a
lowSpaceWatcher process that implements the low space policy.
>
> Preferrably, dynamic control over the limit would be nice: in a headless
> server you want other behavior than during interactive development.
>
Right. One can call the primitive at any time to modify the limit.
>
> Sven
>
> > This is actually the most common situation for loop/cycle that either
> results in an unusable image or a crash.
> > Interrupting with CMD-. is not always possible.
> > A possible solution could be to just limit the stack size and throw an
> exception.
> >
> > For example, in LispWorks, it goes like this:
> >
> > CL-USER 5 > (defun foo () (cons (random 100) (foo)))
> > FOO
> >
> > CL-USER 6 > (foo)
> >
> > Stack overflow (stack size 48128).
> > 1 (continue) Extend stack by 50%.
> > 2 (abort) Return to level 0.
> > 3 Return to top loop level 0.
> >
> > Type :b for backtrace or :c <option number> to proceed.
> > Type :bug-form "<subject>" for a bug report template or :? for other
> options.
> >
> > CL-USER 7 : 1 > :bq
> >
> > ERROR <- RUNTIME:BAD-ARGS-OR-STACK <- LENGTH <- FOO <- FOO <- FOO <- FOO
> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- EVAL <- CAPI::CAPI-TOP-LEVEL-FUNCTION <-
> CAPI::INTERACTIVE-PANE-TOP-LOOP <- MP::PROCESS-SG-FUNCTION
> >
> > Obviously, this is possible, the question is, can it be done easily and
> without paying an unbearable performance price ?
> >
> > Sven
> >
> > --
> > Sven Van Caekenberghe
> > http://stfx.eu
> > Smalltalk is the Red Pill
> >
> >
> >
> >
> >
> >
> >
> > --
> > best,
> > Eliot
> >
>
> --
> Sven Van Caekenberghe
> http://stfx.eu
> Smalltalk is the Red Pill
>
>
>
>
>
--
best,
Eliot
Dec. 11, 2012
Re: [Pharo-project] Protecting against stack overflow ?
by Eliot Miranda
On Tue, Dec 11, 2012 at 3:33 PM, Eliot Miranda <eliot.miranda(a)gmail.com>wrote:
>
>
>
> On Tue, Dec 11, 2012 at 2:44 AM, Sven Van Caekenberghe <sven(a)stfx.eu>wrote:
>
>> Hi Eliot,
>>
>> On 11 Dec 2012, at 02:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote:
>>
>> > 2012/12/9 Sven Van Caekenberghe <sven(a)stfx.eu>
>> > Hi,
>> >
>> > Would it be possible to build some kind of protection against stack
>> overflow ?
>> >
>> > note that there already is some mechanism. The low space mechanism is
>> supposed to protect against stack overflow but with today's memories and
>> the difficulty of testing it, this mechanism often a) takes way too long to
>> kick in, and b) either leaves the system in a very sluggish state or simply
>> fails to kick in and the system crashes with an out-of-memory error.
>> >
>> > Part of the problem is in designing a mechanism which is in keeping
>> with the reflective nature of the system. What I mean is that we now have
>> two different VMs, the Interpreter and the STack/Cog VMs. First one wants
>> a check which is cheap. That means a check which isn't run on every send.
>> In the interpreter a natural time to check for recursion would be on e.g.
>> fullGC, checking for repetition in the current process's context chain.
>> The the Stack/Cog VMs the natural time is on evacuating a stack page when
>> the stack zone is full. But these are two different mechanisms, both of
>> which are deep in the VM, neither of which can be conveniently intercepted
>> from Smalltalk, unlike e.g. doesNotUnderstand:.
>> >
>> > Personally I like the idea of reviving the low space mechanism and
>> maintaining it properly. For example, if the low space mechanism causes
>> the low space semaphore to fire often, because the limit was set only a
>> little way above the current memory size, and not, as it is now, a little
>> below the absolute limit, then when it fires, the low space process could
>> inspect the current process's stack (e.g. via accelerated via a primitive
>> for performance) and see how deep it is, and possibly whether there is
>> recursion (e.g. a bag of context methods would soon reveal high-frequency
>> methods, and then those contexts could be inspected for repetition, but
>> KISS says just have a per-process limit on the maximum depth of stack).
>>
>> Thanks for the reply.
>>
>> Yes, getting a low space warning sooner and then deal with that
>> intelligently at the image level sounds like a good solution. It is
>> probably more portable, requires miminal VM collaboration, allows for many
>> behaviours as a reaction and would catch other problems than just stack
>> overflow.
>>
>> Is this already possible today ?
>>
>
Um, ignore this. I was smoking something. (actually the message selector
browser completely hid the inst var access browser so I didn't see the inst
var accesses). Hang on and I'll try again...
>
> Wow, I'm amazed. The low space limit in the VM has been ripped out So
> no, it is not possible, but it would be easy to put back. There is a
> variable in ObjectMemory (the VM's heap manager/garbage collector class)
> that defines the low-space limit, but there is no setter. Back in the day
> (and still in VisualWorks) there was a primitive
>
> SystemDictionary methods for memory space
> signal: aSemaphore atOopsLeft: numOops wordsLeft: numWords
> "Tell the object memory to signal the Semaphore when either the number
> of object pointers remaining drops below numOops, or the number of
> words in the object space remaining drops below numWords. Fail if the
> frist argument is neither a Semaphore nor nil. Fail if numOops is not a
> 16-bit Integer, or if numWords is not a 32-bit LargePositiveInteger.
> Essential. See Object documentation whatIsAPrimitive."
>
> <primitive: 116>
> self primitiveFailed
>
> But this and its support is gone. Hmm. Hmmph :)
>
>
>
>
>> Preferrably, dynamic control over the limit would be nice: in a headless
>> server you want other behavior than during interactive development.
>>
>
> Right. The low space process would readjust the value every time low
> space fired. But one could (can) call the primitive at any time to install
> a new limit.
>
>
>> Sven
>>
>> > This is actually the most common situation for loop/cycle that either
>> results in an unusable image or a crash.
>> > Interrupting with CMD-. is not always possible.
>> > A possible solution could be to just limit the stack size and throw an
>> exception.
>> >
>> > For example, in LispWorks, it goes like this:
>> >
>> > CL-USER 5 > (defun foo () (cons (random 100) (foo)))
>> > FOO
>> >
>> > CL-USER 6 > (foo)
>> >
>> > Stack overflow (stack size 48128).
>> > 1 (continue) Extend stack by 50%.
>> > 2 (abort) Return to level 0.
>> > 3 Return to top loop level 0.
>> >
>> > Type :b for backtrace or :c <option number> to proceed.
>> > Type :bug-form "<subject>" for a bug report template or :? for other
>> options.
>> >
>> > CL-USER 7 : 1 > :bq
>> >
>> > ERROR <- RUNTIME:BAD-ARGS-OR-STACK <- LENGTH <- FOO <- FOO <- FOO <-
>> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
>> > <- FOO <- FOO <- EVAL <- CAPI::CAPI-TOP-LEVEL-FUNCTION <-
>> CAPI::INTERACTIVE-PANE-TOP-LOOP <- MP::PROCESS-SG-FUNCTION
>> >
>> > Obviously, this is possible, the question is, can it be done easily and
>> without paying an unbearable performance price ?
>> >
>> > Sven
>> >
>> > --
>> > Sven Van Caekenberghe
>> > http://stfx.eu
>> > Smalltalk is the Red Pill
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > --
>> > best,
>> > Eliot
>> >
>>
>> --
>> Sven Van Caekenberghe
>> http://stfx.eu
>> Smalltalk is the Red Pill
>>
>>
>>
>>
>>
>
>
> --
> best,
> Eliot
>
>
--
best,
Eliot
Dec. 11, 2012
Re: [Pharo-project] Protecting against stack overflow ?
by Eliot Miranda
On Tue, Dec 11, 2012 at 10:26 AM, Igor Stasenko <siguctua(a)gmail.com> wrote:
> hehe..
> it looks like we can do it:
>
> - when memory hits the waterline mark, VM should do the following:
> - for every currently scheduled process , push an artificial stack
> frame on top , which is
> StackOverflow signal.
>
If the VM supports the LowSpaceSignal mechanism as it used to be then this
can be done in the image with a high-priority process. And tehrefore it
can e.g. be avoided for processes such as the Delay process, where IMO it
is inappropriate.
>
> Then the first thing what this guy should do, when he will get
> control, is to analyze
> how deep is the stack of given process and if it surpassed the allowed
> threshold,
> and if there's no handler, then decide whether to kill the process or not.
>
> Also, since it is exception, a user code can intercept this exception and
> either ignore it (effectively forcing system to keep running), or otherwise
> terminate process, but in an educated manner, since he might have a
> better knowledge,
> what can eat so much resources in his code.. as well, as do a gracious
> failure
> handling, e.g., correctly clean associated system resources etc.
>
>
> --
> Best regards,
> Igor Stasenko.
>
>
--
best,
Eliot
Dec. 11, 2012
Re: [Pharo-project] Protecting against stack overflow ?
by Eliot Miranda
On Tue, Dec 11, 2012 at 2:44 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote:
> Hi Eliot,
>
> On 11 Dec 2012, at 02:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote:
>
> > 2012/12/9 Sven Van Caekenberghe <sven(a)stfx.eu>
> > Hi,
> >
> > Would it be possible to build some kind of protection against stack
> overflow ?
> >
> > note that there already is some mechanism. The low space mechanism is
> supposed to protect against stack overflow but with today's memories and
> the difficulty of testing it, this mechanism often a) takes way too long to
> kick in, and b) either leaves the system in a very sluggish state or simply
> fails to kick in and the system crashes with an out-of-memory error.
> >
> > Part of the problem is in designing a mechanism which is in keeping with
> the reflective nature of the system. What I mean is that we now have two
> different VMs, the Interpreter and the STack/Cog VMs. First one wants a
> check which is cheap. That means a check which isn't run on every send.
> In the interpreter a natural time to check for recursion would be on e.g.
> fullGC, checking for repetition in the current process's context chain.
> The the Stack/Cog VMs the natural time is on evacuating a stack page when
> the stack zone is full. But these are two different mechanisms, both of
> which are deep in the VM, neither of which can be conveniently intercepted
> from Smalltalk, unlike e.g. doesNotUnderstand:.
> >
> > Personally I like the idea of reviving the low space mechanism and
> maintaining it properly. For example, if the low space mechanism causes
> the low space semaphore to fire often, because the limit was set only a
> little way above the current memory size, and not, as it is now, a little
> below the absolute limit, then when it fires, the low space process could
> inspect the current process's stack (e.g. via accelerated via a primitive
> for performance) and see how deep it is, and possibly whether there is
> recursion (e.g. a bag of context methods would soon reveal high-frequency
> methods, and then those contexts could be inspected for repetition, but
> KISS says just have a per-process limit on the maximum depth of stack).
>
> Thanks for the reply.
>
> Yes, getting a low space warning sooner and then deal with that
> intelligently at the image level sounds like a good solution. It is
> probably more portable, requires miminal VM collaboration, allows for many
> behaviours as a reaction and would catch other problems than just stack
> overflow.
>
> Is this already possible today ?
>
Wow, I'm amazed. The low space limit in the VM has been ripped out So no,
it is not possible, but it would be easy to put back. There is a variable
in ObjectMemory (the VM's heap manager/garbage collector class) that
defines the low-space limit, but there is no setter. Back in the day (and
still in VisualWorks) there was a primitive
SystemDictionary methods for memory space
signal: aSemaphore atOopsLeft: numOops wordsLeft: numWords
"Tell the object memory to signal the Semaphore when either the number
of object pointers remaining drops below numOops, or the number of
words in the object space remaining drops below numWords. Fail if the
frist argument is neither a Semaphore nor nil. Fail if numOops is not a
16-bit Integer, or if numWords is not a 32-bit LargePositiveInteger.
Essential. See Object documentation whatIsAPrimitive."
<primitive: 116>
self primitiveFailed
But this and its support is gone. Hmm. Hmmph :)
> Preferrably, dynamic control over the limit would be nice: in a headless
> server you want other behavior than during interactive development.
>
Right. The low space process would readjust the value every time low space
fired. But one could (can) call the primitive at any time to install a new
limit.
> Sven
>
> > This is actually the most common situation for loop/cycle that either
> results in an unusable image or a crash.
> > Interrupting with CMD-. is not always possible.
> > A possible solution could be to just limit the stack size and throw an
> exception.
> >
> > For example, in LispWorks, it goes like this:
> >
> > CL-USER 5 > (defun foo () (cons (random 100) (foo)))
> > FOO
> >
> > CL-USER 6 > (foo)
> >
> > Stack overflow (stack size 48128).
> > 1 (continue) Extend stack by 50%.
> > 2 (abort) Return to level 0.
> > 3 Return to top loop level 0.
> >
> > Type :b for backtrace or :c <option number> to proceed.
> > Type :bug-form "<subject>" for a bug report template or :? for other
> options.
> >
> > CL-USER 7 : 1 > :bq
> >
> > ERROR <- RUNTIME:BAD-ARGS-OR-STACK <- LENGTH <- FOO <- FOO <- FOO <- FOO
> <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO <-
> FOO <- FOO <- FOO <- FOO <- FOO <- FOO <- FOO
> > <- FOO <- FOO <- EVAL <- CAPI::CAPI-TOP-LEVEL-FUNCTION <-
> CAPI::INTERACTIVE-PANE-TOP-LOOP <- MP::PROCESS-SG-FUNCTION
> >
> > Obviously, this is possible, the question is, can it be done easily and
> without paying an unbearable performance price ?
> >
> > Sven
> >
> > --
> > Sven Van Caekenberghe
> > http://stfx.eu
> > Smalltalk is the Red Pill
> >
> >
> >
> >
> >
> >
> >
> > --
> > best,
> > Eliot
> >
>
> --
> Sven Van Caekenberghe
> http://stfx.eu
> Smalltalk is the Red Pill
>
>
>
>
>
--
best,
Eliot
Dec. 11, 2012
[Pharo-project] pharo-2.0-tests » win - Build # 499 - Failure!
by jenkins-pharo.ci.inria.frï¼ jenkins-server.lille.inria.fr
Dec. 11, 2012
Re: [Pharo-project] About (backwards) Compatibility
by dimitris chloupis
There is definitely the other side of things, that side that says you cant have an omelet if you don't brake some eggs. I always admire people that go against the flow , people who are willing to sacrifice convenience for progress. ÂÂ
I do believe in the direction pharo is going , I do believe that keeping code clean and reliable should be main priority. And even though people love to call attempts dead and futile , ideas are near impossible to kill because they always find brains to flourish on. And even a bad idea in time can turn into something great.
Because as one Greek philosopher once said "everything is changing" , everything has a life of its own, and as far as software is concern as long as there are coders for it , it aint dead yet. And there definitely benefits to be gained from braking compatibility. But progress is definitely a tricky things full of pitfalls and new is not always better. Its ironic how software seems to going back through time nowdays and rediscovering old relics like smalltalk and lisp. And "old" becomes the new "new". Definitely keeping an open mind is the wise thing to do any direction one may chooses to go. And I am certainly going to follow pharo and squeak progress closely because both have still a lot to say.
________________________________
From: Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com>
To: Pharo-project(a)lists.gforge.inria.fr; ma.chris.m(a)gmail.com
Sent: Tuesday, 11 December 2012, 22:50
Subject: Re: [Pharo-project] About (backwards) Compatibility
2012/12/11 Chris Muller <asqueaker(a)gmail.com>:
> If my goal is to make my computer work for ME, then I want my
> development system to maximize my leverage and minimize my effort.
> Breaking compatibility for cleaner code subverts this, as Hannes said,
>
One thing is clear, forking is not anything near minimizing the
effort, quite the opposite.
It's the way to build one's own jail - some will say to be free ;).
Clearly, Marcus and Stephane have other goals and it's a good thing to
have both Squeak and Pharo.
>>> Constant input in maintenance effort is needed.
>
> I want to use my time applying Smalltalk to real-world problems, not
> API changes.
>
> A blue-plane innovation is worth breaking backward compatibility,
> reinventing the past isn't.
>
Ah, if future is the past, then I wonder whether it's not yet another
damned Smalltalk circularity trick ;)
>
>>> maintaining libraries and maybe compatibility layers are very welcome.
>>>
>> Yes, and how did we ever thought we could invent the future with Squeak
>> when in reality, we could not even change a typo in a comment?
>>
>>    Marcus
>>
>>
>>
>>
>
Dec. 11, 2012
Re: [Pharo-project] About (backwards) Compatibility
by Nicolas Cellier
2012/12/11 Chris Muller <asqueaker(a)gmail.com>:
> If my goal is to make my computer work for ME, then I want my
> development system to maximize my leverage and minimize my effort.
> Breaking compatibility for cleaner code subverts this, as Hannes said,
>
One thing is clear, forking is not anything near minimizing the
effort, quite the opposite.
It's the way to build one's own jail - some will say to be free ;).
Clearly, Marcus and Stephane have other goals and it's a good thing to
have both Squeak and Pharo.
>>> Constant input in maintenance effort is needed.
>
> I want to use my time applying Smalltalk to real-world problems, not
> API changes.
>
> A blue-plane innovation is worth breaking backward compatibility,
> reinventing the past isn't.
>
Ah, if future is the past, then I wonder whether it's not yet another
damned Smalltalk circularity trick ;)
>
>>> maintaining libraries and maybe compatibility layers are very welcome.
>>>
>> Yes, and how did we ever thought we could invent the future with Squeak
>> when in reality, we could not even change a typo in a comment?
>>
>> Marcus
>>
>>
>>
>>
>
Dec. 11, 2012
Re: [Pharo-project] About (backwards) Compatibility
by Chris Muller
If my goal is to make my computer work for ME, then I want my
development system to maximize my leverage and minimize my effort.
Breaking compatibility for cleaner code subverts this, as Hannes said,
>> Constant input in maintenance effort is needed.
I want to use my time applying Smalltalk to real-world problems, not
API changes.
A blue-plane innovation is worth breaking backward compatibility,
reinventing the past isn't.
>> maintaining libraries and maybe compatibility layers are very welcome.
>>
> Yes, and how did we ever thought we could invent the future with Squeak
> when in reality, we could not even change a typo in a comment?
>
> Marcus
>
>
>
>
Dec. 11, 2012
[Pharo-project] another kid born: NBExternalEnumerationType
by Ciprian Teodorov
Hi guys,
today I have commited to the NativeBoost repository a new core class, the
NBExternalEnumerationType,
which reifies the enum declarations in C and offers a nice interface
(especially in terms of debug/inspect).
To use it just subclass the NBExternalEnumeration and add an
#enumDeclmethod to the class side such as:
NBExternalEnumeration subclass: #NBTestEnumeration
instanceVariableNames: ''
classVariableNames: ''
poolDictionaries: ''
category: 'NativeBoost-Tests'
NBTestEnumeration class>>enumDecl
^ {(#AAA -> 1).
(#BBB -> 2).
(#CCC -> 3).
(#DDD -> 2400)} asDictionary
DO NOT FORGET to call the #initialize method on your class. The rest is
automatically done ;)
You can use your new enum in two ways:
- add it to a client class poolDictionaries list (see
#NBExternalEnumTests for an example), and then just write CCC in your
code -- CCC here is an item of your enum
- or you can send the name of an item to your class --- NBTestEnumeration
DDD
The NBExternalEnumeration implements (instance and class) some more API
methods like:
- #itemAt: retrieves the item having a specific value --- NBTestEnumeration
itemAt: 2
- #includes: checks the existence of a specific item in the enum
--- NBTestEnumeration
includes: #AAA
Having this already gives us a nice interface to working with enums. Now I
plan to improve a little bit on performance and usage scenarios:
- the possibility to accept smalltalk integers as valid input/output for
functions(or callbacks) that use enums as arguments/return.
- support the different representation types for the enum values
(int/uint)
So that will be for now. Feel free to try it out and send me any comments
and/or improvement ideas, It will be my pleasure to improve it.
Cheers,
--
Dr. Ciprian TEODOROV
Ingénieur Développement CAO
tél : 06 08 54 73 48
mail : ciprian.teodorov(a)gmail.com
www.teodorov.ro
Dec. 11, 2012