[Pharo-project] Block questions
I'm taking some times to go back the block closure chapter and I was wondering how I can get access to block temporaries [ |a | a :=2. a] for example I tried [ |a | a :=2. a] outerContext contextForLocalVariables tempAt: 1 and variations but none worked. Stef
b := [ |a| a:=2 ]. b asContext tempNames -> #('a') b asContext tempNamed: 'a' -> Boom in MethodContext>>#tempAt: primitive tempAt: in Method context uses <primitive: 210>, which is failing. Maybe there is a new primitive we are not aware of for Blocks... On Sat, Apr 14, 2012 at 1:09 PM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
I'm taking some times to go back the block closure chapter and I was wondering how I can get access to block temporaries
[ |a | a :=2. a]
for example I tried
[ |a | a :=2. a]
outerContext contextForLocalVariables tempAt: 1 and variations but none worked.
Stef
On Sun, Apr 15, 2012 at 1:22 AM, Guillermo Polito <guillermopolito@gmail.com
wrote:
b := [ |a| a:=2 ]. b asContext tempNames -> #('a') b asContext tempNamed: 'a' -> Boom in MethodContext>>#tempAt: primitive
tempAt: in Method context uses <primitive: 210>, which is failing. Maybe there is a new primitive we are not aware of for Blocks...
No, 210 is what is needed. The issue is my implementation of temporary initialization in blocks, which simply uses pushNil to allocate and initialize each temp. So if one inspects [|a|a:=2] and sends it self method symbolic you get: 13 <8F 00 00 05> closureNumCopied: 0 numArgs: 0 bytes 17 to 21 17 <73> pushConstant: nil 18 <77> pushConstant: 2 19 <81 40> storeIntoTemp: 0 21 <7D> blockReturn 22 <7C> returnTop And when you say self asContext pc you get 17, which is *before* the nil is pushed. So you'll want to modify tempNamed: to compare the index of the temporary with the stack pointer and answer nil if the index is beyond the stack pointer. The same bug exists in Squeak, e.g. the below gave an error until I fixed DebuggerMethodMap>>namedTempAt:in: | b | b := [ |a| a:=2 ]. { b asContext tempNames. b method debuggerMap tempsAndValuesForContext: b asContext } #(#('b' 'a') 'b: [closure] in UndefinedObject>>DoIt a: nil ') Why did I use pushNil to allocate temps? It was not a good choice; it means for example that to determine how many temps a block has one has to parse all the bytecodes and see where the stack pointer is at the end (since initial pushNils are ambiguous; an initial pushNil might be producing a parameter or result or initializing a temp). But not using pushNil would have meant a more complex bytecode for closure creation plus another inst var in BlockClosure. I made another similar misstep in not storing the method in a BlockClosure, hence preventing simple implementation of clean blocks (one needs to synthesize an outerContext also). We live and learn.
On Sat, Apr 14, 2012 at 1:09 PM, Stéphane Ducasse < stephane.ducasse@inria.fr> wrote:
I'm taking some times to go back the block closure chapter and I was wondering how I can get access to block temporaries
[ |a | a :=2. a]
for example I tried
[ |a | a :=2. a]
outerContext contextForLocalVariables tempAt: 1 and variations but none worked.
Stef
-- best, Eliot
Thanks eliot I'm refreshing my fuzzy understanding of closure implementation details (I would love to avoid to have students, kids, bosses and ⦠around to get some calm time :). I will start rereading your blog.
tempAt: in Method context uses <primitive: 210>, which is failing. Maybe there is a new primitive we are not aware of for Blocks...
No, 210 is what is needed. The issue is my implementation of temporary initialization in blocks, which simply uses pushNil to allocate and initialize each temp. So if one inspects [|a|a:=2] and sends it self method symbolic you get:
13 <8F 00 00 05> closureNumCopied: 0 numArgs: 0 bytes 17 to 21 17 <73> pushConstant: nil 18 <77> pushConstant: 2 19 <81 40> storeIntoTemp: 0 21 <7D> blockReturn 22 <7C> returnTop
And when you say self asContext pc you get 17, which is *before* the nil is pushed. So you'll want to modify tempNamed: to compare the index of the temporary with the stack pointer and answer nil if the index is beyond the stack pointer.
Is it correct? I think so but I'm learning ContextPart tempNamed: aName "Returns the value of the temporaries, aName." "Implementation notes: temporary initialization in blocks simply uses pushNil to allocate and initialize each temp. So if one inspects [|a|a:=2] and sends it self method symbolic you get: 13 <8F 00 00 05> closureNumCopied: 0 numArgs: 0 bytes 17 to 21 17 <73> pushConstant: nil 18 <77> pushConstant: 2 19 <81 40> storeIntoTemp: 0 21 <7D> blockReturn 22 <7C> returnTop And when we check self asContext pc we get 17, which is *before* the nil is pushed. Therefore we should pay attention when querying a temporary if the temporary allocation was executed." | index | index := (self tempNames indexOf: aName). ^ index >= stackp ifTrue: [ nil] ifFalse: [self tempAt: (self tempNames indexOf: aName)]
The same bug exists in Squeak, e.g. the below gave an error until I fixed DebuggerMethodMap>>namedTempAt:in:
I checked and this method is the same in pharo and latest squeak namedTempAt: index in: aContext "Answer the value of the temp at index in aContext where index is relative to the array of temp names answered by tempNamesForContext:" ^self privateTempAt: index in: aContext startpcsToBlockExtents: aContext method startpcsToBlockExtents
| b | b := [ |a| a:=2 ]. { b asContext tempNames. b method debuggerMap tempsAndValuesForContext: b asContext } #(#('b' 'a') 'b: [closure] in UndefinedObject>>DoIt a: nil ')
Why did I use pushNil to allocate temps? It was not a good choice; it means for example that to determine how many temps a block has one has to parse all the bytecodes and see where the stack pointer is at the end (since initial pushNils are ambiguous; an initial pushNil might be producing a parameter or result or initializing a temp). But not using pushNil would have meant a more complex bytecode for closure creation plus another inst var in BlockClosure. I made another similar misstep in not storing the method in a BlockClosure, hence preventing simple implementation of clean blocks (one needs to synthesize an outerContext also). We live and learn.
Do you mean that the closure cannot access directly method but should do go to its outer context to access the methods. When this is needed to set the instruction pointer? Another thoughts while writing the chapter on block execution.. Am I correct to say: we need contexts (to me contexts are activation objects by opposition to bindings which could be used to represent scope) to query local temps of a closure because they will be allocated at execution while the temporaries that are shared are defined in their defining environment (now the trick is that in Smalltalk environment is represented as a context). I get why I'm often slow because I did not implemented closure in Smalltalk only in lisp foo |b| b := 3. ^ [ |a | a :=2. a] In my stupid lisp implementation an environment is represented by bindings and closure points to their own environment so [ |a | a :=2. a] points to (b.3) (toplevel) (and not top-level) and for closure execution we just added temporaly a binding for a to the closure environment. eval a:=2. a in (a) (b.3) (toplevel)
On Sat, Apr 21, 2012 at 12:51 AM, Stéphane Ducasse < stephane.ducasse@inria.fr> wrote:
Thanks eliot
I'm refreshing my fuzzy understanding of closure implementation details (I would love to avoid to have students, kids, bosses and ⦠around to get some calm time :).
I will start rereading your blog.
tempAt: in Method context uses <primitive: 210>, which is failing. Maybe there is a new primitive we are not aware of for Blocks...
No, 210 is what is needed. The issue is my implementation of temporary initialization in blocks, which simply uses pushNil to allocate and initialize each temp. So if one inspects [|a|a:=2] and sends it self method symbolic you get:
13 <8F 00 00 05> closureNumCopied: 0 numArgs: 0 bytes 17 to 21 17 <73> pushConstant: nil 18 <77> pushConstant: 2 19 <81 40> storeIntoTemp: 0 21 <7D> blockReturn 22 <7C> returnTop
And when you say self asContext pc you get 17, which is *before* the nil is pushed. So you'll want to modify tempNamed: to compare the index of the temporary with the stack pointer and answer nil if the index is beyond the stack pointer.
Is it correct? I think so but I'm learning ContextPart
tempNamed: aName "Returns the value of the temporaries, aName." "Implementation notes: temporary initialization in blocks simply uses pushNil to allocate and initialize each temp. So if one inspects [|a|a:=2] and sends it self method symbolic you get:
13 <8F 00 00 05> closureNumCopied: 0 numArgs: 0 bytes 17 to 21 17 <73> pushConstant: nil 18 <77> pushConstant: 2 19 <81 40> storeIntoTemp: 0 21 <7D> blockReturn 22 <7C> returnTop
And when we check self asContext pc we get 17, which is *before* the nil is pushed. Therefore we should pay attention when querying a temporary if the temporary allocation was executed."
| index | index := (self tempNames indexOf: aName). ^ index >= stackp ifTrue: [ nil] ifFalse: [self tempAt: (self tempNames indexOf: aName)]
The same bug exists in Squeak, e.g. the below gave an error until I
fixed DebuggerMethodMap>>namedTempAt:in:
I checked and this method is the same in pharo and latest squeak namedTempAt: index in: aContext "Answer the value of the temp at index in aContext where index is relative to the array of temp names answered by tempNamesForContext:" ^self privateTempAt: index in: aContext startpcsToBlockExtents: aContext method startpcsToBlockExtents
| b | b := [ |a| a:=2 ]. { b asContext tempNames. b method debuggerMap tempsAndValuesForContext: b asContext } #(#('b' 'a') 'b: [closure] in UndefinedObject>>DoIt a: nil ')
Why did I use pushNil to allocate temps? It was not a good choice; it means for example that to determine how many temps a block has one has to parse all the bytecodes and see where the stack pointer is at the end (since initial pushNils are ambiguous; an initial pushNil might be producing a parameter or result or initializing a temp). But not using pushNil would have meant a more complex bytecode for closure creation plus another inst var in BlockClosure. I made another similar misstep in not storing the method in a BlockClosure, hence preventing simple implementation of clean blocks (one needs to synthesize an outerContext also). We live and learn.
Do you mean that the closure cannot access directly method but should do go to its outer context to access the methods. When this is needed to set the instruction pointer?
Another thoughts while writing the chapter on block execution..
Am I correct to say: we need contexts (to me contexts are activation objects by opposition to bindings which could be used to represent scope) to query local temps of a closure because they will be allocated at execution while the temporaries that are shared are defined in their defining environment (now the trick is that in Smalltalk environment is represented as a context).
I get why I'm often slow because I did not implemented closure in Smalltalk only in lisp
foo
|b| b := 3. ^ [ |a | a :=2. a]
In my stupid lisp implementation an environment is represented by bindings and closure points to their own environment so [ |a | a :=2. a] points to (b.3) (toplevel) (and not top-level)
and for closure execution we just added temporaly a binding for a to the closure environment. eval a:=2. a in (a) (b.3) (toplevel)
This is the wrong example. You need to consider how to implement this example: Object subclass: #AbstractSuperclass AbstractSuperclass>>computeV | v | v := 0. self maybeCapture: [v := v + 1. v]. ^self maybeBlock: v AbstractSuperclass subclass: #CaptureBlock instanceVariableNames: 'block' CaptureBlock>>maybeCapture: aBlock block := aBlock CaptureBlock>>maybeBlock: value ^block AbstractSubclass subclass: #DontCaptureBlock DontCaptureBlock>>maybeCapture: aBlock DontCaptureBlock>>maybeBlock: value ^value Then | thingOne thingTwo | thingOne := CaptureBlock computeV. thingTwo := DontCaptureBlock computeV. (1 to: 10) collect: [:ignore| { thingOne value. thingTwo value }] should answer #((1 0) (2 0) (3 0) ... (10 0)) right? So where does the l-value for v in computeV live? Iy must be an l-value since it is assigned to in the block in computeV. It must outlive the activation of computeV since CaptureBlock>maybeCapture: captures the block and CaptureBlock>>maybeBlock: answers it and the doit evaluates it. If it lives on the stack of the activation of computeV then there is no problem in a Context VM, but in a VM that maps activations to stack frames something special (and slow) has to happen when returning from computeV. However, if the l-value lives in a separate Array (as it does in my VisualWorks and Squeak closure implementations, and as happens in some Lisp implementations), nothing special has to happen. The block refers to the Array, *not* to the activation of computeV. Make sense now? -- best, Eliot
foo
|b| b := 3. ^ [ |a | a :=2. a]
In my stupid lisp implementation an environment is represented by bindings and closure points to their own environment so [ |a | a :=2. a] points to (b.3) (toplevel) (and not top-level)
and for closure execution we just added temporaly a binding for a to the closure environment. eval a:=2. a in (a) (b.3) (toplevel)
This is the wrong example.
I understand your example below and the fact that closure environment is allocated on the heap via an array to be independent of the activation context but I do not get why consing the local environment to be one current one is wrong. (define (%apply proc largs) (cond ((%primitive? proc) (%apply-internal proc largs)) ((%closure? proc) (%apply-closure proc largs)) (else (error "Bad ! Un-apply-able object !" proc)))) (define (%apply-closure proc largs) ;; apply a closure: evaluate proc body in ;; extended the closure environment with ;; proc arguments and largs (%eval (%closure-body proc) (%extend-env (%closure-args proc) largs (%closure-env proc)))) (define (%eval-lambda expr env) ;; lambda in Scheme captures the environment at compile time ;; expr = ((x) (+ x 2)) (%make-closure (car expr) (cadr expr) env))
You need to consider how to implement this example:
Object subclass: #AbstractSuperclass
AbstractSuperclass>>computeV | v | v := 0. self maybeCapture: [v := v + 1. v]. ^self maybeBlock: v
AbstractSuperclass subclass: #CaptureBlock instanceVariableNames: 'block'
CaptureBlock>>maybeCapture: aBlock block := aBlock
CaptureBlock>>maybeBlock: value ^block
AbstractSubclass subclass: #DontCaptureBlock
DontCaptureBlock>>maybeCapture: aBlock
DontCaptureBlock>>maybeBlock: value ^value
Then | thingOne thingTwo | thingOne := CaptureBlock computeV. thingTwo := DontCaptureBlock computeV. (1 to: 10) collect: [:ignore| { thingOne value. thingTwo value }]
should answer #((1 0) (2 0) (3 0) ... (10 0))
if 0 value returns 0? yes. (it confused me for a while).
right? So where does the l-value for v in computeV live? Iy must be an l-value since it is assigned to in the block in computeV. It must outlive the activation of computeV since CaptureBlock>maybeCapture: captures the block and CaptureBlock>>maybeBlock: answers it and the doit evaluates it. If it lives on the stack of the activation of computeV then there is no problem in a Context VM, but in a VM that maps activations to stack frames something special (and slow) has to happen when returning from computeV. However, if the l-value lives in a separate Array (as it does in my VisualWorks and Squeak closure implementations, and as happens in some Lisp implementations), nothing special has to happen. The block refers to the Array, *not* to the activation of computeV.
Make sense now?
Yes you allocate on the heap closure binding to be orthogonal to activation context. I will reread your blog now. Stef
On Mon, Apr 30, 2012 at 4:01 AM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
foo
|b| b := 3. ^ [ |a | a :=2. a]
In my stupid lisp implementation an environment is represented by
bindings and closure points to their own environment
so [ |a | a :=2. a] points to (b.3) (toplevel) (and not top-level)
and for closure execution we just added temporaly a binding for a to the closure environment. eval a:=2. a in (a) (b.3) (toplevel)
This is the wrong example.
I understand your example below and the fact that closure environment is allocated on the heap via an array to be independent of the activation context but I do not get why consing the local environment to be one current one is wrong.
I mean the example is not a useful example because it doesn't show the problem "my" representation solves, which is to make closures independent of their enclosing activations if they outlive their dynamic extent.
(define (%apply proc largs) (cond ((%primitive? proc) (%apply-internal proc largs)) ((%closure? proc) (%apply-closure proc largs)) (else (error "Bad ! Un-apply-able object !" proc))))
(define (%apply-closure proc largs) ;; apply a closure: evaluate proc body in ;; extended the closure environment with ;; proc arguments and largs (%eval (%closure-body proc) (%extend-env (%closure-args proc) largs (%closure-env proc))))
(define (%eval-lambda expr env) ;; lambda in Scheme captures the environment at compile time ;; expr = ((x) (+ x 2)) (%make-closure (car expr) (cadr expr) env))
You need to consider how to implement this example:
Object subclass: #AbstractSuperclass
AbstractSuperclass>>computeV | v | v := 0. self maybeCapture: [v := v + 1. v]. ^self maybeBlock: v
AbstractSuperclass subclass: #CaptureBlock instanceVariableNames: 'block'
CaptureBlock>>maybeCapture: aBlock block := aBlock
CaptureBlock>>maybeBlock: value ^block
AbstractSubclass subclass: #DontCaptureBlock
DontCaptureBlock>>maybeCapture: aBlock
DontCaptureBlock>>maybeBlock: value ^value
Then | thingOne thingTwo | thingOne := CaptureBlock computeV. thingTwo := DontCaptureBlock computeV. (1 to: 10) collect: [:ignore| { thingOne value. thingTwo value }]
should answer #((1 0) (2 0) (3 0) ... (10 0))
if 0 value returns 0? yes. (it confused me for a while).
right? So where does the l-value for v in computeV live? Iy must be an l-value since it is assigned to in the block in computeV. It must outlive the activation of computeV since CaptureBlock>maybeCapture: captures the block and CaptureBlock>>maybeBlock: answers it and the doit evaluates it. If it lives on the stack of the activation of computeV then there is no problem in a Context VM, but in a VM that maps activations to stack frames something special (and slow) has to happen when returning from computeV. However, if the l-value lives in a separate Array (as it does in my VisualWorks and Squeak closure implementations, and as happens in some Lisp implementations), nothing special has to happen. The block refers to the Array, *not* to the activation of computeV.
Make sense now?
Yes you allocate on the heap closure binding to be orthogonal to activation context.
I will reread your blog now.
Stef
-- best, Eliot
I mean the example is not a useful example because it doesn't show the problem "my" representation solves, which is to make closures independent of their enclosing activations if they outlive their dynamic extent.
Yes I'm starting to finish the chapter explanation on explicit return and now I will go in "opening the trunk section" and study for real your implementation because I really want to fully understand it. Stef
(define (%apply proc largs) (cond ((%primitive? proc) (%apply-internal proc largs)) ((%closure? proc) (%apply-closure proc largs)) (else (error "Bad ! Un-apply-able object !" proc))))
(define (%apply-closure proc largs) ;; apply a closure: evaluate proc body in ;; extended the closure environment with ;; proc arguments and largs (%eval (%closure-body proc) (%extend-env (%closure-args proc) largs (%closure-env proc))))
(define (%eval-lambda expr env) ;; lambda in Scheme captures the environment at compile time ;; expr = ((x) (+ x 2)) (%make-closure (car expr) (cadr expr) env))
You need to consider how to implement this example:
Object subclass: #AbstractSuperclass
AbstractSuperclass>>computeV | v | v := 0. self maybeCapture: [v := v + 1. v]. ^self maybeBlock: v
AbstractSuperclass subclass: #CaptureBlock instanceVariableNames: 'block'
CaptureBlock>>maybeCapture: aBlock block := aBlock
CaptureBlock>>maybeBlock: value ^block
AbstractSubclass subclass: #DontCaptureBlock
DontCaptureBlock>>maybeCapture: aBlock
DontCaptureBlock>>maybeBlock: value ^value
Then | thingOne thingTwo | thingOne := CaptureBlock computeV. thingTwo := DontCaptureBlock computeV. (1 to: 10) collect: [:ignore| { thingOne value. thingTwo value }]
should answer #((1 0) (2 0) (3 0) ... (10 0))
if 0 value returns 0? yes. (it confused me for a while).
right? So where does the l-value for v in computeV live? Iy must be an l-value since it is assigned to in the block in computeV. It must outlive the activation of computeV since CaptureBlock>maybeCapture: captures the block and CaptureBlock>>maybeBlock: answers it and the doit evaluates it. If it lives on the stack of the activation of computeV then there is no problem in a Context VM, but in a VM that maps activations to stack frames something special (and slow) has to happen when returning from computeV. However, if the l-value lives in a separate Array (as it does in my VisualWorks and Squeak closure implementations, and as happens in some Lisp implementations), nothing special has to happen. The block refers to the Array, *not* to the activation of computeV.
Make sense now?
Yes you allocate on the heap closure binding to be orthogonal to activation context.
I will reread your blog now.
Stef
-- best, Eliot
Following eliot feedback, I fixed it see the bug tracker. But my question was totally stupid because when you have [ |a | a;=2 . ^ a ] | a | exists only when executed. and the reflective model of block does not let us simply gets the information. Stef On Apr 15, 2012, at 10:22 AM, Guillermo Polito wrote:
b := [ |a| a:=2 ]. b asContext tempNames -> #('a') b asContext tempNamed: 'a' -> Boom in MethodContext>>#tempAt: primitive
tempAt: in Method context uses <primitive: 210>, which is failing. Maybe there is a new primitive we are not aware of for Blocks...
On Sat, Apr 14, 2012 at 1:09 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote: I'm taking some times to go back the block closure chapter and I was wondering how I can get access to block temporaries
[ |a | a :=2. a]
for example I tried
[ |a | a :=2. a]
outerContext contextForLocalVariables tempAt: 1 and variations but none worked.
Stef
participants (3)
-
Eliot Miranda -
Guillermo Polito -
Stéphane Ducasse