Methods with more than 15 args sketch
Hi everyone. It seems that some people are struggling with the fact that methods in Pharo (and other Smalltalk on top of Cog) cannot have more than 15 arguments. Typically the problem arises while generating Smalltalk code or porting code from other Smalltalks. I have been thinking for a while about this problem. In this mail I write a small sketch to solve it, it's something that could go to production very quickly and that requires almost only image-side changes. Eliot has already reviewed and approved the sketch, but I''d like to know if someone is strongly against that change. The generic idea is to change the send calling convention at the bytecode level, using a temp vector to pass overflowing arguments. The normal calling convention in the Smalltalk bytecode is as follows: push receiver push arg 1 push arg 2 .... push arg N send selector N is limited to 15 arguments due compiled method header & bytecode encoding. In the new design, if the send has less than 15 arguments, the calling convention remains the same. Over 15 arguments, the overflowing arguments are passed in a temp vector in a similar way to the closure temp vectors. The convention will look like that: push receiver push arg 1 push arg 2 .... push arg N popIntoArray N-15 values send selector The compilation of a method with more than 15 arguments is then changed: - As for temp vectors, the 15th arg array can't be become or read-only, hence the temp vector instructions can be used to access the overflowing arguments through the 15th arg. - the compiled method header is still marked with 15 arguments In addition one needs to change CompiledMethod to: - allow larger frames (instead of 56 we could allow 128 or 256) - have numArgs answering the right number of methods with many arguments. I think the number of arguments could be in the additional method state in this case as it is very uncommon. And to change the fall-back of a couple primitives, such as: - Object>>perform: withArgs: - Object>>perform: withArgs:inSupedClass: - BlockClosure>>valueWithArgs: This design would allow methods to up to 141 arguments. What do you guys think ?
Hi, Does anyone want to pick this up on the Pharo image side? I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground) In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good. - If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish. - If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong. Did I misunderstand something? Cheers, Doru
On Jun 16, 2016, at 8:10 AM, Clément Bera <bera.clement@gmail.com> wrote:
Hi everyone.
It seems that some people are struggling with the fact that methods in Pharo (and other Smalltalk on top of Cog) cannot have more than 15 arguments. Typically the problem arises while generating Smalltalk code or porting code from other Smalltalks.
I have been thinking for a while about this problem. In this mail I write a small sketch to solve it, it's something that could go to production very quickly and that requires almost only image-side changes. Eliot has already reviewed and approved the sketch, but I''d like to know if someone is strongly against that change.
The generic idea is to change the send calling convention at the bytecode level, using a temp vector to pass overflowing arguments.
The normal calling convention in the Smalltalk bytecode is as follows:
push receiver push arg 1 push arg 2 .... push arg N send selector
N is limited to 15 arguments due compiled method header & bytecode encoding.
In the new design, if the send has less than 15 arguments, the calling convention remains the same. Over 15 arguments, the overflowing arguments are passed in a temp vector in a similar way to the closure temp vectors.
The convention will look like that:
push receiver push arg 1 push arg 2 .... push arg N popIntoArray N-15 values send selector
The compilation of a method with more than 15 arguments is then changed: - As for temp vectors, the 15th arg array can't be become or read-only, hence the temp vector instructions can be used to access the overflowing arguments through the 15th arg. - the compiled method header is still marked with 15 arguments
In addition one needs to change CompiledMethod to: - allow larger frames (instead of 56 we could allow 128 or 256) - have numArgs answering the right number of methods with many arguments. I think the number of arguments could be in the additional method state in this case as it is very uncommon.
And to change the fall-back of a couple primitives, such as: - Object>>perform: withArgs: - Object>>perform: withArgs:inSupedClass: - BlockClosure>>valueWithArgs:
This design would allow methods to up to 141 arguments.
What do you guys think ?
-- www.tudorgirba.com www.feenk.com "It's not what we do that matters most, it's how we do it."
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦) What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail). Marcus
Hi,
On Jun 20, 2016, at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦)
Sure. No problem. I asked the question because I am new in this area and I just wanted to see if I maybe I misunderstood something.
What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail).
Yes, I know :). So, I come back to my original question: Is anyone interested in following this up on the image level in Pharo? Cheers, Doru
Marcus
-- www.tudorgirba.com www.feenk.com "Problem solving efficiency grows with the abstractness level of problem understanding."
On 20 Jun 2016, at 08:58, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jun 20, 2016, at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦)
Sure. No problem. I asked the question because I am new in this area and I just wanted to see if I maybe I misunderstood something.
What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail).
Yes, I know :). So, I come back to my original question: Is anyone interested in following this up on the image level in Pharo?
Isnâ t that what the mail of clement was all about? To change the compiler to emit code according to the proposed scheme? Marcus
Hi,
On Jun 20, 2016, at 9:03 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 20 Jun 2016, at 08:58, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jun 20, 2016, at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦)
Sure. No problem. I asked the question because I am new in this area and I just wanted to see if I maybe I misunderstood something.
What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail).
Yes, I know :). So, I come back to my original question: Is anyone interested in following this up on the image level in Pharo?
Isnâ t that what the mail of clement was all about? To change the compiler to emit code according to the proposed scheme?
Yes, it is. But, as that is only a mail, the question is who is going to implement this concretely :) Doru
Marcus
-- www.tudorgirba.com www.feenk.com "One cannot do more than one can do."
On Mon, Jun 20, 2016 at 9:03 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 20 Jun 2016, at 08:58, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jun 20, 2016, at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the
errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet:
http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦)
Sure. No problem. I asked the question because I am new in this area and I just wanted to see if I maybe I misunderstood something.
What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail).
Yes, I know :). So, I come back to my original question: Is anyone interested in following this up on the image level in Pharo?
Isnâ t that what the mail of clement was all about? To change the compiler to emit code according to the proposed scheme?
I think the question Tudor asked is: Is anyone willing to implement the scheme I detailed ?
Marcus
Hi,
On Jun 20, 2016, at 10:07 AM, Clément Bera <bera.clement@gmail.com> wrote:
On Mon, Jun 20, 2016 at 9:03 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
[â¦] Isnâ t that what the mail of clement was all about? To change the compiler to emit code according to the proposed scheme?
I think the question Tudor asked is: Is anyone willing to implement the scheme I detailed ?\
Indeed, this is my question :). Doru -- www.tudorgirba.com www.feenk.com "Value is always contextual."
Hi Doru, On Thu, Jun 23, 2016 at 1:24 PM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
On Jun 20, 2016, at 10:07 AM, Clément Bera <bera.clement@gmail.com> wrote:
On Mon, Jun 20, 2016 at 9:03 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
[â¦] Isnâ t that what the mail of clement was all about? To change the compiler to emit code according to the proposed scheme?
I think the question Tudor asked is: Is anyone willing to implement the scheme I detailed ?\
Indeed, this is my question :).
I wil do a prototype in Squeak as soon as time is available (I have some Cadence things to finish first). Then I might try and do it in Pharo.
Doru
-- www.tudorgirba.com www.feenk.com
"Value is always contextual."
-- _,,,^..^,,,_ best, Eliot
Hi, I think there was a misunderstanding Tim. The idea is not necessarily to let developers write code with this many arguments, the IDE could actually enforces the programmer not to write methods with too many arguments. It's really about non programmers, like code generating tools such as SmaCC. In Java for example, the "Clean code" code convention limits the number of arguments per method to 3, the "Code complete" code convention limits it to 7, but Java can compile methods up to 255 arguments for code generators. If you want to limit the number of arguments the programmer can write, enforce it in the IDE according to your code convention. *Please talk about the maximum number of arguments a developer can write in another thread, this is non related to the implementation discussed here.* As Jan said, there are already too many limitations in Smalltalk to write proper code generators. We've worked on those limitations already: - the SistaV1 bytecode set will go to production in Pharo 6 (likely Squeak at some point too) and it removes an important part of the limitations (jump sizes up to 65k instructions instead of 1024, ...). - Spur removed other limitations (methods with up to 32k literals instead of 255, behaviors with up to 65k inst vars instead of 255). The important limitations remaining are the max frame size (56), which in practice limits the number of temps to around 55, whereas we could have 64 temps or more, and the number of arguments of methods and blocks. This scheme is about solving those problems. There are 0.5% of methods using a large frame so we could grow from 56 to 128 or 256 without any significant problems. For arguments bytecode calling convention looks so much simpler than VM support. On Mon, Jun 20, 2016 at 8:30 AM, Marcus Denker <marcus.denker@inria.fr> wrote:
On 19 Jun 2016, at 11:35, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Does anyone want to pick this up on the Pharo image side?
I took a brief look at this and while playing I noticed that the errors when we send more than 15 arguments seem a bit strange. Take a look at this snippet: http://ws.stfx.eu/958BXP5GR136 (you can paste it in Spotter to get the playground)
In summary: - If we compile a method with a signature containing more than 15 arguments we get: "SyntaxErrorNotification: Too many argumentsââ. This is good.
- If we compile a method with a message send containing between 16-31 arguments we get: "'InMidstOfFileinNotificationââ. Ok-ish.
- If we compile a method with a message send containing more than 31 arguments we get: "'Error: genSend:numArgs: numArgs index 32 is out of range 0 to 31â" This is wrong.
Did I misunderstand something?
The VM does not support it, and as nobody is using it, the error messages are not that perfect. (to have them nicer, someone would have seen it and be bothered to fix itâ¦)
What Clement proposes is to implement support for more than 15 arguments in the compiler. (which means that instead of error messages, it would compile, with the idea as he described in the mail).
Marcus
participants (4)
-
Clément Bera -
Eliot Miranda -
Marcus Denker -
Tudor Girba