Yes, I’ve reported that already and it was fixed in Pharo 4 Uko

On 03 Feb 2015, at 09:17, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 02 Feb 2015, at 21:47, Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>> wrote:

Hi All,

code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

<Untitled.png>

In general, I really am convinced that we should continue to reduce the number of Smalltalk parsers in the system. Imagine there would be only *one*, then bugs like this would never happen, we would have less code to maintain and every improvement would benefit directly all subsystems that use the Parser or the AST.

We already did that for the tools (RB) and the Compiler. We use the handwritten RBParser for now (I would like to use PetitParser…). The next would be to replace the Shout Parser+Tokenstream… we already in all tools create an AST as soon as you select a method anyway. (the is for per-AST-Node Navigation + the “suggestions” menu).

RBParser already has #parseFaultyMethod:, this means it can generate an AST even for incomplete input (with an error node). e.g. you can parse the expression ‘1+’:

<Untitled.png>

There is already a SHRBTextStyler which uses it, we should finish that and replace the Shout parser.

Marcus

On 03 Feb 2015, at 18:36, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 03 Feb 2015, at 09:17, Marcus Denker <marcus.denker@inria.fr <mailto:marcus.denker@inria.fr>> wrote:

...

On 02 Feb 2015, at 21:47, Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>> wrote:

Hi All,

code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

I should search the issue in the issue tracker… it seems to be Pharo4, so just 1296 closed issues to check there… I will search for it.

It seems to be this one: https://pharo.fogbugz.com/f/cases/14630/Pipes-method-highlighting integrated in Pharo4 update #420 Marcus

On Tue, Feb 3, 2015 at 7:06 PM, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi Marcus,

thanks! When I visit the fogbugz page there's a line that says "slice in inbox" but no link to any file containing the slice I can see. How do I track down the slice file itself?

The slice has the issue number as part of it's name. So you can just search for that in the inbox.

To the the changes, sometimes I just download the version one before integration ( 40 #419 here) from http://files.pharo.org/image/40/ and do a merge there. Marcus

On Tue, Feb 3, 2015 at 9:48 AM, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 03 Feb 2015, at 18:36, Marcus Denker <marcus.denker@inria.fr> wrote:

On 03 Feb 2015, at 09:17, Marcus Denker <marcus.denker@inria.fr> wrote:

On 02 Feb 2015, at 21:47, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi All,

code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

I should search the issue in the issue tracker… it seems to be Pharo4, so just 1296 closed issues to check there… I will search for it.

It seems to be this one:

https://pharo.fogbugz.com/f/cases/14630/Pipes-method-highlighting

integrated in Pharo4 update #420

Marcus

-- best, Eliot

-- -- Marcus Denker -- denker@acm.org http://www.marcusdenker.de

On Tue, Feb 3, 2015 at 6:51 PM, Levente Uzonyi <leves@elte.hu> wrote:

On Tue, 3 Feb 2015, Marcus Denker wrote:

...

On 03 Feb 2015, at 09:17, Marcus Denker <marcus.denker@inria.fr> wrote:

On 02 Feb 2015, at 21:47, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi All, code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

I should search the issue in the issue tracker… it seems to be Pharo4, so just 1296 closed issues to check there… I will search for it.

Another question: In the code I saw. ReadOnlyVariableBinding. I removed that in Pharo relatively early as it was not used: half of the classes were stored that binding (old ones) all newer ones where just associations. The code to make a binding "read only" was never called.

Is this now used in Squeak? Is it worth the complexity?

Squeak doesn't use ReadOnlyVariableBinding anymore. The bindings of classes are instances of the ClassBinding class. Without using separate class it's a bit cumbersome (and less OO) to decide if an assignment to a global variable should be allowed or not. E.g.:

Foo := 1.

should work if Foo is a global, but not a behavior. It should raise an error if it's a behavior.

I thought this is how ReadOnlyVariableBinding was supposed to be used: classes would use it, globals not. Marcus

Levente Uzonyi-2 wrote

On Tue, 3 Feb 2015, Marcus Denker wrote:

...

On 03 Feb 2015, at 09:17, Marcus Denker <

marcus.denker@

> wrote:

...

On 02 Feb 2015, at 21:47, Eliot Miranda <

eliot.miranda@

> wrote:

...

Hi All,     code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

I should search the issue in the issue tracker… it seems to be Pharo4, so just 1296 closed issues to check there… I will search for it.

Another question: In the code I saw. ReadOnlyVariableBinding. I removed that in Pharo relatively early as it was not used: half of the classes were stored that binding (old ones) all newer ones where just associations. The code to make a binding "read only" was never called.

Is this now used in Squeak? Is it worth the complexity?

Squeak doesn't use ReadOnlyVariableBinding anymore. The bindings of classes are instances of the ClassBinding class. Without using separate class it's a bit cumbersome (and less OO) to decide if an assignment to a global variable should be allowed or not. E.g.:

Foo := 1.

should work if Foo is a global, but not a behavior. It should raise an error if it's a behavior.

I have mixed feelings about that. In most Smalltalks one can write something like the following and expect it to work. | original | original := SomeClass. [SomeClass := SomeReplacementClass. ... some code involving SomeClass (typically a passed in Block) ... ] ensure: [SomeClass := original] That is, the name of the thing is not the thing itself. Of course, as long as one could write /Smalltalk at: #SomeClass put: SomeReplacementClass/, it would still be possible to achieve the same effect. (And of course, "Smalltalk" could be any namespace.)

Levente

...

Marcus

-- View this message in context: http://forum.world.st/-tp4803298p4803783.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine? Levente On Tue, 3 Feb 2015, Marcus Denker wrote:

...

On 02 Feb 2015, at 21:47, Eliot Miranda <eliot.miranda@gmail.com> wrote:

Hi All,

code as in the double bars forming the end of block arguments and the beginning of block temporaries in

This is fixed in Pharo4 (I think we did that in Pharo3 already):

In general, I really am convinced that we should continue to reduce the number of Smalltalk parsers in the system. Imagine there would be only *one*, then bugs like this would never happen, we would have less code to maintain and every improvement would benefit directly all subsystems that use the Parser or the AST.

We already did that for the tools (RB) and the Compiler. We use the handwritten RBParser for now (I would like to use PetitParser…). The next would be to replace the Shout Parser+Tokenstream… we already in all tools create an AST as soon as you select a method anyway. (the is for per-AST-Node Navigation + the “suggestions” menu).

RBParser already has #parseFaultyMethod:, this means it can generate an AST even for incomplete input (with an error node). e.g. you can parse the expression ‘1+’:

There is already a SHRBTextStyler which uses it, we should finish that and replace the Shout parser.

Marcus

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 04 Feb 2015, at 22:04, Levente Uzonyi <leves@elte.hu> wrote:

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine?

I have not yet benchmarked it… PetitParser as it is is too slow, but we will soon have a faster version (factor 10).

We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. (there is a setting 'AST based coloring’ in Pharo3 and Pharo4, but it is turned off by default).

One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined by the AST nodes and structural navigation in the editor.

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978? What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year… This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical. Marcus

On 05 Feb 2015, at 10:46, Thierry Goubier <thierry.goubier@gmail.com> wrote:

2015-02-05 10:12 GMT+01:00 Marcus Denker <marcus.denker@inria.fr>:

...

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 04 Feb 2015, at 22:04, Levente Uzonyi <leves@elte.hu> wrote:

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine?

I have not yet benchmarked it… PetitParser as it is is too slow, but we will soon have a faster version (factor 10).

We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. (there is a setting 'AST based coloring’ in Pharo3 and Pharo4, but it is turned off by default).

One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined by the AST nodes and structural navigation in the editor.

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

Badly for today's machines :)

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year…

You're spoiled by your nice and expensive macbooks :)

Honestly, on some of today's machines, you'd better avoid long methods or GT tools stuff because they are slow.

SmaCC code generation is slow.

PluggableTreeMorph is slow.

Nautilus is slow.

Loading large configurations is slow.

Roassal complex graphs are slow.

Serge's modeling stuff is slow.

PetitParser has performance problems.

BitBlt is slow (and Cairo has bugs).

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical.

Being radical in the context of Pharo is offering to remove stuff or layers :):)

Thierry

It is obviously a compromise (or a continuum) between abstractions and performance. But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

2015-02-05 11:14 GMT+01:00 kilon alios <kilon.alios@gmail.com>:

And there is also the matter of hardware evolution.

When I was running Pharo on my 2007 imac 20'' with dual core 2.GHZ , Morphic was slow. And by slow I mean that I could see it was struggling to move windows around. I could see windows flickering trying to render themselves moving around.

But now with my 2014 imac even though the screen is double the size and the resolution much bigger (27'' retina) , Morphic is now quite fast. The reason is of course the fact that the CPU is a quad core at 3 Ghz thats almost a 3x times increase in speed and it really shows.

Even when Pharo take full the huge area of 27'' Morphic is responsive and quite fast.

Which means if I see you complaining of speed issues, then it must be really bad :) Thierry

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :) Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*). Thierry (*) I tried to get rid of them in some of my stuff, where it is used in the same way GTInspector and GTSpooter uses pragmas, and I had to stop because of that feature. All other uses of pragmas are replaceable (and often, as in the gt stuff, redundant).

2015-02-05 18:51 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <thierry.goubier@gmail.com

...

wrote:

...

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

...

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax). They have been broadly used. IME they have simplified and reduced code where ever they have been used. They don't have to be there but they're a good thing. Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example: One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)). The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot). Each method is one line longer ("the pragma"). Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma. (There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: ) Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

(and yes, I'm biassed)

Then you're the right person to give me counter arguments... (Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with) Thierry (Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

2015-02-05 21:28 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

2015-02-05 18:51 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

...

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax). They have been broadly used. IME they have simplified and reduced code where ever they have been used. They don't have to be there but they're a good thing. Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

OK, the claim is too strong. But they /have/ simplified code in cases where they're appropriate. And I can cite several examples.

...

One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

But triggering in the background happens for maintaining change sets, notifying other clients too. It's not as if pragmas introduced such triggering; that kind of triggering has been in use for a long time. And being able to reshape the GUI automatically is very useful.

I don't contest the possibilities, it's just that they add a significant layer of complexity when non mastered (how many Pharo developpers know which event you have to register to to receive all new methods notifications? Is that documented in one of the books?), and that, except for using them as <primitives> or for extensibility, I see other syntaxes and smalltalk code which are simpler. A good example is that the pragma syntax is never included in the one page Smalltalk syntax description :)

...

...

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Why? A good use of pragmas is to associate meta data with a particular method. Having calls off to the side always introduces the need for book-keeping to keep those methods off to the side in sync with the methods they're describing. Typically everyone rolls their own. But here we're adding a level of triggering just to keep the metadata methods in sync.

I agree with the "metadata", but I'd prefer a executable, evaluate that block as a medata literal than the pragma. Something that says "onceAndStoreAsMetadata", to a block, for example. An API to compiled methods which says add metadata. I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods. Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

This is one idiom that covers a host of other cases. That's why I claim that whenever I've seen it used it has reduced complexity.

Some history. Steve Dahl, I developed pragmas at ParcPlace, with Vassili Bykov adding abstractions for accessing them. The first step was to replace some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by a pragma the compiler would recognise and set the bits itself. The first real use was to make the VisualWorks launcher's menus extensible. Before pragmas the launcher's menu was static and had lots of disabled entries for launching tools that were sold separately such as DLLAndCConnect. With pragmas the launcher's menu was defined with the base system's tools and then extended as each tool package was loaded, or cut-back as each tool was unloaded. So that decoupled the launcher from introducing new tools. A nice result.

We then started using it for the browser and one could plug-in a single tool without redefining the browser's menu methods, which decoupled each extension. All this was done in the context of the parcel system, where we could rapidly load packages (parcels ~= Fuel). Pragmas allowed us to decouple these tools where they collided in places like menu definition, tool registration.

Then Tami Lee, who was managing the COM connection that turned a VW image into a COM server, became the first "user" of pragmas outside of myself and Steve. She used it to replace a lot of class-side methods that defined the signatures of methods that comprised the server. It was a lovely clean-up. One could define the COM signature for a method in the method itself, and the class side lost about three separate methods that defined all that metadata. One could read the server method itself and understand its semantics without having to consult the class-side methods. One didn't have to know that there was metadata hidden on the class side because it was right there in your face.

Then Vassili used it for his cool inspector framework, Trippy, which was similar to Glamour in some ways, and was a huge improvement over the old Inspector framework, again resulting in a much more pluggable, decoupled and extensible system. Vassili also added the abstractions for accessing pragmas in methods.

Then we added checking so that one could restrict the compiler to accept only legal pragmas for a given class. But if we defined the legal pragmas in a class-side method, say legalPragmas, then this would be exactly the kind of single point for extensions that causes collisions between packages, each of which might want to add its own set of pragmas. The solution... use a pragma to mark a class-side method as defining a set of legal pragmas for a class. One could have more than one method defining a set of legal pragmas; packages wishing to add their own cool pragmas were decoupled. Once the system because recursive, it had to be a good idea ;-).

Ok, I start to see where the abstraction wasn't working so well... since pragmas are not executed, when writing a method you can't know if the pragma is correct, because even executing the method may not trigger the pragma induced code. So you need the legalPragmas to give metadata on metadata for the compiler to do a bit of static checking, but it doesn't work for system-wide pragmas unless you extend Object :( And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There are other uses; you've seen them. I used them in VMMaker to eliminate metadata that was embedded as sends to methods defined as ^self that Slang had to extract and analyse, and filter-out from generated code. They simplified Slang's code anaylsis, made the simulator more efficient (since there were no longer sends to execute). My point is that in all the cases I've seen, using pragmas has - simplified the code - made it obvious that methods have metadata associated with them - replaced specialized ways of associating metadata with code by the general pragma mechanism and in many of the cases it has - provided a more decoupled system - provided a more dynamic and extensible system

Yes, and I can point out some of its shortcomings: it's non-obvious, it's limited, tools, even that many years later don't support them well (in Squeak or Pharo, at least), its redundant in quite a few variants. Please, could we improve a bit? Methods belonging to multiple protocols would give us the same decoupling as pragmas, and I would be free to avoid them where I shouldn't have to use them :)

I've been meaning to write up the history of pragmas for ages, but Vassili, Steve or I have always been too busy. I think a community paper on their use and history would be worth-while, and might go a long way to reduce antipathies like yours. I will forever be in debt to anyone who wants to volunteer to help me write such a paper.

That would certainly be interesting :)

...

Thierry

(Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

They /have/ to follow literal message syntax. t's all the compiler will accept. That's why there needs to be a paper.

Yes! Thanks for taking the time to argument, Thierry

-- best, Eliot

Hi Eliot, maybe I'll frame the core question a bit differently, thanks to your explanations. I see pragmas used mainly for two things: 1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified. 2- categorizing things in a decoupled, package-extension compatible way. I still think there is something in the design that bother me. 1- and 2- are convention-based, so as is most of Smalltalk anyway, so diregarding another approach to 2- as convention-based is a bit dishonest: pragmas get more authority because they look like they have been integrated in the language definition compared to competing solutions. No, this is just because I start to wonder if methods shouldn't have a bit more in that "define, add and manipulate" metadata idea, i.e. make it more explicit and start to have tools which represent methods as more like objects and not only text items (methods having metadata, additional slots or instance variables, counters, statistics, ast, traces, break points, etc...). And, in a way, a need to have two levels of execution: one which happens at compile time (i.e. when one accept a method), one which happens at runtime. For that, I think pragmas are a start, but they are not the complete answer. And I think we should solve the package-extension compatible thing in 2- to not have 2- (which is just for me a plain categorisation issue) polluting the design by introducing additional/unrelated requirements. 2015-02-05 22:47 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

Hi Thierry,

A block is far more difficult to deal with than a pragma. A pragma has a simple key, its selector, and simple access to its arameters, the arguments. A block is opaque. Essentially it can only be executable. Pragmas are both executable and function as annotations.

Pragmas are executable, but in a context which is not visible at the place they are written.

...

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

...

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods. Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

Yes, but there end up being lots of naming conventions and they are non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

They are still conventions which needs documentation. Pragmas are, in allmost all uses, indirections (i.e. a tag meaning something to a far away object), and the meaning associated to a pragma is, in every case, a roll-your-own solution. And they are clearly "in your face" :)

That's not necessarily true. Some pragmas do cause processing at compile time. For example, an FFI signature pragma can be checked at compile-time. But it's in keeping with smalltalk that type checking is not performed at compile time in most cases, isn't it? Why should one require that pragmas be semantically checked at compile time when normal SMalltalk code isn't? At least one knows that the message instance the pragma is compiled to is a valid object and *can* be performed. So pone does know at least that the pragma is executable. Obviously whether that ability to be executed only becomes potent with the right receiver. So any compiled pragma as the potential to be usefully evaluated.

Ok, but its all fuzzy. It may be executed, but it may be not. It may send an error if incorrect, but it may fail silently. It may be executed at compile time, it may be executed sometime later triggered by the compilation, or it may be executed at runtime, as if it was normal code. And of course it may not be executed at all ... You see what I mean?

And often it doesn't matter if the pragma reference a completely non

...

existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There is lots about Smalltalk that is non-obvious, about programming in general that is non-obvious. I don't see that as a specific criticism of pragmas. Once one knows the idiom it is easy to use; its wide-spread use is evidence of that. And hopefully this conversation will help make it more obvious :-). That it is limited is also not perhaps a useful criticism. The issue is whether it is adequate. I think it is. A literal message goes a long way. I'e not heard of complaints about limitations so far. Do you have specific examples where pragmas are inadequate? The tools /do/ support them. One can do senders and implementors in the browser and see the methods that include them, and the implementors of pragmas. browseAllSelect: will narrow down the search. There is an API for programmatic use (pragmasDo: et al).

See [*][**] for the tools support. For me, pragmas, even in their definition, have serious semantic issues. If they were strictly, as you describe them, a metadata API access executed at compile time, then they would be a lot stronger. Make them sent at compile-time on the method object, and that would probably solve all the true arguments I have against them :) for example: <worldMenu> against <tagAndAnnounce: #worldMenu>; <tag: #gtInspectorExtension> instead of <gtInspectorExtension>, etc... (<tagAndAnnounce:> would gain a bit of performance, as well). Thierry [*] Searching a bit for menu pragmas in a Pharo image seems to return a bit of a mess. There is so many of them, all differents, and interestingly, this does not return the code triggered (or using) them, which makes discovery of what the pragma is used for (and its parameters) an 'interesting' problem. I suggest a task then: an illustrated sequence of how to search for the code triggering world menu addition of a new command, for someone which is new to Pharo ;) [**] even when used for categorisation, the API is prone to bugs, such as having a method without pragma overriding a superclass method with pragma because they have the same name.

Hi Ben, 2015-02-06 16:03 GMT+01:00 Ben Coman <btc@openinworld.com>:

On Fri, Feb 6, 2015 at 5:58 PM, Thierry Goubier <thierry.goubier@gmail.com

...

wrote:

...

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

btw, I just used a pragma to solve this issue of getting correct debugger behaviour for halting and stepping over #halt statements [1]. There seem about a dozen methods that can cause a #halt spread between Object and Halt classes which should all behave similarly, so the options would seem * hardcode all these methods either into to check for in Halt>>signalerContext (or into a helper method called from #singalerContext) * have a convention for a certain protocol to hold these methods, which Halt>>signalerContext can look up (except currently the methods on Object are an extension protocol) * tag each such method with a pragma. This simplified the solution a lot. The search of the call stack just asked each method if it had the <debuggerCompleteToSender> pragma.

Yes, I agree with your assessment. Solution 2 require multiple protocols for a single method to be extensible, and this usage is perfect for metadata associated with the method.

I think discoverability is an issue with pragmas. From Nautilus you can't "see" which methods contain which pragmas. I wonder if one solution for this would be for pramga methods to be grouped under a <virtual-protocol> in the third pane of Nautilus.

[1] https://pharo.fogbugz.com/default.asp?12970

The good old finder has a pragmas search mode which allows for a bit of discoverability (pragma users), but it doesn't link to the code exploiting the pragmas. Do you know if we have something specific on SmartSuggestions for pragmas? Thierry

On Fri, Feb 6, 2015 at 1:58 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

There's a third use that is, I think, the most powerful. Maybe you see this as the same as 1, but I don't. The method is a component to be included in some larger structure, e.g. it is an action method on a menu, or it is an implementation of a pane in an inspector. The pragma is the message to be sent to the object that manipulates that larger structure to add the method to it. This is how menu pragmas work in VW. There is an object called a MenuBuilder. To add a method to a menu (and which menu is described by the pragma) the menu builder sets the method as its current method and then performs the pragma. The parameters in the pragma allow the MenuBuilder to add the method in the right way to the menu. But the execution of the pragma is what actually adds the mehtod to the menu. So its a combination of specification and execution. 2- categorizing things in a decoupled, package-extension compatible way.

I still think there is something in the design that bother me. 1- and 2- are convention-based, so as is most of Smalltalk anyway, so diregarding another approach to 2- as convention-based is a bit dishonest: pragmas get more authority because they look like they have been integrated in the language definition compared to competing solutions.

Yes, it's right to say they have been integrated (but no documented). No, this is just because I start to wonder if methods shouldn't have a bit

more in that "define, add and manipulate" metadata idea, i.e. make it more explicit and start to have tools which represent methods as more like objects and not only text items (methods having metadata, additional slots or instance variables, counters, statistics, ast, traces, break points, etc...). And, in a way, a need to have two levels of execution: one which happens at compile time (i.e. when one accept a method), one which happens at runtime.

I think whether methods are objects or text is in the eye of the beholder.

From my perspective methods /are/ objects. The textual display and compilation of them is only the conventional programmer view of them. Bu from a system perspective methods are objects. Look for example at all the debugger machinery. That is largely concerned with bytecodes in method objects.

For that, I think pragmas are a start, but they are not the complete

answer. And I think we should solve the package-extension compatible thing in 2- to not have 2- (which is just for me a plain categorisation issue) polluting the design by introducing additional/unrelated requirements.

I have no problem with your criticism of pragmas used for categorisation. Method categories are already there and sometimes much more appropriate than pragmas. For example, in the SPur bootstrap the methods to be added or modified in Spur are included in categories of SpurBootstrapPrototypes and its subclasses. If there were pragmas in these methods then the pragmas would be copied into Spur, which is not at all what I want. Instead I copy across the methods in the relevant categories. But this "not a complete solution" feels to me an unfair criticism. Pragmas are what they are and they can do a lot. But no part of the system does everything. A system is composed of components.

2015-02-05 22:47 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Thierry,

A block is far more difficult to deal with than a pragma. A pragma has a simple key, its selector, and simple access to its arameters, the arguments. A block is opaque. Essentially it can only be executable. Pragmas are both executable and function as annotations.

Pragmas are executable, but in a context which is not visible at the place they are written.

...

...

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

...

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods. Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

Yes, but there end up being lots of naming conventions and they are non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

They are still conventions which needs documentation. Pragmas are, in allmost all uses, indirections (i.e. a tag meaning something to a far away object), and the meaning associated to a pragma is, in every case, a roll-your-own solution. And they are clearly "in your face" :)

...

That's not necessarily true. Some pragmas do cause processing at compile time. For example, an FFI signature pragma can be checked at compile-time. But it's in keeping with smalltalk that type checking is not performed at compile time in most cases, isn't it? Why should one require that pragmas be semantically checked at compile time when normal SMalltalk code isn't? At least one knows that the message instance the pragma is compiled to is a valid object and *can* be performed. So pone does know at least that the pragma is executable. Obviously whether that ability to be executed only becomes potent with the right receiver. So any compiled pragma as the potential to be usefully evaluated.

Ok, but its all fuzzy.

It may be executed, but it may be not.

It may send an error if incorrect, but it may fail silently.

It may be executed at compile time, it may be executed sometime later triggered by the compilation, or it may be executed at runtime, as if it was normal code. And of course it may not be executed at all ...

You see what I mean?

...

And often it doesn't matter if the pragma reference a completely non

...

existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

There is lots about Smalltalk that is non-obvious, about programming in general that is non-obvious. I don't see that as a specific criticism of pragmas. Once one knows the idiom it is easy to use; its wide-spread use is evidence of that. And hopefully this conversation will help make it more obvious :-). That it is limited is also not perhaps a useful criticism. The issue is whether it is adequate. I think it is. A literal message goes a long way. I'e not heard of complaints about limitations so far. Do you have specific examples where pragmas are inadequate? The tools /do/ support them. One can do senders and implementors in the browser and see the methods that include them, and the implementors of pragmas. browseAllSelect: will narrow down the search. There is an API for programmatic use (pragmasDo: et al).

See [*][**] for the tools support. For me, pragmas, even in their definition, have serious semantic issues. If they were strictly, as you describe them, a metadata API access executed at compile time, then they would be a lot stronger.

Make them sent at compile-time on the method object, and that would probably solve all the true arguments I have against them :)

for example: <worldMenu> against <tagAndAnnounce: #worldMenu>; <tag: #gtInspectorExtension> instead of <gtInspectorExtension>, etc...

(<tagAndAnnounce:> would gain a bit of performance, as well).

Thierry

[*] Searching a bit for menu pragmas in a Pharo image seems to return a bit of a mess. There is so many of them, all differents, and interestingly, this does not return the code triggered (or using) them, which makes discovery of what the pragma is used for (and its parameters) an 'interesting' problem.

I suggest a task then: an illustrated sequence of how to search for the code triggering world menu addition of a new command, for someone which is new to Pharo ;)

[**] even when used for categorisation, the API is prone to bugs, such as having a method without pragma overriding a superclass method with pragma because they have the same name.

-- best, Eliot

2015-02-06 18:41 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

Or are you talking about the lack of class pragmas? This is a point. But classes already have an executable hook they can use for system modification, their class-side initialize and obsolete methods. That's been used for a long time to address system modification needs on installing and removing classes. So form that perspective there's less of a need to class pragmas. Instead we're focussing on more specialised class definition facilities such as slots, which are intrinsic to classes, i.e. define the class itself, rather than pragmas, which are extrinsic to methods in that they define how methods interact in the system outside themselves.

Maybe what you are saying here holds what I am longing for. Something as simple and as well defined as aClass class>>initialize (simple, solve-it-all mechanism for all initialization needs in a class, full smalltalk based so easily discoverable) or now the class slots, but applied to methods and their metadata. Pragmas are a first step towards that, but I have a feeling we could do it better (and it seems, with some of the changes coming such as traces, in-image optimisations, breakpoints, etc... it may be usefull or overgrow the pragmas). Thierry

On Fri, Feb 6, 2015 at 2:12 PM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

2015-02-06 18:29 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Fri, Feb 6, 2015 at 1:58 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

Hi Eliot,

maybe I'll frame the core question a bit differently, thanks to your explanations.

I see pragmas used mainly for two things:

1- expressing a link to some metadata used by low level tools: compiler, C lang generator, etc... primitives for me are part of that. Kind of neat to see that unified.

There's a third use that is, I think, the most powerful. Maybe you see this as the same as 1, but I don't. The method is a component to be included in some larger structure, e.g. it is an action method on a menu, or it is an implementation of a pane in an inspector. The pragma is the message to be sent to the object that manipulates that larger structure to add the method to it. This is how menu pragmas work in VW. There is an object called a MenuBuilder. To add a method to a menu (and which menu is described by the pragma) the menu builder sets the method as its current method and then performs the pragma. The parameters in the pragma allow the MenuBuilder to add the method in the right way to the menu. But the execution of the pragma is what actually adds the mehtod to the menu. So its a combination of specification and execution.

You mean as a sort of lightweight command object? Pharo uses that for its world menu, Nautilus for its menus, and probably in a few other places. I also like to define command objects for that purpose, because it gives you more flexibility and power while keeping all concerns together: complex commands depending on the context of their creator, having both a menu entry and a shortcut, controlling whether they are enabled or not... I also like command objects because they usually force a bit of a dedicated category for them, and may avoid a spread of pragmas tagged methods left and right in the package.

Yes, a Message instance is a command object, pragmas are Message instances, therefore pragmas are command objects. So using pragmas to associate menu action methods with their menus embeds the relevant command objects in the menu action methods. I have no problem with your criticism of pragmas used for categorisation.

...

Method categories are already there and sometimes much more appropriate than pragmas. For example, in the SPur bootstrap the methods to be added or modified in Spur are included in categories of SpurBootstrapPrototypes and its subclasses. If there were pragmas in these methods then the pragmas would be copied into Spur, which is not at all what I want. Instead I copy across the methods in the relevant categories.

But this "not a complete solution" feels to me an unfair criticism. Pragmas are what they are and they can do a lot. But no part of the system does everything. A system is composed of components.

I agree with that, and the fact that well, maybe, someone will come along with something better :)

Thierry

-- best, Eliot

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Yes, but there end up being lots of naming conventions and they are non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-). Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions. And pragma is a clever tagging.

Stef

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages . About python decorators I disagree that are similar to pragmas. Pragmas are focused on being descriptive , python decorators are descriptive as by product. The main focus of python decorators is to shorten code by introducing syntactic sugar. I agree though this is a very interesting discussion and I dont understand most of the things stated here so I leave an open door and mind for pragmas. Maybe one day I will "get it". On Sat, Feb 7, 2015 at 10:02 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

2015-02-06 22:00 GMT+01:00 stepharo <stepharo@free.fr>:

...

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Thanks. It's a pleasure to discuss that way :)

...

Yes, but there end up being lots of naming conventions and they are

...

non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions.

Pragma as well: just explain the conventions behind the gtInspector pragmas, for example.

But give me multiple protocols and I'll show you the same conventions rewritten in less lines (and a slightly more efficient code).

...

And pragma is a clever tagging.

Then maybe we should remove protocols and replace them with pragmas :)

Thierry

...

Stef

I agree, my objection too is how it looks syntax wise. As I said I don't understand pragmas deeply enough to make a decision if I want them removed or not. Tag wise, well I think a tag should be an IDE and not a language feature. But thats my personal preference. Protocols for me at least is another way to tag things. As I said in the past I would prefer a more elaborate system of tagging for methods and classes similar how Stackoverflow questions work of default and custom tags. On the other hand, people like different things so I never made a big deal out of it. So for now I just tolerate pragmas and they do tolerate me :D On Sat, Feb 7, 2015 at 5:41 PM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path. There are other ways to add metadata to methods. Without tagging. And they don't need to be in the method pane itself. It is like having to specify protocol because there is no list pane to create them.

Hernán

...

About python decorators I disagree that are similar to pragmas. Pragmas are focused on being descriptive , python decorators are descriptive as by product. The main focus of python decorators is to shorten code by introducing syntactic sugar.

I agree though this is a very interesting discussion and I dont understand most of the things stated here so I leave an open door and mind for pragmas. Maybe one day I will "get it".

...

On Sat, Feb 7, 2015 at 10:02 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

2015-02-06 22:00 GMT+01:00 stepharo <stepharo@free.fr>:

...

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Thanks. It's a pleasure to discuss that way :)

...

Yes, but there end up being lots of naming conventions and they are

...

non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions.

Pragma as well: just explain the conventions behind the gtInspector pragmas, for example.

But give me multiple protocols and I'll show you the same conventions rewritten in less lines (and a slightly more efficient code).

...

And pragma is a clever tagging.

Then maybe we should remove protocols and replace them with pragmas :)

Thierry

...

Stef

"Kilon, pragmas are not limited to being descriptive." Ok , I went back and carefully read your posts. I did not noticed that pragmas are executable too, I stand corrected. It appears I underestimated Pragmas, after reading your posts I see where you going with this. Especially the part of adding methods to objects like menu entries definitely makes some sense. The more I think how I would do this, the more I start to appreciate pragmas. For example I would attach some variables to Object class to create a dictionary that will manage meta data for each method. That would move the definition of the method meta data outside the method definition itself but I am not sure if that is such a good thing afterall. I think I will let myself get a few years more experience in Pharo to make up my mind and fully appreciate pragmas.

On Sat, Feb 7, 2015 at 10:21 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

2015-02-07 18:48 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Hernan,

On Sat, Feb 7, 2015 at 7:41 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path.

How exactly is the syntax wrong? The <literalMessagePattern> syntax predates pragmas. It was used first for numeric primitives in Smalltalk-76 or Smalltalk-80. Squeak extended it with primitive:module:. I and others extended it to include arbitrary literal message patterns. The syntax of a literal message pattern is the syntax of a send with no receiver and only literal arguments. The use of this syntax means a) no new syntax b) each pragma is potentially executable c) methods using specific pragmas can be found using "find senders" d) the execution side of a pragma can be found using "find implementors"

So what's wrong with that? How is it wrong to use a common Smalltalk object, a Message, that is well-supported in the system, for method metadata?

The point was: descriptive and the world could be described by tags :) Message syntax for pragmas is just a consequence of the original primitive syntax, nothing else (I.e. natural syntax extension). Given how it is used and that the "potentially executable" is just that, potential, array literals would work just as well (and I'm nearly sure that a todays design for pragmas would give us something like STON content :))

Array literals are not executable. They are literals. They need an interpreter to add semantics. This is one of the reasons why Spec is poor. Pragmas are Message instances. They respond to sendTo:/sentTo:. STON is not executable, it needs an interpreter. There are other ways to add metadata to methods. Without tagging.

...

...

Haven't we discussed that? Putting metadata off to the side introduces a bookkeeping problem. It is a bad idea.

And pragmas only partially address that.

Partially? It completely solves the issue of keeping metadata associated with a method. Imagine that Sista works, that it has dozens of tuning points for a method

(int, no recurse, vect, is a reduce, openCL, thread-safe, hot point, this loop should be enrolled) do you imagine writing all that into the method to guide Sista with pragmas? You'll end up in exactly the same situation as OpenMP 4, where it is possible to write programs with more #pragma lines than C/C++ code.

One doesn't need a line per option. One can use... literal arrays someImportantMethod <sistaTuningHints: #(threadSafe unroll)> but for Sista we imagine very few tuning options, perhaps only one, which is to say to the optimizer optimize this method even if it doesn't show up as a hot spot. Sista is /not/ going to look like gcc.

(And C #pragmas are clearly more powerfull than Smalltalk's, since they are not required to be at the start of the function only)

Sure. Pragmas associate with methods, not statements.

And they don't need to be in the method pane itself.

...

...

For some kinds of metadata, for metadata with semantics, not just a documentary function, it is important for the metadata to be obvious. No one has argued for hiding the primitive specification off-to-the-side. "Need" is a poor criticism here because using turing equivalence lots of things don't "need" to be the way they are. Instead why not ask what are the pros and cons?

...

It is like having to specify protocol because there is no list pane to create them.

I disagree. When the pragma is specifying type information, specifying that the method is an action on a specific menu, is a pane in an inspector, and many other uses, it is essential that that information be represented, and putting it in a one-off bookkeeping system is a bad idea. We're not talking about simple documentary metadata like author, category etc. We're using pragmas for semantics, semantics of the method that are outside of its execution semantics, semantics about how the method fits in the broader system. And putting that in non-obvious places is a bad idea.

Good, you're giving my favorite examples for what pragmas are redundant :)

The method with the menu pragma is named "worldMenuCommand" ...

I don't see examples of this in the pharo 4 image I have to hand. What's the full selector?

The method with the gtInspector pragma is named "gtInspectorPresentation..."

This one is ok. The pragma both says the method is a pane in a glamour inspector, and says what the order is relative to others. One thing that's broken is that the finder doesn't find any senders. Another thing is that the pragma doesn't do anything. If the Glamour inspector added panes to itself by using sentTo: to evaluate the pragma then when you did "implementors" you'd find the builder. So those examples are not taking full advantage. In VW they're more interesting, e.g. searchForStringInSource "Spawn a Browser on all methods which contain a specified string." <menuItem: #(#MethodsWithPhraseDot #menus 'Methods with &Phrase...') nameKey: nil menu: #(#listMenu #browse) position: 21.03 > ... which says which sub menu it is in (listMenu browse), where it is relative to other entries (21.03), what its hot key is (P). And MenuAutomaticGeneraotr implements the pragma (which shows up though the "implementors" menu on the menu pragma method): menuItem: label nameKey: key menu: menuIDs position: position "If the method is to be inserted into our menu, create a MenuItem, otherwise answer nil." | realLabel | menuIDs first = menuName ifFalse: [^nil]. realLabel := self decodeLabel: label. ^Array with: ((MenuItem labeled: realLabel) nameKey: key) with: (menuIDs copyFrom: 2 to: menuIDs size) with: position

Additionally, as I told you before: the pragma semantic isn't in the method... It's somewhere else in the system and senders of that pragma doesn't get it.

I know that very well. If things are done right, that the finder will find pragmas via "senders", and if the designer has chosen to implement the pragma in some processing object, then the link is made. But I disagree that the semantic is entirely elsewhere in the system. The /implementation/ is elsewhere in the system. The semantic is in the pragma, in exactly the same way as a message specifies an operation to be performed and a method specifies an implementation, because... pragmas are messages.

Just have a look at OmniBrowser and tell me that you don't understand where the menu commands are :)

I don't want to criticise OmniBrowser, but if pragmas are used in the pattern above those relationships become clear. Thierry

-- best, Eliot

Hi Eliot, 2015-02-07 14:48 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

Hi Hernan,

On Sat, Feb 7, 2015 at 7:41 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path.

How exactly is the syntax wrong?

I am not saying syntax is wrong, I just don't like it because: 1) Adds another level of representation inside an already reflective system. 2) It could be done with plain common message sends. 3) Should be (to me) "hidden" from method pane, and displayed by a specific tool inside the system browser.

The <literalMessagePattern> syntax predates pragmas. It was used first for numeric primitives in Smalltalk-76 or Smalltalk-80. Squeak extended it with primitive:module:. I and others extended it to include arbitrary literal message patterns. The syntax of a literal message pattern is the syntax of a send with no receiver and only literal arguments.

The use of this syntax means

a) no new syntax b) each pragma is potentially executable c) methods using specific pragmas can be found using "find senders" d) the execution side of a pragma can be found using "find implementors"

So what's wrong with that? How is it wrong to use a common Smalltalk object, a Message, that is well-supported in the system, for method metadata?

I get that pragmas are messages, but they assume an implicit receiver, which actually depends on context. If I understood right with pragmas always receiver = destination? If you change things in the communication model (receiver, sender, message, etc.), by removing or adding elements, you have a loss in the message because it gets harder to reconstruct the message send. Consider <eventRegistration> | eventRegistration | { eventRegistration }. self eventRegistration. which of them "it sounds like" you cannot infer their purpose? To me the first because you need the context MyClass>>methodName and you need to browse the pragma to get its purpose.

...

There are other ways to add metadata to methods. Without tagging.

Haven't we discussed that? Putting metadata off to the side introduces a bookkeeping problem. It is a bad idea.

Yes, but I still have a lot of doubts about feedback with pragmas (I hope you view them as constructive criticism). Is like pragmas assume there is always a perfect message transaction. How can I halt the execution of a specific pragma? Can I write <myPragma halt>? How do I know if it will get executed? Is <shortcut> a functional pragma? How do I distinguis any pragma from being functional or "documentation"?

...

And they don't need to be in the method pane itself.

For some kinds of metadata, for metadata with semantics, not just a documentary function, it is important for the metadata to be obvious.

I think this depends on context. Because your work could be towards low-level machinery details or software reenginering, doesn't mean that we all need to view compiler directives, functional and structural annotations, etc. And if I can write some prediction... When someone invents automatic annotation (like happens now with genomes) and decides it will be cool to have them in the image (because you will gain performance, more reflective capabilitites, etc) methods could explode with pragmas to the point gets to be annoying to browse any method. Then the tool will have to be built by need.

No one has argued for hiding the primitive specification off-to-the-side. "Need" is a poor criticism here because using turing equivalence lots of things don't "need" to be the way they are. Instead why not ask what are the pros and cons?

Ok, many of us knows the pros. I have been using pragmas for a while. But if we stop discussing then it will never adapt to new ideas. So you may want to share your view of cons too :)

...

It is like having to specify protocol because there is no list pane to create them.

I disagree. When the pragma is specifying type information, specifying that the method is an action on a specific menu, is a pane in an inspector, and many other uses, it is essential that that information be represented, and putting it in a one-off bookkeeping system is a bad idea. We're not talking about simple documentary metadata like author, category etc.

Well, I think documentary metadata is tremendously complex.

We're using pragmas for semantics, semantics of the method that are outside of its execution semantics, semantics about how the method fits in the broader system. And putting that in non-obvious places is a bad idea.

I agree. But obvious could be a place where can be located by a method though not necessarily visible. Cheers, Hernán

...

Hernán

...

About python decorators I disagree that are similar to pragmas. Pragmas are focused on being descriptive , python decorators are descriptive as by product. The main focus of python decorators is to shorten code by introducing syntactic sugar.

Kilon, pragmas are not limited to being descriptive.

...

...

I agree though this is a very interesting discussion and I dont understand most of the things stated here so I leave an open door and mind for pragmas. Maybe one day I will "get it".

That would be nice :)

...

...

On Sat, Feb 7, 2015 at 10:02 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

2015-02-06 22:00 GMT+01:00 stepharo <stepharo@free.fr>:

...

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Thanks. It's a pleasure to discuss that way :)

...

Yes, but there end up being lots of naming conventions and they are

...

non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions.

Pragma as well: just explain the conventions behind the gtInspector pragmas, for example.

But give me multiple protocols and I'll show you the same conventions rewritten in less lines (and a slightly more efficient code).

...

And pragma is a clever tagging.

Then maybe we should remove protocols and replace them with pragmas :)

Thierry

...

Stef

-- best, Eliot

Hi Hernan, On Feb 7, 2015, at 12:55 PM, Hernán Morales Durand <hernan.morales@gmail.com> wrote:

Hi Eliot,

2015-02-07 14:48 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Hernan,

On Sat, Feb 7, 2015 at 7:41 AM, Hernán Morales Durand <hernan.morales@gmail.com> wrote:

...

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path.

How exactly is the syntax wrong?

I am not saying syntax is wrong, I just don't like it because:

1) Adds another level of representation inside an already reflective system.

What do you mean exactly? I don't have clear recollection of whether they were in Squeak in 08 or if I ported then, but at that time CompiledMethod had a properties object, and pragmas were merged into this. What I did do was save a lot of space getting rid if the properties object unless a method needed it. And IIRC I changed the implementing class from MethodProperties to AdditionalMethodState. So there was no additional level of representation. a) the syntax was already there fir primitives and b) the state was already there in methods.

2) It could be done with plain common message sends.

No it can't. Common sends are executable and we don't want pragmas executed when the method is run, we want the pragma executed when the method is added, or analysed. That was what was ugly about the use of plain sends in e.f. VMMaker, that there had to be lots of empty implementations of the messages used for metadata. So using normal sends - prevents the metadata from being directly executed because executing them when the method is run is wrong - confuses meta level and normal level, putting them at the same level, and hence... - confusing for the programmer because its not obvious what's metadata and what's a normal send - makes analysis harder for the same reason

3) Should be (to me) "hidden" from method pane, and displayed by a specific tool inside the system browser.

I disagree. I want to see the pragma. It has essential information that shouldn't be hidden. I want to edit it easily. And how can you in the one hand say it can be implemented as a normal message send a d at the same time want it hidden? Be consistent :-)

...

The <literalMessagePattern> syntax predates pragmas. It was used first for numeric primitives in Smalltalk-76 or Smalltalk-80. Squeak extended it with primitive:module:. I and others extended it to include arbitrary literal message patterns. The syntax of a literal message pattern is the syntax of a send with no receiver and only literal arguments. The use of this syntax means a) no new syntax b) each pragma is potentially executable c) methods using specific pragmas can be found using "find senders" d) the execution side of a pragma can be found using "find implementors"

So what's wrong with that? How is it wrong to use a common Smalltalk object, a Message, that is well-supported in the system, for method metadata?

I get that pragmas are messages, but they assume an implicit receiver, which actually depends on context. If I understood right with pragmas always receiver = destination?

Not for pragmas that are just labels. And what's destination? In the menu building example above the destination"is the menu but the receiver is the menu builder. So I think that what the receiver is depends on usage.

If you change things in the communication model (receiver, sender, message, etc.), by removing or adding elements, you have a loss in the message because it gets harder to reconstruct the message send.

I don't understand this. Can you expand?

Consider

<eventRegistration> | eventRegistration | { eventRegistration }. self eventRegistration.

which of them "it sounds like" you cannot infer their purpose?

I don't get the example, because of the confusion of meta layers. <eventRegistration> says "this is metadata" | eventRegistration | says "this is a bug in the method because the variable isn't used, and eg Opal will remove it so it won't be there". { eventRegistration }. I guess you mean { #eventRegistration }. If so this is the same. An optimizing compiler will delete it. self eventRegistration. says "send myself eventRegustration". Which /isn't/ what's wanted. So for me only the pragma implies "this method is to be registered with the event system". The middle two say "I am a bug, or syntax error". The last one says "I am a normal send".

To me the first because you need the context MyClass>>methodName and you need to browse the pragma to get its purpose.

...

...

There are other ways to add metadata to methods. Without tagging.

Haven't we discussed that? Putting metadata off to the side introduces a bookkeeping problem. It is a bad idea.

Yes, but I still have a lot of doubts about feedback with pragmas (I hope you view them as constructive criticism).

I'm finding this whole conversation exhausting.

Is like pragmas assume there is always a perfect message transaction. How can I halt the execution of a specific pragma?

By putting a break point in its implementations.

Can I write <myPragma halt>?

No. But you can out a conditional breakpoint in CompiledMethod>>pragmaAt: or in the pragma's implementation.

How do I know if it will get executed?

By reading the broader system or adding break pints as just described.

Is <shortcut> a functional pragma? How do I distinguis any pragma from being functional or "documentation"?

Using "implementors" and "senders" you can find out where it may be used. One if the problems of late-bound systems is figuring out what binds to what. Pragmas are more difficult to figure out unless pragmas have clearly commented implementations.

...

...

And they don't need to be in the method pane itself.

For some kinds of metadata, for metadata with semantics, not just a documentary function, it is important for the metadata to be obvious.

I think this depends on context. Because your work could be towards low-level machinery details or software reenginering, doesn't mean that we all need to view compiler directives, functional and structural annotations, etc.

That's a mischaracterisation. Most of their uses are for pluggable GUIs. And if this is so onerous why weren't you protesting <primitive: 123>?

And if I can write some prediction... When someone invents automatic annotation (like happens now with genomes) and decides it will be cool to have them in the image (because you will gain performance, more reflective capabilitites, etc) methods could explode with pragmas to the point gets to be annoying to browse any method. Then the tool will have to be built by need.

...

No one has argued for hiding the primitive specification off-to-the-side. "Need" is a poor criticism here because using turing equivalence lots of things don't "need" to be the way they are. Instead why not ask what are the pros and cons?

Ok, many of us knows the pros. I have been using pragmas for a while. But if we stop discussing then it will never adapt to new ideas. So you may want to share your view of cons too :)

The cons are that for years, and at length, one has to defend a good idea ;-). Pragmas were a new idea and here in thus conversation we see how resistant we are to new ideas. I've seen nothing against the need for per-method metadata and a lot on how it can be done with other mechanisms. But the existing mechanisms don't do one of two things - separate normal from meta levels - obviously associate metadata with methods. So what I'm missing in this conversation is addressing the issues: - how in a componentised loadable system do we decouple component loads where components collide in areas like global menu definitions, inspectors, etc? - how do we associate metadata with methods so that - the metadata is of a form that fits well with the host system? - the metadata is clearly associated with the method it is data of? - the bookkeeping is managed? You could add good questions here. But these are the questions I was thinking of when we added pragmas to VW.

...

...

It is like having to specify protocol because there is no list pane to create them.

I disagree. When the pragma is specifying type information, specifying that the method is an action on a specific menu, is a pane in an inspector, and many other uses, it is essential that that information be represented, and putting it in a one-off bookkeeping system is a bad idea. We're not talking about simple documentary metadata like author, category etc.

Well, I think documentary metadata is tremendously complex.

...

We're using pragmas for semantics, semantics of the method that are outside of its execution semantics, semantics about how the method fits in the broader system. And putting that in non-obvious places is a bad idea.

I agree. But obvious could be a place where can be located by a method though not necessarily visible.

Cheers,

Hernán

...

...

Hernán

...

About python decorators I disagree that are similar to pragmas. Pragmas are focused on being descriptive , python decorators are descriptive as by product. The main focus of python decorators is to shorten code by introducing syntactic sugar.

Kilon, pragmas are not limited to being descriptive.

...

...

I agree though this is a very interesting discussion and I dont understand most of the things stated here so I leave an open door and mind for pragmas. Maybe one day I will "get it".

That would be nice :)

...

...

On Sat, Feb 7, 2015 at 10:02 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-06 22:00 GMT+01:00 stepharo <stepharo@free.fr>:

...

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Thanks. It's a pleasure to discuss that way :)

...

...

Yes, but there end up being lots of naming conventions and they are non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-). Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions.

Pragma as well: just explain the conventions behind the gtInspector pragmas, for example.

But give me multiple protocols and I'll show you the same conventions rewritten in less lines (and a slightly more efficient code).

...

And pragma is a clever tagging.

Then maybe we should remove protocols and replace them with pragmas :)

Thierry

...

Stef

-- best, Eliot

Hi Eliot, 2015-02-08 12:50 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

Hi Hernan,

On Feb 7, 2015, at 12:55 PM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

Hi Eliot,

2015-02-07 14:48 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Hernan,

On Sat, Feb 7, 2015 at 7:41 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path.

How exactly is the syntax wrong?

I am not saying syntax is wrong, I just don't like it because:

1) Adds another level of representation inside an already reflective system.

What do you mean exactly? I don't have clear recollection of whether they were in Squeak in 08 or if I ported then, but at that time CompiledMethod had a properties object, and pragmas were merged into this. What I did do was save a lot of space getting rid if the properties object unless a method needed it. And IIRC I changed the implementing class from MethodProperties to AdditionalMethodState. So there was no additional level of representation. a) the syntax was already there fir primitives and b) the state was already there in methods.

The place where pragmas are located internally are not an issue for me. The fact that special tags are not following the message pattern template is. It says there is a special collector, parser, validator, etc. a whole mechanism because something is - supposedly - not easily expressible with plain reflective messages. 2) It could be done with plain common message sends.

No it can't. Common sends are executable and we don't want pragmas executed when the method is run, we want the pragma executed when the method is added, or analysed. That was what was ugly about the use of plain sends in e.f. VMMaker, that there had to be lots of empty implementations of the messages used for metadata.

So using normal sends - prevents the metadata from being directly executed because executing them when the method is run is wrong - confuses meta level and normal level, putting them at the same level, and hence... - confusing for the programmer because its not obvious what's metadata and what's a normal send - makes analysis harder for the same reason

We are talking about different worlds here. Theoretically I could mark methods this way: (Pragma for: (ASTBlockClosure >> #value:) selector: #primitive: arguments: (Array with: 60)) addPragma. So the annotations could be written with normal message sends. And they could be tool supported, so no one gets confused.

3) Should be (to me) "hidden" from method pane, and displayed by a specific tool inside the system browser.

I disagree. I want to see the pragma. It has essential information that shouldn't be hidden. I want to edit it easily.

I want to see and edit the pragma too, when I am really interested. Just like with the bytecodes.

And how can you in the one hand say it can be implemented as a normal message send a d at the same time want it hidden? Be consistent :-)

There is no inconsistency. I am saying they should be hidden from the method body. The method pane you see in the browser should be used to send messages to your modeling domain (which already imposes its own difficulty), not being exploited to add meta information where meta could belong to instrumentation analysis (lint, profiling) or UI building (keymap, menu) or whatever excuse found to add more tags. I wonder how one could refactor this method: Greetings>>helloWorld <year: 2015> <status: #toDo> <author: 'Hernán'> <scope: #private> <soapOperationName: #HelloWorld> <soapDocumentation: #'Returns Hello World'> <soapResult: #String> <soapStyle: #RPC> <rmiOperationName: #HelloWorld> <rmiDocumentation: #'Returns Hello World'> <rmiResult: #String> <asn1OperationName: #HelloWorld> <asn1Documentation: #'Returns Hello World'> <asn1Result: #UTF8String> <dcomOperationName: #HelloWorld> <dcomDocumentation: #'Returns Hello World'> <dcomResult: #String> ^'Hello World'

...

The <literalMessagePattern> syntax predates pragmas. It was used first for numeric primitives in Smalltalk-76 or Smalltalk-80. Squeak extended it with primitive:module:. I and others extended it to include arbitrary literal message patterns. The syntax of a literal message pattern is the syntax of a send with no receiver and only literal arguments.

The use of this syntax means

...

a) no new syntax b) each pragma is potentially executable c) methods using specific pragmas can be found using "find senders" d) the execution side of a pragma can be found using "find implementors"

So what's wrong with that? How is it wrong to use a common Smalltalk object, a Message, that is well-supported in the system, for method metadata?

I get that pragmas are messages, but they assume an implicit receiver, which actually depends on context. If I understood right with pragmas always receiver = destination?

Not for pragmas that are just labels. And what's destination? In the menu building example above the destination"is the menu but the receiver is the menu builder. So I think that what the receiver is depends on usage.

If you change things in the communication model (receiver, sender, message, etc.), by removing or adding elements, you have a loss in the message because it gets harder to reconstruct the message send.

I don't understand this. Can you expand?

Now I realize how far we are from understanding. But I start to feel tired to write about communication models here.

Consider

<eventRegistration> | eventRegistration | { eventRegistration }. self eventRegistration.

which of them "it sounds like" you cannot infer their purpose?

I don't get the example, because of the confusion of meta layers.

Now you understand me :)

<eventRegistration> says "this is metadata"

yes, but not what type of metadata is. Is metadata for instrumentation? is for documentation?

| eventRegistration |

says "this is a bug in the method because the variable isn't used, and eg Opal will remove it so it won't be there".

but you get this is a local variable supposed to be used in the current method. You can get the scope easily.

{ eventRegistration }. I guess you mean { #eventRegistration }. If so this is the same. An optimizing compiler will delete it.

Yes. The idea there is the same, you would associate the eventRegistration as an Array element to be accessed in the current method for example.

self eventRegistration. says "send myself eventRegustration". Which /isn't/ what's wanted.

So for me only the pragma implies "this method is to be registered with the event system". The middle two say "I am a bug, or syntax error". The last one says "I am a normal send".

To me says "this method contains code which defines an event". Where the event is defined should need more metadata until explicit semantics are enough complete.

To me the first because you need the context MyClass>>methodName and you need to browse the pragma to get its purpose.

...

...

There are other ways to add metadata to methods. Without tagging.

Haven't we discussed that? Putting metadata off to the side introduces a bookkeeping problem. It is a bad idea.

Yes, but I still have a lot of doubts about feedback with pragmas (I hope you view them as constructive criticism).

I'm finding this whole conversation exhausting.

Me too :(

Is like pragmas assume there is always a perfect message transaction. How can I halt the execution of a specific pragma?

By putting a break point in its implementations.

Can I write <myPragma halt>?

No. But you can out a conditional breakpoint in CompiledMethod>>pragmaAt: or in the pragma's implementation.

How do I know if it will get executed?

By reading the broader system or adding break pints as just described.

Is <shortcut> a functional pragma? How do I distinguis any pragma from being functional or "documentation"?

Using "implementors" and "senders" you can find out where it may be used. One if the problems of late-bound systems is figuring out what binds to what. Pragmas are more difficult to figure out unless pragmas have clearly commented implementations.

It could be broken because I pressed Alt+M in <gtInspectorAction> and nothing happens: gtInspectorActionBrowse <gtInspectorAction> ^ GLMGenericAction new action: [ self browse ]; icon: GLMUIThemeExtraIcons glamorousBrowse; title: 'Browse'

...

...

And they don't need to be in the method pane itself.

For some kinds of metadata, for metadata with semantics, not just a documentary function, it is important for the metadata to be obvious.

I think this depends on context. Because your work could be towards low-level machinery details or software reenginering, doesn't mean that we all need to view compiler directives, functional and structural annotations, etc.

That's a mischaracterisation. Most of their uses are for pluggable GUIs. And if this is so onerous why weren't you protesting <primitive: 123>?

Pluggable GUIs in systems without GUI builders. But yes, the same applies to the <primitive: 123> artifact. If we would have a Primitives Browser, what would be the point of accessing them through the normal browser?

And if I can write some prediction... When someone invents automatic annotation (like happens now with genomes) and decides it will be cool to have them in the image (because you will gain performance, more reflective capabilitites, etc) methods could explode with pragmas to the point gets to be annoying to browse any method. Then the tool will have to be built by need.

...

No one has argued for hiding the primitive specification off-to-the-side. "Need" is a poor criticism here because using turing equivalence lots of things don't "need" to be the way they are. Instead why not ask what are the pros and cons?

Ok, many of us knows the pros. I have been using pragmas for a while. But if we stop discussing then it will never adapt to new ideas. So you may want to share your view of cons too :)

The cons are that for years, and at length, one has to defend a good idea ;-).

Ok, I am starting to think pragmas with tags are the best thing possibly ever created and the final word of evolution in this area ;) Seriously, I am not saying is a bad idea at all. I say they could be visualized in other ways. I cannot believe people accept happily that < ... > is the best way of representing annotations.

Pragmas were a new idea and here in thus conversation we see how resistant we are to new ideas. I've seen nothing against the need for per-method metadata and a lot on how it can be done with other mechanisms. But the existing mechanisms don't do one of two things - separate normal from meta levels - obviously associate metadata with methods.

I don't have the final answer but I suspect that because we used to see Smalltlak meta-levels through tools, we also "demand" that new meta-objects should cope well with our toolset.

So what I'm missing in this conversation is addressing the issues: - how in a componentised loadable system do we decouple component loads where components collide in areas like global menu definitions, inspectors, etc? - how do we associate metadata with methods so that - the metadata is of a form that fits well with the host system? - the metadata is clearly associated with the method it is data of? - the bookkeeping is managed?

You could add good questions here. But these are the questions I was thinking of when we added pragmas to VW.

I have more questions about having a controlled vocabulary for metadata and annotations, and how we could visualize them better. How could we add meta-annotations and if descriptive declarative programming could be avoided with sequential OO. Naturally they could fit as bad questions for anyone. Cheers, Hernán

It is like having to specify protocol because there is no list pane to

...

...

create them.

I disagree. When the pragma is specifying type information, specifying that the method is an action on a specific menu, is a pane in an inspector, and many other uses, it is essential that that information be represented, and putting it in a one-off bookkeeping system is a bad idea. We're not talking about simple documentary metadata like author, category etc.

Well, I think documentary metadata is tremendously complex.

...

We're using pragmas for semantics, semantics of the method that are outside of its execution semantics, semantics about how the method fits in the broader system. And putting that in non-obvious places is a bad idea.

I agree. But obvious could be a place where can be located by a method though not necessarily visible.

Cheers,

Hernán

...

...

Hernán

...

About python decorators I disagree that are similar to pragmas. Pragmas are focused on being descriptive , python decorators are descriptive as by product. The main focus of python decorators is to shorten code by introducing syntactic sugar.

Kilon, pragmas are not limited to being descriptive.

...

...

I agree though this is a very interesting discussion and I dont understand most of the things stated here so I leave an open door and mind for pragmas. Maybe one day I will "get it".

That would be nice :)

...

...

On Sat, Feb 7, 2015 at 10:02 AM, Thierry Goubier < thierry.goubier@gmail.com> wrote:

...

2015-02-06 22:00 GMT+01:00 stepharo <stepharo@free.fr>:

...

Really interesting discussion. I like pragmas but this is interesting to see them challenged.

Thanks. It's a pleasure to discuss that way :)

...

Yes, but there end up being lots of naming conventions and they are

...

non-obvious. Whereas pragmas, because they are in-your-face in the methods in question, don't need conventions. They just need documenting ;-).

Thierry I'm skeptical that multiple protocol will save the problem because you will rely on coding conventions.

Pragma as well: just explain the conventions behind the gtInspector pragmas, for example.

But give me multiple protocols and I'll show you the same conventions rewritten in less lines (and a slightly more efficient code).

...

And pragma is a clever tagging.

Then maybe we should remove protocols and replace them with pragmas :)

Thierry

...

Stef

-- best, Eliot

2015-02-08 19:46 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

On Sun, Feb 8, 2015 at 1:26 PM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

Hi Eliot,

2015-02-08 12:50 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Hernan,

On Feb 7, 2015, at 12:55 PM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

Hi Eliot,

2015-02-07 14:48 GMT-03:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

Hi Hernan,

On Sat, Feb 7, 2015 at 7:41 AM, Hernán Morales Durand < hernan.morales@gmail.com> wrote:

...

2015-02-07 5:59 GMT-03:00 kilon alios <kilon.alios@gmail.com>:

...

Personally I don't like Pragmas, they have not convinced me so far that they fit the style of Smalltalk syntax. I have to confess though I never liked descriptive elements and languages .

Me neither. Actually the pragma idea is not wrong per se, it is the tag syntax to write them which bothers me. Because the world can be described with tags if you start that path.

How exactly is the syntax wrong?

I am not saying syntax is wrong, I just don't like it because:

1) Adds another level of representation inside an already reflective system.

What do you mean exactly? I don't have clear recollection of whether they were in Squeak in 08 or if I ported then, but at that time CompiledMethod had a properties object, and pragmas were merged into this. What I did do was save a lot of space getting rid if the properties object unless a method needed it. And IIRC I changed the implementing class from MethodProperties to AdditionalMethodState. So there was no additional level of representation. a) the syntax was already there fir primitives and b) the state was already there in methods.

The place where pragmas are located internally are not an issue for me. The fact that special tags are not following the message pattern template is. It says there is a special collector, parser, validator, etc. a whole mechanism because something is - supposedly - not easily expressible with plain reflective messages.

I don't understand. Pragmas are literal message patterns, period. There is an exception for the primitive pragma so that the variable name for a primitive's error code can be introduced, but that's part of primitive parsing, not general pragma parsing. At least in Squeak here's the code:

Parser>>pragmaSequence "Parse a sequence of method pragmas." [ (self matchToken: #<) ifFalse: [ ^ self ]. self pragmaStatement. (self matchToken: #>) ifFalse: [ ^ self expected: '>' ] ] repeat

pragmaStatement "Read a single pragma statement. Parse all generic pragmas in the form of: <key1: val1 key2: val2 ...> and remember them, including primitives." | selector arguments words index keyword | (hereType = #keyword or: [ hereType = #word or: [ hereType = #binary ] ]) ifFalse: [ ^ self expected: 'pragma declaration' ].

" This is a ugly hack into the compiler of the FFI package. FFI should be changed to use propre pragmas that can be parsed with the code here. " (here = #apicall: or: [ here = #cdecl: ]) ifTrue: [ ^ self externalFunctionDeclaration ].

selector := String new. arguments := OrderedCollection new. words := OrderedCollection new. [ hereType = #keyword or: [ (hereType = #word or: [ hereType = #binary ]) and: [ selector isEmpty ] ] ] whileTrue: [ index := self startOfNextToken + requestorOffset. selector := selector , self advance. words add: (index to: self endOfLastToken + requestorOffset). (selector last = $: or: [ selector first isLetter not ]) ifTrue: [ arguments add: (self pragmaLiteral: selector) ] ]. selector numArgs ~= arguments size ifTrue: [ ^ self expected: 'pragma argument' ]. (Symbol hasInterned: selector ifTrue: [ :value | keyword := value]) ifFalse: [ keyword := self correctSelector: selector wordIntervals: words exprInterval: (words first first to: words last last) ifAbort: [ ^ self fail ] ]. self addPragma: (Pragma keyword: keyword arguments: arguments asArray). ^ true

I can't speak for how its been implemented in Opal but I'd be surprised if it is using separate machinery. The Squeak code above is different from the normal message parsing precisely so that it can insist that arguments are only literals (pragmaLiteral: above).

Ok, I will stop here because I see contradictions and no way we can agree on them. Amazing code BTW.

...

2) It could be done with plain common message sends.

...

No it can't. Common sends are executable and we don't want pragmas executed when the method is run, we want the pragma executed when the method is added, or analysed. That was what was ugly about the use of plain sends in e.f. VMMaker, that there had to be lots of empty implementations of the messages used for metadata.

...

So using normal sends - prevents the metadata from being directly executed because executing them when the method is run is wrong - confuses meta level and normal level, putting them at the same level, and hence... - confusing for the programmer because its not obvious what's metadata and what's a normal send - makes analysis harder for the same reason

We are talking about different worlds here.

So let;s stop talking because I;m getting frustrated. I make a valid point about the inability to use normal selectors and you ignore my point.

...

Theoretically I could mark methods this way:

(Pragma for: (ASTBlockClosure >> #value:) selector: #primitive: arguments: (Array with: 60)) addPragma.

...

So the annotations could be written with normal message sends. And they could be tool supported, so no one gets confused.

So this is embedded in a method? So it gets evaluated every time the method is executed? Surely you're not serious.

Precisely that is what should happens behind the curtains. I cannot believe why is so hard to understand.

...

...

3) Should be (to me) "hidden" from method pane, and displayed by a specific tool inside the system browser.

I disagree. I want to see the pragma. It has essential information that shouldn't be hidden. I want to edit it easily.

I want to see and edit the pragma too, when I am really interested. Just like with the bytecodes.

...

And how can you in the one hand say it can be implemented as a normal message send a d at the same time want it hidden? Be consistent :-)

There is no inconsistency. I am saying they should be hidden from the method body.

The method pane you see in the browser should be used to send messages to your modeling domain (which already imposes its own difficulty), not being exploited to add meta information where meta could belong to instrumentation analysis (lint, profiling) or UI building (keymap, menu) or whatever excuse found to add more tags.

I wonder how one could refactor this method:

Greetings>>helloWorld <year: 2015> <status: #toDo> <author: 'Hernán'> <scope: #private>

<soapOperationName: #HelloWorld> <soapDocumentation: #'Returns Hello World'> <soapResult: #String> <soapStyle: #RPC>

<rmiOperationName: #HelloWorld> <rmiDocumentation: #'Returns Hello World'> <rmiResult: #String>

<asn1OperationName: #HelloWorld> <asn1Documentation: #'Returns Hello World'> <asn1Result: #UTF8String>

<dcomOperationName: #HelloWorld> <dcomDocumentation: #'Returns Hello World'> <dcomResult: #String>

^'Hello World'

Thus:

first, date & author can be recovered from the method source via timestamp, but if you want to separate the author of the text from the last editor of the method then...

Greetings>>helloWorld <addDocPage: #Greetings year: 2015 status: #toDo author: 'Hernán' scope: #private> <soapOperation: #HelloWorld documentation: #'Returns Hello World' resultType: #String style: #RPC> <rmiOperation: #HelloWorld documentation: #'Returns Hello World' resultType: #String> <asn1OperationOperation: #HelloWorld documentation: #'Returns Hello World' resultType:#UTF8String> <dcomOperation: #HelloWorld documentation: #'Returns Hello World' resultType: #String>

^'Hello World'

and then have implementations of e.g. SoapInterfaceBuilder>>soapOperation:documentation:returnType:style:, ASN1InterfaceBuilder>> asn1Operation:documentation:returnType:style:, etc, that add the method to the interface they're building.

So, assuming your interface methods are in a class hierarchy from SomeSoapInterfaceSubclass up to SomeSoapInterfaceBaseClass building looks like, e.g.:

| builder | builder := SoapInterfaceBuilder new. (Pragma allNamed: #soapOperation:documentation:returnType:style: from: SomeSoapInterfaceSubclass to: SomeSoapInterfaceBaseClass) do: [:pragma| builder method: pragma method. pragma message sendTo: builder]

But if you're able you'll do it just like this:

Greetings>>helloWorld <addDocPage: #Greetings year: 2015 status: #toDo author: 'Hernán' scope: #private> <externalOperation: #HelloWorld documentation: #'Returns Hello World' resultType: #UTF8String style: #RPC>

^'Hello World'

implement the same method in the various builders, and coerce UTF8String to String in those that only handle simple notions of string type.

Why on _earth_ are people writing 17 pragmas where I see only two?

Not the point I wanted to address, but thanks for the example and answering the questions patiently Cheers, Hernán

On 5 February 2015 at 21:20, Thierry Goubier <thierry.goubier@gmail.com> wrote:

2015-02-05 21:28 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-05 18:51 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

It is obviously a compromise (or a continuum) between abstractions and performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

...

But there should remain a focus on efficiency (not just speed but also memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax). They have been broadly used. IME they have simplified and reduced code where ever they have been used. They don't have to be there but they're a good thing. Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

OK, the claim is too strong. But they /have/ simplified code in cases where they're appropriate. And I can cite several examples.

...

One of the uses of pragmas is associating methods containing Gui commands or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

But triggering in the background happens for maintaining change sets, notifying other clients too. It's not as if pragmas introduced such triggering; that kind of triggering has been in use for a long time. And being able to reshape the GUI automatically is very useful.

I don't contest the possibilities, it's just that they add a significant layer of complexity when non mastered (how many Pharo developpers know which event you have to register to to receive all new methods notifications? Is that documented in one of the books?), and that, except for using them as <primitives> or for extensibility, I see other syntaxes and smalltalk code which are simpler.

A good example is that the pragma syntax is never included in the one page Smalltalk syntax description :)

...

...

...

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of script hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Why? A good use of pragmas is to associate meta data with a particular method. Having calls off to the side always introduces the need for book-keeping to keep those methods off to the side in sync with the methods they're describing. Typically everyone rolls their own. But here we're adding a level of triggering just to keep the metadata methods in sync.

I agree with the "metadata", but I'd prefer a executable, evaluate that block as a medata literal than the pragma. Something that says "onceAndStoreAsMetadata", to a block, for example. An API to compiled methods which says add metadata.

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

...

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods. Instead we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

...

This is one idiom that covers a host of other cases. That's why I claim that whenever I've seen it used it has reduced complexity.

Some history. Steve Dahl, I developed pragmas at ParcPlace, with Vassili Bykov adding abstractions for accessing them. The first step was to replace some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by a pragma the compiler would recognise and set the bits itself. The first real use was to make the VisualWorks launcher's menus extensible. Before pragmas the launcher's menu was static and had lots of disabled entries for launching tools that were sold separately such as DLLAndCConnect. With pragmas the launcher's menu was defined with the base system's tools and then extended as each tool package was loaded, or cut-back as each tool was unloaded. So that decoupled the launcher from introducing new tools. A nice result.

We then started using it for the browser and one could plug-in a single tool without redefining the browser's menu methods, which decoupled each extension. All this was done in the context of the parcel system, where we could rapidly load packages (parcels ~= Fuel). Pragmas allowed us to decouple these tools where they collided in places like menu definition, tool registration.

Then Tami Lee, who was managing the COM connection that turned a VW image into a COM server, became the first "user" of pragmas outside of myself and Steve. She used it to replace a lot of class-side methods that defined the signatures of methods that comprised the server. It was a lovely clean-up. One could define the COM signature for a method in the method itself, and the class side lost about three separate methods that defined all that metadata. One could read the server method itself and understand its semantics without having to consult the class-side methods. One didn't have to know that there was metadata hidden on the class side because it was right there in your face.

Then Vassili used it for his cool inspector framework, Trippy, which was similar to Glamour in some ways, and was a huge improvement over the old Inspector framework, again resulting in a much more pluggable, decoupled and extensible system. Vassili also added the abstractions for accessing pragmas in methods.

Then we added checking so that one could restrict the compiler to accept only legal pragmas for a given class. But if we defined the legal pragmas in a class-side method, say legalPragmas, then this would be exactly the kind of single point for extensions that causes collisions between packages, each of which might want to add its own set of pragmas. The solution... use a pragma to mark a class-side method as defining a set of legal pragmas for a class. One could have more than one method defining a set of legal pragmas; packages wishing to add their own cool pragmas were decoupled. Once the system because recursive, it had to be a good idea ;-).

Ok, I start to see where the abstraction wasn't working so well... since pragmas are not executed, when writing a method you can't know if the pragma is correct, because even executing the method may not trigger the pragma induced code. So you need the legalPragmas to give metadata on metadata for the compiler to do a bit of static checking, but it doesn't work for system-wide pragmas unless you extend Object :(

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

Pragmas don't execute. They're _data_. There is no "calls unknown sender" because the don't execute. Eliot's point is that pragmas _describe_, and then other systems act on those descriptions. They're just like Java or C# attributes, or Python decorators. Only they're better, because Java/C# attributes can do anything, whereas pragmas merely describe. frank

...

There are other uses; you've seen them. I used them in VMMaker to eliminate metadata that was embedded as sends to methods defined as ^self that Slang had to extract and analyse, and filter-out from generated code. They simplified Slang's code anaylsis, made the simulator more efficient (since there were no longer sends to execute). My point is that in all the cases I've seen, using pragmas has - simplified the code - made it obvious that methods have metadata associated with them - replaced specialized ways of associating metadata with code by the general pragma mechanism and in many of the cases it has - provided a more decoupled system - provided a more dynamic and extensible system

Yes, and I can point out some of its shortcomings: it's non-obvious, it's limited, tools, even that many years later don't support them well (in Squeak or Pharo, at least), its redundant in quite a few variants.

Please, could we improve a bit? Methods belonging to multiple protocols would give us the same decoupling as pragmas, and I would be free to avoid them where I shouldn't have to use them :)

...

I've been meaning to write up the history of pragmas for ages, but Vassili, Steve or I have always been too busy. I think a community paper on their use and history would be worth-while, and might go a long way to reduce antipathies like yours. I will forever be in debt to anyone who wants to volunteer to help me write such a paper.

That would certainly be interesting :)

...

...

Thierry

(Look. I started using Smalltalk in 1992... and up to your description, I wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

They /have/ to follow literal message syntax. t's all the compiler will accept. That's why there needs to be a paper.

Yes!

Thanks for taking the time to argument,

Thierry

...

-- best, Eliot

Hi Frank, On Fri, Feb 6, 2015 at 2:29 AM, Frank Shearar <frank.shearar@gmail.com> wrote:

On 5 February 2015 at 21:20, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-05 21:28 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Thu, Feb 5, 2015 at 11:34 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-05 18:51 GMT+01:00 Eliot Miranda <eliot.miranda@gmail.com>:

...

On Thu, Feb 5, 2015 at 2:31 AM, Thierry Goubier <thierry.goubier@gmail.com> wrote:

...

2015-02-05 10:55 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

It is obviously a compromise (or a continuum) between abstractions

and

...

...

...

...

...

...

performance.

I agree. With a special view in that we are in a sub domain where simple things well designed (Smalltalk, that is) are amazingly good at supporting complex designs.

...

But there should remain a focus on efficiency (not just speed but

also

...

memory), it is hard to fix these things years later.

And I like the fact that efficient code and design is often a pleasure to read and learn from :)

Now, being radical: could we get rid of pragmas ? The only reason I see to them is that they allow extension by external packages, because we can't have methods which belong to two protocols (*).

They are a Smalltalk-centric way of adding arbitrary metadata to methods; Smalltalk-centric in that a pragma is a Message instance, may be queried for senders, performed, etc, and that it can be parsed using the standard compiler (they add no new syntax). They have been broadly used. IME they have simplified and reduced code where ever they have been used. They don't have to be there but they're a good thing. Why do you want to get rid of them?

Because the "they have simplified and reduced code where ever they have been used" is wrong. I just have to give you a counter example:

OK, the claim is too strong. But they /have/ simplified code in cases where they're appropriate. And I can cite several examples.

...

One of the uses of pragmas is associating methods containing Gui

commands

...

or settings to specific objects. Based on an object inspected or selected, you search among all its methods the ones containing a specific pragma (and you order them by a parameter to that pragma, if you want), and you execute that method to retrieve the objects you want (presentations, menu commands, shortcuts, you name it, I use it :)).

The code to do that is exactly as long as the one which, on the same object, retrieve all methods under a certain protocol (the latter being faster than the pragma one, to boot).

Each method is one line longer ("the pragma").

Each such method usually has in its name a copy of the pragma (gtInspectorXXX methods, I'm looking at you), because of course this is far more user friendly to indicate its purpose in the method name than in only the pragma.

(There are two more arguments for the use of pragmas in that context, one which has a direct counter-example, one which hasn't: )

Moreover, the semantic of pragmas is "interesting" to describe, and in some cases, require a good amount of dark magic about a global object listening to all methods changes and capturing (and executing) certain methods in a vague relation about when this is going to happen, or being triggered on specific system events (main menu rebuilding, anyone?). The funny thing is to see that pattern visible on a profile when loading packages (talk of a scalable approach).

But triggering in the background happens for maintaining change sets, notifying other clients too. It's not as if pragmas introduced such triggering; that kind of triggering has been in use for a long time. And being able to reshape the GUI automatically is very useful.

I don't contest the possibilities, it's just that they add a significant layer of complexity when non mastered (how many Pharo developpers know which event you have to register to to receive all new methods notifications? Is that documented in one of the books?), and that, except for using them as <primitives> or for extensibility, I see other syntaxes and smalltalk code which are simpler.

A good example is that the pragma syntax is never included in the one page Smalltalk syntax description :)

...

...

...

(and yes, I'm biassed)

Then you're the right person to give me counter arguments...

(Now, I'd look differently at pragmas used for gradual typing and so on... But even for something like FFI, I'd seriously prefer to have Smalltalk calls to describe the call and its arguments than a kind of

script

...

...

hidden inside pragmas, just for the discoverability and because it makes one less idiom to deal with)

Why? A good use of pragmas is to associate meta data with a particular method. Having calls off to the side always introduces the need for book-keeping to keep those methods off to the side in sync with the methods they're describing. Typically everyone rolls their own. But here we're adding a level of triggering just to keep the metadata methods in sync.

I agree with the "metadata", but I'd prefer a executable, evaluate that block as a medata literal than the pragma. Something that says "onceAndStoreAsMetadata", to a block, for example. An API to compiled methods which says add metadata.

I strongly agree with your keep it in sync argument, still. Pragmas are better than nothing.

...

There is no such need with pragmas; they are always in sync with the methods they describe because they are embedded in their methods.

Instead

...

we can use triggering to do useful things, adding a pane to open inspectors as soon as we define the method that describes the pane, adding or removing a menu entry, etc.

Just a naming convention does just that perfectly fine, and with less lines (except for extensions by external packages) and faster code in many cases.

...

This is one idiom that covers a host of other cases. That's why I claim that whenever I've seen it used it has reduced complexity.

Some history. Steve Dahl, I developed pragmas at ParcPlace, with

Vassili

...

Bykov adding abstractions for accessing them. The first step was to replace some ugly class-side code to set unwind bits in ensure: and ifCurtailed: by a pragma the compiler would recognise and set the bits itself. The first real use was to make the VisualWorks launcher's menus extensible. Before pragmas the launcher's menu was static and had lots of disabled entries for launching tools that were sold separately such as DLLAndCConnect. With pragmas the launcher's menu was defined with the base system's tools and then extended as each tool package was loaded, or cut-back as each tool was unloaded. So that decoupled the launcher from introducing new tools. A nice result.

We then started using it for the browser and one could plug-in a single tool without redefining the browser's menu methods, which decoupled each extension. All this was done in the context of the parcel system, where we could rapidly load packages (parcels ~= Fuel). Pragmas allowed us to decouple these tools where they collided in places like menu definition, tool registration.

Then Tami Lee, who was managing the COM connection that turned a VW image into a COM server, became the first "user" of pragmas outside of myself and Steve. She used it to replace a lot of class-side methods that defined the signatures of methods that comprised the server. It was a lovely clean-up. One could define the COM signature for a method in the method itself, and the class side lost about three separate methods that defined all that metadata. One could read the server method itself and understand its semantics without having to consult the class-side methods. One didn't have to know that there was metadata hidden on the class side because it was right there in your face.

Then Vassili used it for his cool inspector framework, Trippy, which was similar to Glamour in some ways, and was a huge improvement over the old Inspector framework, again resulting in a much more pluggable, decoupled and extensible system. Vassili also added the abstractions for accessing pragmas in methods.

Then we added checking so that one could restrict the compiler to accept only legal pragmas for a given class. But if we defined the legal pragmas in a class-side method, say legalPragmas, then this would be exactly the kind of single point for extensions that causes collisions between packages, each of which might want to add its own set of pragmas. The solution... use a pragma to mark a class-side method as defining a set of legal pragmas for a class. One could have more than one method defining a set of legal pragmas; packages wishing to add their own cool pragmas were decoupled. Once the system because recursive, it had to be a good idea ;-).

Ok, I start to see where the abstraction wasn't working so well... since pragmas are not executed, when writing a method you can't know if the pragma is correct, because even executing the method may not trigger the pragma induced code. So you need the legalPragmas to give metadata on metadata for the compiler to do a bit of static checking, but it doesn't work for system-wide pragmas unless you extend Object :(

And often it doesn't matter if the pragma reference a completely non existent method or api, since it is probably never executed by anybody (and if it is, it won't probably reify the error message properly as a compilation error as it should, because it may be triggered miles away from the system browser).

Pragmas don't execute. They're _data_. There is no "calls unknown sender" because the don't execute.

I think they can do both, which is nice. I actually prefer uses of pragmas where they /are/ executable. They ca be performed by some object, and that execution can modify the system in the desired way, for example having a menu builder perform the menu definition pragma in a menu action method to add that action to a menu.

Eliot's point is that pragmas _describe_, and then other systems act on those descriptions.

Right. But that description is executable by something else. Unlike, for example, a block as metadata which is only executable.

They're just like Java or C# attributes, or Python decorators. Only they're better, because Java/C# attributes can do anything, whereas pragmas merely describe.

Are you sure Java and C# attributes are executable? When I read the wikipedia page for Java attributes I get the notion that they're structured non-executable data, i.e.: "When Java source code is compiled, annotations can be processed by compiler plug-ins called annotation processors. Processors can produce informational messages or create additional Java source files or resources, which in turn may be compiled and processed, and also modify the annotated code itself. The Java compiler conditionally stores annotation metadata in the class files, if the annotation has aRetentionPolicy of CLASS or RUNTIME. Later, the JVM or other programs can look for the metadata to determine how to interact with the program elements or change their behavior. In addition to processing an annotation using an annotation processor, a Java programmer can write their own code that uses reflections to process the annotation. Java SE 5 supports a new interface that is defined in the java.lang.reflect package. This package contains the interface called AnnotatedElement that is implemented by the Java reflection classes including Class, Constructor, Field,Method, and Package. The implementations of this interface are used to represent an annotated element of the program currently running in the Java Virtual Machine. This interface allows annotations to be read reflectively." The microsoft doc on C# indicates that they're declarative, but no mention of those declarations being executable: "C# enables programmers to invent new kinds of declarative information, called attributes. Programmers can then attach attributes to various program entities, and retrieve attribute information in a run-time environment. For instance, a framework might define a HelpAttribute attribute that can be placed on certain program elements (such as classes and methods) to provide a mapping from those program elements to their documentation."

frank

...

...

There are other uses; you've seen them. I used them in VMMaker to eliminate metadata that was embedded as sends to methods defined as ^self that Slang had to extract and analyse, and filter-out from generated code. They simplified Slang's code anaylsis, made the simulator more efficient (since there were no longer sends to execute). My point is that in all the cases I've seen, using pragmas has - simplified the code - made it obvious that methods have metadata associated with them - replaced specialized ways of associating metadata with code by the general pragma mechanism and in many of the cases it has - provided a more decoupled system - provided a more dynamic and extensible system

Yes, and I can point out some of its shortcomings: it's non-obvious, it's limited, tools, even that many years later don't support them well (in Squeak or Pharo, at least), its redundant in quite a few variants.

Please, could we improve a bit? Methods belonging to multiple protocols would give us the same decoupling as pragmas, and I would be free to avoid them where I shouldn't have to use them :)

...

I've been meaning to write up the history of pragmas for ages, but Vassili, Steve or I have always been too busy. I think a community

paper on

...

their use and history would be worth-while, and might go a long way to reduce antipathies like yours. I will forever be in debt to anyone who wants to volunteer to help me write such a paper.

That would certainly be interesting :)

...

...

Thierry

(Look. I started using Smalltalk in 1992... and up to your

description, I

...

...

wasn't aware pragmas were supposed to follow message syntax ;) Thanks for the explanation, by the way)

They /have/ to follow literal message syntax. t's all the compiler will accept. That's why there needs to be a paper.

Yes!

Thanks for taking the time to argument,

Thierry

...

-- best, Eliot

-- best, Eliot

On 05 Feb 2015, at 10:12, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 04 Feb 2015, at 22:04, Levente Uzonyi <leves@elte.hu> wrote:

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine?

I have not yet benchmarked it… PetitParser as it is is too slow, but we will soon have a faster version (factor 10).

We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. (there is a setting 'AST based coloring’ in Pharo3 and Pharo4, but it is turned off by default).

One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined by the AST nodes and structural navigation in the editor.

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year…

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical.

Seemingly I still can not explain what I mean in away that people get it, so please just ignore this mail. Marcus

Am 07.02.2015 um 16:18 schrieb David T. Lewis <lewis@mail.msen.com>:

On Thu, Feb 05, 2015 at 01:54:51PM +0100, Marcus Denker wrote:

...

...

On 05 Feb 2015, at 10:12, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year???

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste??? while I would say that we have already *now* the resources to be even more radical.

Seemingly I still can not explain what I mean in away that people get it, so please just ignore this mail.

Marcus

Your point makes good sense to me. In 1978, a system in which a low-level integer was represented as an object with behavior would have been perceived as a rediculously wasteful idea. And can you imagine someone seriously suggesting something so wasteful as automatic memory management?

So if the "same" system was being designed today, it might very well include new concepts that today are perceived as wasteful. Some of those concepts might turn out to be very good thing once we get used to them. The times have changed. Today waste seems to be a requirement. Whatever application you have - reimplement it using „web technologies“. Whatever data you have - store it in „the cloud“ and tunnel its transportation over port 80, gain extra points for using XML here.

Today, beautiful small things like Smalltalk are ignored by the masses and being laughed at. Andreas

On Thu, 5 Feb 2015, Marcus Denker wrote:

...

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 04 Feb 2015, at 22:04, Levente Uzonyi <leves@elte.hu> wrote:

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine?

I have not yet benchmarked it… PetitParser as it is is too slow, but we will soon have a faster version (factor 10).

We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. (there is a setting 'AST based coloring’ in Pharo3 and Pharo4, but it is turned off by default).

One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined by the AST nodes and structural navigation in the editor.

Rebuilding the whole AST after every keystroke is possible, but keeping it real-time is a bit challenging. I would love to see an editor, which works on the AST directly - aka it maps the source code to AST nodes, and just updates the smallest possible subtree at each keystroke. Implementing such editor has it's challenges ofc, like typing a single character can invalidate the whole code, but the editor should keep the AST somehow in that case too.

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year…

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical.

I think this is a different thing. Improvements are always welcome, as long as they don't step on your toes. About Slots: I don't see their advantages yet (other than bitfields, but those are so rare in Smalltalk that I implement them causes no trouble. And they are just an optimization over using multiple fields.). Levente

Marcus

On 5 February 2015 at 15:35, Levente Uzonyi <leves@elte.hu> wrote:

On Thu, 5 Feb 2015, Marcus Denker wrote:

...

...

On 05 Feb 2015, at 10:04, Marcus Denker <marcus.denker@inria.fr> wrote:

...

On 04 Feb 2015, at 22:04, Levente Uzonyi <leves@elte.hu> wrote:

A single parser is a nice goal, but performance is top priority for Shout, because it should do it's job real-time. When it starts lagging behind, then people just turn it off, because it doesn't help them. Can those parsers (SHRBTextStyler and a Smalltalk parser written using PetitParser) parse an average method in less than 20ms on an average machine?

I have not yet benchmarked it… PetitParser as it is is too slow, but we will soon have a faster version (factor 10).

We should do some benchmarks. For using, it seems ok. With a fast machine + JIT, which does not say much of course. (there is a setting 'AST based coloring’ in Pharo3 and Pharo4, but it is turned off by default).

One thing that is nice with the AST is that it can be used for other things, too. e.g. in Pharo we have a menu that is defined by the AST nodes and structural navigation in the editor.

Rebuilding the whole AST after every keystroke is possible, but keeping it real-time is a bit challenging.

I would love to see an editor, which works on the AST directly - aka it maps the source code to AST nodes, and just updates the smallest possible subtree at each keystroke. Implementing such editor has it's challenges ofc, like typing a single character can invalidate the whole code, but the editor should keep the AST somehow in that case too.

C#'s Roslyn does this: http://blogs.msdn.com/b/ericlippert/archive/2012/06/08/persistence-facades-a... frank

...

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year…

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical.

I think this is a different thing. Improvements are always welcome, as long as they don't step on your toes.

About Slots: I don't see their advantages yet (other than bitfields, but those are so rare in Smalltalk that I implement them causes no trouble. And they are just an optimization over using multiple fields.).

Levente

...

Marcus

I would love to see an editor, which works on the AST directly - aka it maps the source code to AST nodes, and just updates the smallest possible subtree at each keystroke. Implementing such editor has it's challenges ofc, like typing a single character can invalidate the whole code, but the editor should keep the AST somehow in that case too.

Me too and I want pretty print as you type :) A student will probably want to start that adventure so we will have fun.

...

Another way to see it: How would the original Smalltalk be designed if they would have had 4GB RAM in 1978?

What fascinates me still is that Smalltalk used the existing resources (even building their own machines) to an extreme, while today we are obsessed to find reasons why we can not do anything that makes the system slower or use more memory than yesterday. And that even with resources growing every year…

This is why we e.g. now have a meta object describing every instance variable in Pharo. I am sure there are people who will see these ~7000 objects as pure waste… while I would say that we have already *now* the resources to be even more radical.

I think this is a different thing. Improvements are always welcome, as long as they don't step on your toes.

About Slots: I don't see their advantages yet (other than bitfields, but those are so rare in Smalltalk that I implement them causes no trouble. And they are just an optimization over using multiple fields.).

Levente

...

Marcus