November 2009 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] (no subject)

by Lukas Renggli

2009/11/22 Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com>: > Without looking at RB code, this is how I would handle it: > 1) Reify Smalltalk tokens, and create a SpaceToken (can include > various separators character and comments) > Â the tokens just include a copy of corresponding source text. As far as I know this is what Gwenael did. > 2) Scanner creates a linked list of tokens including SpaceTokens This approach works very well for instance variable or class renames, because a single identifier token is replace with a different one. Most refactorings however replace complex node trees being generated from code, coming from a rewrite expression or another method. This typically involves several nested replacements, inserting/removing parenthesis, inserting/removing statement separators, wrapping/unwrapping into blocks, changing indention, etc. Furthermore the code needs to decide if the space/comments between two adjacent nodes belong to the node before or after the tokens. > 3) Parser parses the stream of tokens and build an AST that retain > pointers to the links (firstLink and lastLink) > Â The advantage of storing links rather than position is that we can > add/remove links easily without recomputing positions Yes, but at the same time the engine has to ensure that all the transformations are performed correctly. As soon as you have to keep the two data-structures (the token-list and the AST) in sync, this becomes highly non-trivial for anything more complex than a simple rename. > 4) refactoring operates on AST and replaces links content, > Â eventually if it add or remove links, then some special care of > intersticial SpaceToken has to happen. > 5) printing refactored source code is printing the tokens verbatim > (that is just nextPutAll: their contents) The current code does a much more conservative approach, that is very simple and ensures correctness at each step. Whenever a node is replaced, at the same time change-objects on the original source string are created. The new source code is then validated against the transformed AST nodes to ensure no bugs were introduced. If there is a problem, pretty printing kicks in. Lukas -- Lukas Renggli http://www.lukas-renggli.ch

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Schwab,Wilhelm K

If anyone builds this, I can easily clone one of my images and give it torture test with comments littering method bodies. If it can keep the "data collector" code in tact, it works. Bill -----Original Message----- From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Nicolas Cellier Sent: Sunday, November 22, 2009 9:56 AM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] (no subject) Without looking at RB code, this is how I would handle it: 1) Reify Smalltalk tokens, and create a SpaceToken (can include various separators character and comments) the tokens just include a copy of corresponding source text. 2) Scanner creates a linked list of tokens including SpaceTokens 3) Parser parses the stream of tokens and build an AST that retain pointers to the links (firstLink and lastLink) The advantage of storing links rather than position is that we can add/remove links easily without recomputing positions 4) refactoring operates on AST and replaces links content, eventually if it add or remove links, then some special care of intersticial SpaceToken has to happen. 5) printing refactored source code is printing the tokens verbatim (that is just nextPutAll: their contents) Nicolas 2009/11/22 Lukas Renggli <renggli(a)gmail.com>: >> at one point gwenael changed the parser to handle comments but >> apparently the code could not be loaded and it was never integrated. I imagine that now it is too divergent. > > The code never worked for me. It broke most parts of the Refactoring Engine. > > The new code is a fresh port from John's repository. There existing > port was a total mess, partly broken, partly incomplete, very > inefficient and polluted with code from other projects. > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Schwab,Wilhelm K

Lukas, Interesting news about #promptOnRefactoring. However, you are the first person involved with the RB to ever mention it to me. Previously, I was simply told that commenting methods is a waste of time and I would be happier and healthier if I would change my misguided ways. Obviously, I take a different view of it, and all the more so having recently ported a lot of aging code to Pharo; comments saved my skin yet again. If the RB can be made to respect them, I will eagerly use it. Bill -----Original Message----- From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Lukas Renggli Sent: Sunday, November 22, 2009 9:05 AM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] (no subject) > What about the handling of comments? Â The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless. The new RB has a better representation of where comments are put. Since I personally almost never put comments within the method body, I cannot comment if the formatting of comments has improved. For tools the Smalltalk syntax is a nightmare. A parser can only use heuristics to assign a comment to the right statement. Writing a formatter is even more difficult, because about everything can be nested in everything else. > If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad. That's what RB does (and always did) when you enabled the preference #promptOnRefactoring (which is the default in OB-Refactory). Lukas -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Nicolas Cellier

Without looking at RB code, this is how I would handle it: 1) Reify Smalltalk tokens, and create a SpaceToken (can include various separators character and comments) the tokens just include a copy of corresponding source text. 2) Scanner creates a linked list of tokens including SpaceTokens 3) Parser parses the stream of tokens and build an AST that retain pointers to the links (firstLink and lastLink) The advantage of storing links rather than position is that we can add/remove links easily without recomputing positions 4) refactoring operates on AST and replaces links content, eventually if it add or remove links, then some special care of intersticial SpaceToken has to happen. 5) printing refactored source code is printing the tokens verbatim (that is just nextPutAll: their contents) Nicolas 2009/11/22 Lukas Renggli <renggli(a)gmail.com>: >> at one point gwenael changed the parser to handle comments but apparently the code could not be loaded and >> it was never integrated. I imagine that now it is too divergent. > > The code never worked for me. It broke most parts of the Refactoring Engine. > > The new code is a fresh port from John's repository. There existing > port was a total mess, partly broken, partly incomplete, very > inefficient and polluted with code from other projects. > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Nov. 22, 2009

Re: [Pharo-project] Interval>>basicAt: fails

by Simon Denier

OK, this sounds like an education/learning the tricks problem then :) On 21 nov. 2009, at 17:37, Adrian Lienhard wrote: > Why would you want to call #basicAt:? > > As the comment in Object>>basicAt: says, this works only for indexable > objects, and Interval is not indexable. So this looks like the > expected behavior to me. > > Cheers, > Adrian > > On Nov 21, 2009, at 21:32 , Simon Denier wrote: > >> Try the following (1) >> (1 to: 10) at: 2 >> returns 2 >> >> then (2) >> (1 to: 10) basicAt: 2 >> raises an exception >> >> It is surprising. >> >> Problem is that Interval redefines at: >> but in (2), the lookup retrieves Object>>basicAt: which only works >> for >> indexable objects (which an interval is not). >> >> >> So Interval should also redefine basicAt: ? >> >> >> -- >> Simon >> >> >> >> >> _______________________________________________ >> Pharo-project mailing list >> Pharo-project(a)lists.gforge.inria.fr >> http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project -- Simon

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Lukas Renggli

> at one point gwenael changed the parser to handle comments but apparently the code could not be loaded and > it was never integrated. I imagine that now it is too divergent. The code never worked for me. It broke most parts of the Refactoring Engine. The new code is a fresh port from John's repository. There existing port was a total mess, partly broken, partly incomplete, very inefficient and polluted with code from other projects. Lukas -- Lukas Renggli http://www.lukas-renggli.ch

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Lukas Renggli

> What about the handling of comments? Â The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless. The new RB has a better representation of where comments are put. Since I personally almost never put comments within the method body, I cannot comment if the formatting of comments has improved. For tools the Smalltalk syntax is a nightmare. A parser can only use heuristics to assign a comment to the right statement. Writing a formatter is even more difficult, because about everything can be nested in everything else. > If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad. That's what RB does (and always did) when you enabled the preference #promptOnRefactoring (which is the default in OB-Refactory). Lukas -- Lukas Renggli http://www.lukas-renggli.ch

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by StÃ©phane Ducasse

at one point gwenael changed the parser to handle comments but apparently the code could not be loaded and it was never integrated. I imagine that now it is too divergent. Stef On Nov 22, 2009, at 2:44 PM, Schwab,Wilhelm K wrote: > Lukas, > > What about the handling of comments? The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless. If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad. > > Bill > > > -----Original Message----- > From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Lukas Renggli > Sent: Sunday, November 22, 2009 7:33 AM > To: Pharo-project(a)lists.gforge.inria.fr > Subject: Re: [Pharo-project] (no subject) > >> I still have issues with not preserving code formatting in the latest >> pharo-dev. Particularly, method renaming cause code to be reformatted. > > The formatting cannot be preserved in all cases. For example, if the renamed method has many arguments or is sent with complex arguments (block, nested message sends, ...) the code is still reformatted. For the cases where an automatic formatting is not avoidable, the class RBConfigurableFormatter can be configured to produce something that is more to your likings. > > Lukas > > -- > Lukas Renggli > http://www.lukas-renggli.ch > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Nov. 22, 2009

[Pharo-project] [update 1.1] #11059

by Marcus Denker

11059 ----- Issue 1490: clean some dead code (another 20 methods...) Issue 1482: ScriptLoader new cleanUpMethods broken Marcus -- Marcus Denker -- http://www.marcusdenker.de

Nov. 22, 2009

Re: [Pharo-project] (no subject)

by Schwab,Wilhelm K

Lukas, What about the handling of comments? The RB or its parser, etc. made *no* attempt to get them in the correct position as presented to the formatter, making the effort of creating a formatter fairly pointless. If that has improved, I will take another look, but my instinct is to ask for the RB to make a list what it would need to reformat so I can take care of it myself - it was that bad. Bill -----Original Message----- From: pharo-project-bounces(a)lists.gforge.inria.fr [mailto:pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Lukas Renggli Sent: Sunday, November 22, 2009 7:33 AM To: Pharo-project(a)lists.gforge.inria.fr Subject: Re: [Pharo-project] (no subject) > I still have issues with not preserving code formatting in the latest > pharo-dev. Particularly, method renaming cause code to be reformatted. The formatting cannot be preserved in all cases. For example, if the renamed method has many arguments or is sent with complex arguments (block, nested message sends, ...) the code is still reformatted. For the cases where an automatic formatting is not avoidable, the class RBConfigurableFormatter can be configured to produce something that is more to your likings. Lukas -- Lukas Renggli http://www.lukas-renggli.ch _______________________________________________ Pharo-project mailing list Pharo-project(a)lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Nov. 22, 2009