Hi all, Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be? Cheers, Mark
2014-10-27 19:36 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Hi all,
Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be?
RBValueNode is an abstract class.
Cheers, Mark
Thanks. I see that, however RBBlockNode or RBArrayNode doesn't have tokens. These classes have only methods in accessing-token protocol. I think it would be better if we have token object for those classes, because it makes more sense to hold such information in token object. Mark 2014-10-28 11:59 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2014-10-27 19:36 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Hi all,
Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be?
RBValueNode is an abstract class.
Cheers, Mark
On 28 Oct 2014, at 11:23, Mark Rizun <mrizun@gmail.com> wrote:
Thanks. I see that, however RBBlockNode or RBArrayNode doesn't have tokens. These classes have only methods in accessing-token protocol. I think it would be better if we have token object for those classes, because it makes more sense to hold such information in token object.
No, actually we should get rid of tokens.. they are just used for parsing (the scanner produces tokens, there is no token for a block, as a block consists of many tokens⦠so conceptually, a block can not have a token). Tokens expose a very low level implementation artefact of the parser to the AST model, this is not good. There is⦠https://pharo.fogbugz.com/f/cases/11992/Remove-tokens-from-the-AST-Core-Node... We should have a look at that and integrate it. This should simplify lots of things. Marcus
Le 28/10/2014 11:23, Mark Rizun a écrit :
Thanks. I see that, however RBBlockNode or RBArrayNode doesn't have tokens. These classes have only methods in accessing-token protocol. I think it would be better if we have token object for those classes, because it makes more sense to hold such information in token object.
Well, not really. Technically, tokens are used to drive a parser from a scanner. If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant. In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser). Example of how it is done: RBPragmaNode accessing-tokens gives access to left and right, which are positions, not tokens. Thierry
Mark
2014-10-28 11:59 GMT+02:00 Nicolai Hess <nicolaihess@web.de <mailto:nicolaihess@web.de>>:
2014-10-27 19:36 GMT+01:00 Mark Rizun <mrizun@gmail.com <mailto:mrizun@gmail.com>>:
Hi all,
Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be?
RBValueNode is an abstract class.
Cheers, Mark
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Example of how it is done:
RBPragmaNode accessing-tokens gives access to left and right, which are positions, not tokens.
Yes, I know.
Thierry
Mark
2014-10-28 11:59 GMT+02:00 Nicolai Hess <nicolaihess@web.de <mailto:nicolaihess@web.de>>:
2014-10-27 19:36 GMT+01:00 Mark Rizun <mrizun@gmail.com <mailto:mrizun@gmail.com>>:
Hi all,
Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be?
RBValueNode is an abstract class.
Cheers, Mark
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ? Thierry
Yes, because they are wrong. Here is an issue: https://pharo.fogbugz.com/f/cases/14254/AST-method-replaceWith-does-not-chan... 2014-10-28 13:32 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com>:
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ?
Thierry
Le 28/10/2014 12:45, Mark Rizun a écrit :
Yes, because they are wrong. Here is an issue: https://pharo.fogbugz.com/f/cases/14254/AST-method-replaceWith-does-not-chan...
I would say that they are correct. When I write source to source compilers, I admit that anything that I change in the AST (via replaceWith equivalent) has no valid source interval (since it does not exist in the original source). However, all unmodified nodes should keep their 'non-modified' source interval (since I may need it to fetch the relevant text from the source). If I want my modification to the AST to have valid source intervals, then, I need to regenerate the source from the modified AST. And only then they are valid. You may want to update the source interval when you do a replaceWith, but the only thing we will get with what you want to do is that, after a replaceWith, no source interval can be trusted since it may end up past the end of the original source string length. Thierry
2014-10-28 13:32 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ?
Thierry
In the first place why I'm doing this. I work on Rewrite Tool and it's main functionality bases on replacing nodes in AST. Plus it works with sourceIntervals of nodes. Untill now my solution was: if I replace node, I reparse tree to get intervals updated However this solution brought new problems. Second reason is, that I think it makes sense to update interval of all AST if you replace one node. For example, we have: obj1 foo + obj2 bar and we replace obj1 with myObject. The interval of ast was 1 to: 19, and now it should be 1 to: 23. Mark 2014-10-28 14:00 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com>:
Le 28/10/2014 12:45, Mark Rizun a écrit :
Yes, because they are wrong. Here is an issue: https://pharo.fogbugz.com/f/cases/14254/AST-method- replaceWith-does-not-change-source-interval
I would say that they are correct.
When I write source to source compilers, I admit that anything that I change in the AST (via replaceWith equivalent) has no valid source interval (since it does not exist in the original source). However, all unmodified nodes should keep their 'non-modified' source interval (since I may need it to fetch the relevant text from the source).
If I want my modification to the AST to have valid source intervals, then, I need to regenerate the source from the modified AST. And only then they are valid.
You may want to update the source interval when you do a replaceWith, but the only thing we will get with what you want to do is that, after a replaceWith, no source interval can be trusted since it may end up past the end of the original source string length.
Thierry
2014-10-28 13:32 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ?
Thierry
Le 28/10/2014 14:33, Mark Rizun a écrit :
In the first place why I'm doing this. I work on Rewrite Tool and it's main functionality bases on replacing nodes in AST. Plus it works with sourceIntervals of nodes. Untill now my solution was: if I replace node, I reparse tree to get intervals updated
I print the modified tree and parse :)
However this solution brought new problems.
Which ones?
Second reason is, that I think it makes sense to update interval of all AST if you replace one node. For example, we have: obj1 foo + obj2 bar and we replace obj1 with myObject. The interval of ast was 1 to: 19, and now it should be 1 to: 23.
No it shouldn't. If the source has not been regenerated from the modified AST, then 'source copyFrom: theBarASTNode start to: theBarASTNode stop' end past the end of it (20 to 23 with source ending at 19). If you regenerate the source, then you can parse it and you'll have correct intervals. Thierry
Mark
2014-10-28 14:00 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Le 28/10/2014 12:45, Mark Rizun a écrit :
Yes, because they are wrong. Here is an issue: https://pharo.fogbugz.com/f/__cases/14254/AST-method-__replaceWith-does-not-... <https://pharo.fogbugz.com/f/cases/14254/AST-method-replaceWith-does-not-chan...>
I would say that they are correct.
When I write source to source compilers, I admit that anything that I change in the AST (via replaceWith equivalent) has no valid source interval (since it does not exist in the original source). However, all unmodified nodes should keep their 'non-modified' source interval (since I may need it to fetch the relevant text from the source).
If I want my modification to the AST to have valid source intervals, then, I need to regenerate the source from the modified AST. And only then they are valid.
You may want to update the source interval when you do a replaceWith, but the only thing we will get with what you want to do is that, after a replaceWith, no source interval can be trusted since it may end up past the end of the original source string length.
Thierry
2014-10-28 13:32 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com> <mailto:thierry.goubier@gmail.__com <mailto:thierry.goubier@gmail.com>>>:
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ?
Thierry
Which ones?
In my tool each node has property oldNodes, which holds collection of obviously AST nodes:) When I replace one of node I have to update source interval in some way. 1)If I update it with reparsing, I loose all data about oldNodes for each node of my AST. So I have to save old AST with all oldNodes, and somehow detect which nodes were not changed and reassign their lost oldNodes. But sometimes it's difficult to detect where and what you have to assign, as sometimes AST may be changed in dramatic way. 2) But if source interval is updated automatically I don't bother with losing data for all AST. I just have to update oldNodes for node that was replaced. That is way I'd like to have automatically updated source interval.
No it shouldn't. If the source has not been regenerated from the modified AST, then
'source copyFrom: theBarASTNode start to: theBarASTNode stop'
end past the end of it (20 to 23 with source ending at 19).
If you regenerate the source, then you can parse it and you'll have correct intervals.
Marcus, do I have to redo everything back, or you can somehow remove that slice from newest version? Mark
2014-10-29 8:32 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Which ones?
In my tool each node has property oldNodes, which holds collection of obviously AST nodes:) When I replace one of node I have to update source interval in some way.
1)If I update it with reparsing, I loose all data about oldNodes for each node of my AST. So I have to save old AST with all oldNodes, and somehow detect which nodes were not changed and reassign their lost oldNodes. But sometimes it's difficult to detect where and what you have to assign, as sometimes AST may be changed in dramatic way.
Ok; yes, I can relate to that. But, knowing who has designed the RB ast, I'm sure it has a proper equality property where: oldNodeFromAST = sameNodeFromASTreparsed holds true. So you can reparse and rematch old node to new node. But I would only regenerate and reparse when I need to update the source intervals (or display the modified source).
2) But if source interval is updated automatically I don't bother with losing data for all AST. I just have to update oldNodes for node that was replaced.
Well, not really. If during regeneration of your source, you change the way the code is formatted (more tabs here, removing a return there, etc...) then regenerated source intervals are different from modified ast source intervals. So your 'modify source intervals' is very fragile for me. The only way I can see a way out for that problem: Inserted / replaced node have source intervals in a separate source; unmodified nodes keep their source interval. This is the way I do it in my source to source compilers when I want to inject code (and tag in the generated file where that code comes from: usefull for debugging). Or the match of old node to new node as seen above.
That is way I'd like to have automatically updated source interval.
I'm still not entirely sure why. Source intervals are only there to help relating the ast to the source, not much else, really. Thierry
No it shouldn't. If the source has not been regenerated from the modified AST, then
'source copyFrom: theBarASTNode start to: theBarASTNode stop'
end past the end of it (20 to 23 with source ending at 19).
If you regenerate the source, then you can parse it and you'll have correct intervals.
Marcus, do I have to redo everything back, or you can somehow remove that slice from newest version?
Mark
That is way I'd like to have automatically updated source interval.
I'm still not entirely sure why. Source intervals are only there to help relating the ast to the source, not much else, really.
I use source intervals to detect which node is selected and than in the right-click menu user can see only options that are relevanto to selected node, as it is also made in SmartSuggestions.
P.S. I have a solution, but don't know if it's appropriate: I remove updating of source interval from replaceWith: method, but my tool will do all the calculations of interval on it's own. 2014-10-29 10:59 GMT+02:00 Mark Rizun <mrizun@gmail.com>:
That is way I'd like to have automatically updated source interval.
I'm still not entirely sure why. Source intervals are only there to help relating the ast to the source, not much else, really.
I use source intervals to detect which node is selected and than in the right-click menu user can see only options that are relevanto to selected node, as it is also made in SmartSuggestions.
2014-10-29 10:09 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
P.S. I have a solution, but don't know if it's appropriate: I remove updating of source interval from replaceWith: method, but my tool will do all the calculations of interval on it's own.
This is a possibility: have a transform; when you get selection intervals, look if they are inside "replaced areas" or outside of it; increase or decrease the indexes to compensate for added code / removed code. But this is more complex than it looks. But I would try the equality over an AST first. A lot more robust for me. Thierry
2014-10-29 10:59 GMT+02:00 Mark Rizun <mrizun@gmail.com>:
That is way I'd like to have automatically updated source interval.
I'm still not entirely sure why. Source intervals are only there to help relating the ast to the source, not much else, really.
I use source intervals to detect which node is selected and than in the right-click menu user can see only options that are relevanto to selected node, as it is also made in SmartSuggestions.
2014-10-29 9:59 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
I use source intervals to detect which node is selected and than in the right-click menu user can see only options that are relevanto to selected node, as it is also made in SmartSuggestions.
I know that use case ;) Ok, then this means you are regenerating the code (1) (or are you doing replace a node / insert new source at right place? (2)) Thinking a bit about it, I'd try reparse, get node from selection index, find equal old node in old (modified) ast, or replace old (modified) ast with new one. Thierry
2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Second one, I do replace node all the time.
You insert new code inside the text view? For me, if you replace nodes and display that, then you are "slowly" replacing nodes. Anything which has some display to a user in the loop is "slow".
Thinking a bit about it, I'd try reparse, get node from selection index,
find equal old node in old (modified) ast, or replace old (modified) ast with new one.
Can you explain this, sorry I didn't get the point
Use either =, equalTo:withMapping:, match:inContext: to find the relevant new node equal to your old node in the new ast. Thierry
2014-10-29 11:40 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com>:
2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Second one, I do replace node all the time.
You insert new code inside the text view?
Yes
For me, if you replace nodes and display that, then you are "slowly" replacing nodes. Anything which has some display to a user in the loop is "slow".
I'm not doing this in loop. I have AST and text view of it. Than I do one replace and update text view. When I wrote "all the time", I ment replacing nodes is very important in my tool, as it does main functionality.
Thinking a bit about it, I'd try reparse, get node from selection index,
find equal old node in old (modified) ast, or replace old (modified) ast with new one.
Can you explain this, sorry I didn't get the point
Use either =, equalTo:withMapping:, match:inContext: to find the relevant new node equal to your old node in the new ast.
Good, thanks for advice. Firstly, I will check your suggestion with equality. If it fails for me, I'll try my suggestion with calculating inside tool. Thanks again, Mark
2014-10-29 10:44 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
2014-10-29 11:40 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com>:
2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
Second one, I do replace node all the time.
You insert new code inside the text view?
Yes
For me, if you replace nodes and display that, then you are "slowly" replacing nodes. Anything which has some display to a user in the loop is "slow".
I'm not doing this in loop. I have AST and text view of it. Than I do one replace and update text view. When I wrote "all the time", I ment replacing nodes is very important in my tool, as it does main functionality.
This is well how I understood it. Are you implementing something around the use of refactoring?
Thinking a bit about it, I'd try reparse, get node from selection index,
find equal old node in old (modified) ast, or replace old (modified) ast with new one.
Can you explain this, sorry I didn't get the point
Use either =, equalTo:withMapping:, match:inContext: to find the relevant new node equal to your old node in the new ast.
Good, thanks for advice. Firstly, I will check your suggestion with equality. If it fails for me, I'll try my suggestion with calculating inside tool.
Keep us updated on your solution. It is significant because the equality between ast nodes is part of the SmaCC parser generator toolkit, so any solution on that may also be used when manipulating non-smalltalk code (C, Java, C#, Python, R, Ada, etc...). Thierry
Thanks again, Mark
This is well how I understood it.
Are you implementing something around the use of refactoring?
I'm implementing tool for creating rewrite rules. Here is a blog about it: http://myfuncoding.blogspot.com/ Also I write chapter for PharoForEnterprise: https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
Keep us updated on your solution. It is significant because the equality between ast nodes is part of the SmaCC parser generator toolkit, so any solution on that may also be used when manipulating non-smalltalk code (C, Java, C#, Python, R, Ada, etc...).
Sure, I will.
I misunderstood you, of course I can find the node in old ast which was replaced, using operator =. Sure it works, but the problem is I have to pass oldNodes each time from old node to new one.
P.S. Moreover, it's not only that I have to pass oldNodes each time from old node to new one. Each time I replace node I reparse tree and I lose oldNodes from each node of ast:) So, I have to pass oldNodes from each node of ast to each coresponding node of new ast. And it's a real problem. 2014-10-29 12:16 GMT+02:00 Mark Rizun <mrizun@gmail.com>:
I misunderstood you, of course I can find the node in old ast which was replaced, using operator =. Sure it works, but the problem is I have to pass oldNodes each time from old node to new one.
2014-10-29 11:16 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
I misunderstood you, of course I can find the node in old ast which was replaced, using operator =. Sure it works, but the problem is I have to pass oldNodes each time from old node to new one.
I think you can work recreating the ast. I'm not sure there is any reason to keep track of old nodes, because what you do is: apply a change to ast under selection; rebuilt RB matching / transform rule; apply RB rule to sample code. update all four panes. Thierry
I had a look at the video, and it looks great. Certainly the way to go.
Well, it's not that great. And it shows pretty old version of what I have now. I should make a new better one:)
I had some ideas along those, but I never have time to try them. My interest would have been to show how many methods (and where: group-package-tag-class?) the search pattern will impact.
In current version you can generate rule(it creates class for your rule), and than you can select packages/classes/methods you want to apply this rule
One suggestion. When in the menu, try to auto-select the relevant keyword or literal;
You mean in all other panes? It would be really cool
give access to the ast navigation stuff (up select the enclosing ast node, down the first node inside).
That's really great! It'will take time, but I'm going to implement that, just have to remember:) I think you can work recreating the ast. I'm not sure there is any reason
to keep track of old nodes, because what you do is: apply a change to ast under selection; rebuilt RB matching / transform rule; apply RB rule to sample code. update all four panes.
The process on video is: creating your own rule step by step. While doing this user may make a mistake. I keep oldNodes, because I want to have an "Undo on node" action. It returns node into it's previous state.
2014-10-29 11:41 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
I had a look at the video, and it looks great. Certainly the way to go.
Well, it's not that great. And it shows pretty old version of what I have now. I should make a new better one:)
I had some ideas along those, but I never have time to try them. My interest would have been to show how many methods (and where: group-package-tag-class?) the search pattern will impact.
In current version you can generate rule(it creates class for your rule), and than you can select packages/classes/methods you want to apply this rule
This is the one which sounds difficult for me. Patterns are fairly global in nature, and they may match synonyms (i.e. methods of the same name but with different meanings), so I'm worried about the mastery of my changes. For an isNil ifTrue: to ifNil:, it's easy. But for an indexAt: to at: because I'm changing an internal API, I'd like a better preview of what it will affect before launching it (or a way to examine the resulting changes, which is a bit convoluted).
One suggestion. When in the menu, try to auto-select the relevant keyword or literal;
You mean in all other panes? It would be really cool
You're jumping ahead of my mind :) Yes, it would be cool (but not sure it will be that easy to do in all cases; if you select @var1, would it highlight myVariable in the other panes... hum, interesting to have).
give access to the ast navigation stuff (up select the enclosing ast node, down the first node inside).
That's really great! It'will take time, but I'm going to implement that, just have to remember:)
I think this is already available, you just have to use it (or trigger it).
I think you can work recreating the ast. I'm not sure there is any reason
to keep track of old nodes, because what you do is: apply a change to ast under selection; rebuilt RB matching / transform rule; apply RB rule to sample code. update all four panes.
The process on video is: creating your own rule step by step. While doing this user may make a mistake. I keep oldNodes, because I want to have an "Undo on node" action. It returns node into it's previous state.
In that case, I would keep the old node, and simply try to find where is the old node equal in the new ast when undoing (or when building the menu to show that an undo is possible on that node). Thierry
This is the one which sounds difficult for me. Patterns are fairly global in nature, and they may match synonyms (i.e. methods of the same name but with different meanings), so I'm worried about the mastery of my changes.
For an isNil ifTrue: to ifNil:, it's easy. But for an indexAt: to at: because I'm changing an internal API, I'd like a better preview of what it will affect before launching it (or a way to examine the resulting changes, which is a bit convoluted).
You will see upcoming changes in Changes browser, so you can decide if it's what you really want. It's like applying refactoring. I think this is already available, you just have to use it (or trigger it).
I'm not aware of this kind of things, can you specify where it is available? In that case, I would keep the old node, and simply try to find where is
the old node equal in the new ast when undoing (or when building the menu to show that an undo is possible on that node).
That's pretty much the same what I'm doing. Mark
2014-10-29 15:54 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
This is the one which sounds difficult for me. Patterns are fairly global
in nature, and they may match synonyms (i.e. methods of the same name but with different meanings), so I'm worried about the mastery of my changes.
For an isNil ifTrue: to ifNil:, it's easy. But for an indexAt: to at: because I'm changing an internal API, I'd like a better preview of what it will affect before launching it (or a way to examine the resulting changes, which is a bit convoluted).
You will see upcoming changes in Changes browser, so you can decide if it's what you really want.
You mean the changes tool in Monticello ?
It's like applying refactoring.
Hum. Yes.
I think this is already available, you just have to use it (or trigger it).
I'm not aware of this kind of things, can you specify where it is available?
In that case, I would keep the old node, and simply try to find where is
the old node equal in the new ast when undoing (or when building the menu to show that an undo is possible on that node).
That's pretty much the same what I'm doing.
Oh well, then you're done ;) Thierry
Mark
2014-10-29 16:17 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
You mean the changes tool in Monticello ?
No I mean, that before your rule is applied you will see the same window, when you choose refactoring. I mean Changes Browser. Here it is:
[image: ÐбÑдоване зобÑÐ°Ð¶ÐµÐ½Ð½Ñ 1]
Ok, thanks.
this is a gui I never ended up seeing, apart from dismissing it for obvious refactorings when I was using OmniBrowser :( I think I got rid of it somehow ;) Hum, not convenient. Certainly doesn't invite me to do refactorings :( Thierry
On 29 Oct 2014, at 16:27, Thierry Goubier <thierry.goubier@gmail.com> wrote:
2014-10-29 16:17 GMT+01:00 Mark Rizun <mrizun@gmail.com <mailto:mrizun@gmail.com>>: You mean the changes tool in Monticello ?
No I mean, that before your rule is applied you will see the same window, when you choose refactoring. I mean Changes Browser. Here it is:
<changes_browser.png>
Ok, thanks.
this is a gui I never ended up seeing, apart from dismissing it for obvious refactorings when I was using OmniBrowser :( I think I got rid of it somehow ;)
Hum, not convenient.
Certainly doesn't invite me to do refactorings :(
Yes⦠improvements are very welcome⦠Marcus
2014-10-30 7:53 GMT+01:00 Marcus Denker <marcus.denker@inria.fr>:
On 29 Oct 2014, at 16:27, Thierry Goubier <thierry.goubier@gmail.com> wrote:
2014-10-29 16:17 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
You mean the changes tool in Monticello ?
No I mean, that before your rule is applied you will see the same window, when you choose refactoring. I mean Changes Browser. Here it is:
<changes_browser.png>
Ok, thanks.
this is a gui I never ended up seeing, apart from dismissing it for obvious refactorings when I was using OmniBrowser :( I think I got rid of it somehow ;)
Hum, not convenient.
Certainly doesn't invite me to do refactorings :(
Yes... improvements are very welcome...
Out of time :( And there is nothing in Pharo now that allow for a synthetic view of a subset of the overall 'code' :( What is only available is a list of selectors, or a full blown Nautilus on a scope. Nothing I'd call synthetic. I hate thoses, for example: [image: Images intégrées 1] Thierry
Marcus
Hi Stef, this one is in SmaCC. I'll see if I can switch to your solution. I had a look at Nautilus add protocol for fun. 2303 entries for a subclass of MorphTreeModel :( I reduce that to 11 entries in AltBrowser add protocol for the same use case. Thierry
this one is in SmaCC. I'll see if I can switch to your solution.
I had a look at Nautilus add protocol for fun. 2303 entries for a subclass of MorphTreeModel :( I reduce that to 11 entries in AltBrowser add protocol for the same use case. :( Same feeling here but we will live with it for now. I started to change to the feel of the pop up.
Stef
2014-10-29 11:11 GMT+01:00 Mark Rizun <mrizun@gmail.com>:
This is well how I understood it.
Are you implementing something around the use of refactoring?
I'm implementing tool for creating rewrite rules. Here is a blog about it: http://myfuncoding.blogspot.com/ Also I write chapter for PharoForEnterprise: https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tre...
I had a look at the video, and it looks great. Certainly the way to go. I had some ideas along those, but I never have time to try them. My interest would have been to show how many methods (and where: group-package-tag-class?) the search pattern will impact. One suggestion. When in the menu, try to auto-select the relevant keyword or literal; give access to the ast navigation stuff (up select the enclosing ast node, down the first node inside). Thierry
Keep us updated on your solution. It is significant because the equality between ast nodes is part of the SmaCC parser generator toolkit, so any solution on that may also be used when manipulating non-smalltalk code (C, Java, C#, Python, R, Ada, etc...).
Sure, I will.
Thanks thierry and mark for this interesting discussion! Stef On 27/10/14 13:36, Mark Rizun wrote:
Hi all,
Trying to understand here how tokens are used in AST. So far I can not see any order in usage of tokens. For instance, why RBValueNode doesn't have token? Is it haow it's supposed to be?
Cheers, Mark
participants (5)
-
Marcus Denker -
Mark Rizun -
Nicolai Hess -
stepharo -
Thierry Goubier