Le 28/10/2014 14:33, Mark Rizun a écrit :
In the first place why I'm doing this. I work on Rewrite Tool and it's main functionality bases on replacing nodes in AST. Plus it works with sourceIntervals of nodes. Untill now my solution was: if I replace node, I reparse tree to get intervals updated
I print the modified tree and parse :)
However this solution brought new problems.
Which ones?
Second reason is, that I think it makes sense to update interval of all AST if you replace one node. For example, we have: obj1 foo + obj2 bar and we replace obj1 with myObject. The interval of ast was 1 to: 19, and now it should be 1 to: 23.
No it shouldn't. If the source has not been regenerated from the modified AST, then 'source copyFrom: theBarASTNode start to: theBarASTNode stop' end past the end of it (20 to 23 with source ending at 19). If you regenerate the source, then you can parse it and you'll have correct intervals. Thierry
Mark
2014-10-28 14:00 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com>>:
Le 28/10/2014 12:45, Mark Rizun a écrit :
Yes, because they are wrong. Here is an issue: https://pharo.fogbugz.com/f/__cases/14254/AST-method-__replaceWith-does-not-... <https://pharo.fogbugz.com/f/cases/14254/AST-method-replaceWith-does-not-chan...>
I would say that they are correct.
When I write source to source compilers, I admit that anything that I change in the AST (via replaceWith equivalent) has no valid source interval (since it does not exist in the original source). However, all unmodified nodes should keep their 'non-modified' source interval (since I may need it to fetch the relevant text from the source).
If I want my modification to the AST to have valid source intervals, then, I need to regenerate the source from the modified AST. And only then they are valid.
You may want to update the source interval when you do a replaceWith, but the only thing we will get with what you want to do is that, after a replaceWith, no source interval can be trusted since it may end up past the end of the original source string length.
Thierry
2014-10-28 13:32 GMT+02:00 Thierry Goubier <thierry.goubier@gmail.com <mailto:thierry.goubier@gmail.com> <mailto:thierry.goubier@gmail.__com <mailto:thierry.goubier@gmail.com>>>:
Le 28/10/2014 12:12, Mark Rizun a écrit :
Well, not really.
Technically, tokens are used to drive a parser from a scanner.
If an AST node knows how to relate itself to its original source code chunk and is able to print itself correctly, then tokens are redundant.
In short, if you work with parsers, you'd better know what tokens are. If you're only working with the AST, tokens are redundant and noise (i.e. they often have a type (or more than one) which is only understood by the parser).
I'm working with ASTs sourceInterval. Trying to calculate it after method replaceWith:. You see, my proble was that each node of AST doesn't hold its start and stop position in same place. So I thought that token is such a place, however, eventually I understood that RBValueNodes don't have tokens:)
Do you mean you're trying to do a replace and update the positions of all the nodes ?
Thierry