October 2014 - Pharo-users - lists.pharo.org

Re: [Pharo-users] AST tokens question

by Mark Rizun

> > This is well how I understood it. > > Are you implementing something around the use of refactoring? > I'm implementing tool for creating rewrite rules. Here is a blog about it: http://myfuncoding.blogspot.com/ Also I write chapter for PharoForEnterprise: https://github.com/SquareBracketAssociates/PharoForTheEnterprise-english/tr… > Keep us updated on your solution. It is significant because the equality > between ast nodes is part of the SmaCC parser generator toolkit, so any > solution on that may also be used when manipulating non-smalltalk code (C, > Java, C#, Python, R, Ada, etc...). > Sure, I will.

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Thierry Goubier

2014-10-29 10:44 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > > > 2014-10-29 11:40 GMT+02:00 Thierry Goubier <thierry.goubier(a)gmail.com>: > >> >> >> 2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: >> >>> Second one, I do replace node all the time. >>> >> >> You insert new code inside the text view? >> > > Yes > > >> >> For me, if you replace nodes and display that, then you are "slowly" >> replacing nodes. Anything which has some display to a user in the loop is >> "slow". >> > I'm not doing this in loop. I have AST and text view of it. Than I do one > replace and update text view. > When I wrote "all the time", I ment replacing nodes is very important in > my tool, as it does main functionality. > This is well how I understood it. Are you implementing something around the use of refactoring? > > >> >> >>> Thinking a bit about it, I'd try reparse, get node from selection index, >>>> find equal old node in old (modified) ast, or replace old (modified) ast >>>> with new one. >>>> >>> Can you explain this, sorry I didn't get the point >>> >> >> Use either =, equalTo:withMapping:, match:inContext: to find the relevant >> new node equal to your old node in the new ast. >> > Good, thanks for advice. Firstly, I will check your suggestion with > equality. If it fails for me, I'll try my suggestion with calculating > inside tool. > Keep us updated on your solution. It is significant because the equality between ast nodes is part of the SmaCC parser generator toolkit, so any solution on that may also be used when manipulating non-smalltalk code (C, Java, C#, Python, R, Ada, etc...). Thierry > > Thanks again, > Mark > >

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Mark Rizun

2014-10-29 11:40 GMT+02:00 Thierry Goubier <thierry.goubier(a)gmail.com>: > > > 2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > >> Second one, I do replace node all the time. >> > > You insert new code inside the text view? > Yes > > For me, if you replace nodes and display that, then you are "slowly" > replacing nodes. Anything which has some display to a user in the loop is > "slow". > I'm not doing this in loop. I have AST and text view of it. Than I do one replace and update text view. When I wrote "all the time", I ment replacing nodes is very important in my tool, as it does main functionality. > > >> Thinking a bit about it, I'd try reparse, get node from selection index, >>> find equal old node in old (modified) ast, or replace old (modified) ast >>> with new one. >>> >> Can you explain this, sorry I didn't get the point >> > > Use either =, equalTo:withMapping:, match:inContext: to find the relevant > new node equal to your old node in the new ast. > Good, thanks for advice. Firstly, I will check your suggestion with equality. If it fails for me, I'll try my suggestion with calculating inside tool. Thanks again, Mark

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Thierry Goubier

2014-10-29 10:22 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > Second one, I do replace node all the time. > You insert new code inside the text view? For me, if you replace nodes and display that, then you are "slowly" replacing nodes. Anything which has some display to a user in the loop is "slow". > Thinking a bit about it, I'd try reparse, get node from selection index, >> find equal old node in old (modified) ast, or replace old (modified) ast >> with new one. >> > Can you explain this, sorry I didn't get the point > Use either =, equalTo:withMapping:, match:inContext: to find the relevant new node equal to your old node in the new ast. Thierry

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Thierry Goubier

2014-10-29 10:09 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > P.S. I have a solution, but don't know if it's appropriate: I remove > updating of source interval from replaceWith: method, but my tool will do > all the calculations of interval on it's own. > This is a possibility: have a transform; when you get selection intervals, look if they are inside "replaced areas" or outside of it; increase or decrease the indexes to compensate for added code / removed code. But this is more complex than it looks. But I would try the equality over an AST first. A lot more robust for me. Thierry > > 2014-10-29 10:59 GMT+02:00 Mark Rizun <mrizun(a)gmail.com>: > >> That is way I'd like to have automatically updated source interval. >>>> >>> >>> I'm still not entirely sure why. Source intervals are only there to help >>> relating the ast to the source, not much else, really. >>> >> >> I use source intervals to detect which node is selected and than in the >> right-click menu user can see only options that are relevanto to selected >> node, as it is also made in SmartSuggestions. >> > >

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Mark Rizun

Second one, I do replace node all the time. Thinking a bit about it, I'd try reparse, get node from selection index, > find equal old node in old (modified) ast, or replace old (modified) ast > with new one. > Can you explain this, sorry I didn't get the point

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Thierry Goubier

2014-10-29 9:59 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > > I use source intervals to detect which node is selected and than in the > right-click menu user can see only options that are relevanto to selected > node, as it is also made in SmartSuggestions. > I know that use case ;) Ok, then this means you are regenerating the code (1) (or are you doing replace a node / insert new source at right place? (2)) Thinking a bit about it, I'd try reparse, get node from selection index, find equal old node in old (modified) ast, or replace old (modified) ast with new one. Thierry

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Mark Rizun

P.S. I have a solution, but don't know if it's appropriate: I remove updating of source interval from replaceWith: method, but my tool will do all the calculations of interval on it's own. 2014-10-29 10:59 GMT+02:00 Mark Rizun <mrizun(a)gmail.com>: > That is way I'd like to have automatically updated source interval. >>> >> >> I'm still not entirely sure why. Source intervals are only there to help >> relating the ast to the source, not much else, really. >> > > I use source intervals to detect which node is selected and than in the > right-click menu user can see only options that are relevanto to selected > node, as it is also made in SmartSuggestions. >

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Mark Rizun

> > That is way I'd like to have automatically updated source interval. >> > > I'm still not entirely sure why. Source intervals are only there to help > relating the ast to the source, not much else, really. > I use source intervals to detect which node is selected and than in the right-click menu user can see only options that are relevanto to selected node, as it is also made in SmartSuggestions.

Oct. 29, 2014

Re: [Pharo-users] AST tokens question

by Thierry Goubier

2014-10-29 8:32 GMT+01:00 Mark Rizun <mrizun(a)gmail.com>: > > Which ones? > > > In my tool each node has property oldNodes, which holds collection of > obviously AST nodes:) > When I replace one of node I have to update source interval in some way. > > 1)If I update it with reparsing, I loose all data about oldNodes for each > node of my AST. > So I have to save old AST with all oldNodes, and somehow detect which > nodes were not changed and reassign their lost oldNodes. > But sometimes it's difficult to detect where and what you have to assign, > as sometimes AST may be changed in dramatic way. > Ok; yes, I can relate to that. But, knowing who has designed the RB ast, I'm sure it has a proper equality property where: oldNodeFromAST = sameNodeFromASTreparsed holds true. So you can reparse and rematch old node to new node. But I would only regenerate and reparse when I need to update the source intervals (or display the modified source). > 2) But if source interval is updated automatically I don't bother with > losing data for all AST. > I just have to update oldNodes for node that was replaced. > Well, not really. If during regeneration of your source, you change the way the code is formatted (more tabs here, removing a return there, etc...) then regenerated source intervals are different from modified ast source intervals. So your 'modify source intervals' is very fragile for me. The only way I can see a way out for that problem: Inserted / replaced node have source intervals in a separate source; unmodified nodes keep their source interval. This is the way I do it in my source to source compilers when I want to inject code (and tag in the generated file where that code comes from: usefull for debugging). Or the match of old node to new node as seen above. > > That is way I'd like to have automatically updated source interval. > I'm still not entirely sure why. Source intervals are only there to help relating the ast to the source, not much else, really. Thierry > > >> >> No it shouldn't. If the source has not been regenerated from the modified >> AST, then >> >> 'source copyFrom: theBarASTNode start to: theBarASTNode stop' >> >> end past the end of it (20 to 23 with source ending at 19). >> >> If you regenerate the source, then you can parse it and you'll have >> correct intervals. >> > > Marcus, do I have to redo everything back, or you can somehow remove that > slice from newest version? > > Mark > > >

Oct. 29, 2014