On May 20, 2013, at 4:44 PM, Paul Davidowitz <pdavidow@fastmail.fm> wrote:
I just started following this Dev mailing list, and I saw that 'AST Everywhere' and 'Reflectivity' were mentioned as very promising (I already know about Slots). Can someone in a nutshell please describe these developments? Thanks,
Right now, the compiler uses it's own AST, Syntax highlighting uses a very minimal token stream (and it's own parser), the refactoring engine has one, too. And then there is Bytecode which is used by the Debugger, for example. As a first step we plan to use the Refactoring AST for everything (Compiler, Refactoring, Syntax highlighting ) There is a AST Interpreter useful for experimenting, maybe in the future the debugger can use this instead of relying on byte code. (but this is for far later). Now with the AST already there, all the tools can rely on it (Gisela Decuzzi is working on this:) Navigation. When selecting code or navigating expressions, the editor can do better then just using word boundaries: http://www.youtube.com/watch?feature=player_embedded&v=pFLyzEI0jmE Suggestions. When we want to do an actions on a selection, why not just show those that make sense? Why do I need to carefully highlight a selector to browser implementors? The AST model can do much better! http://www.youtube.com/watch?feature=player_embedded&v=WmNKbewOXkE Reflectivity ========= Behavioral Reflection is quite interesting.. but there is not that much in Smalltalk by default to deeply hook into how code is executed. There are MethodWrappers (to be loaded as an additional tool). There is an in-image Bytecode Interpreter (hidden in ContextPart). People started to want more. "In this class, all Ivar accesses should be treated differently". Things like that. So people tried to improve Smalltalk in the past. E.g. adding a CLOS Style MOP (Noury did that with MetaClassTalk). But it is difficult to do it in a way that it does not get very slow (not introduce an overhead for every operation). And in the end, the MetaClass style MOP is quite limiting: All change of semantic is done *per Class*. Already quite nice, but limiting... So in 2003 Eric Tanter finished his PhD and he invented this: Eric Tanter , Jacques Noye , Denis Caromel , Pierre Cointe: Partial Behavioral Reflection: Spatial and Temporal Selection of Reification http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.58.8141 And as fate wanted, he was visiting in Bern the week before Christmas 2004, which gave me enough time to implement a Bytecode Transformation thing in the style of Javassist that this stuff was using. But it was cumbersome⦠so at some point I thought "Bytecode Sucks", and took the Reflex model but transported it on the AST Level. http://scg.unibe.ch/archive/phd/denker-phd.pdf So⦠what this means is that the AST can be annotated with so-called "meta links". When you set a link, the system behind the scene makes sure that new code is generated on the next call. One way to think about this is that these links are just extremely generalized break-points. So you can set a link that says "before this node, send #halt to Halt". But they are much more powerful: -> selector -> metaObject -> what to pass as parameters -> a condition -> is it #before, #after, #around? can be set as wanted. So after some thinking one realizes that this subsumes -> BreakPoints -> MethodWrappers -> CLOS style MOP where the class is the meta object -> AOP style crosscutting. (Aspects is just a pattern on top) -> â¦. e.g. Code Coverage: ------------------------------ - add a #markExecuted to RBProgramNode. - install this link an all the nodes that you are interested in: link := GPLink new metaObject: #node; selector: #markExecuted. (markExecuted can even deinstall the link) CLOS style MOP ---------------------- (for sends:) link := GPLink new metaObject: #class; selector: #send:to:with:; arguments: #(selector receiver arguments) control: #instead. Add this link to all send nodes in your class after the compiler compiles something, and your class hierarchy has a CLOS style where every send is indirected through a method in the class side. Javassist (or Bytesurgeon) style "execute this code before this operation" ------------------------ link := GPLink new metaObject: [:object | object color = Color red ifTrue: [Beeper beep]]; selector: #value:;  arguments: #(object). The nice thing is that these links are highly dynamic: they can be installed quickly, remove themselves, propagate themselves over the AST at runtime even. Marcus