Hi Andrew, Le 17/11/2017 à 12:26, Prof. Andrew P. Black a écrit :
On 17 Nov 2017, at 14:10 , Thierry Goubier <thierry.goubier@gmail.com> wrote:
there is an 'E O F' token generated by SmaCC; I haven't tried to use it in a parser yet.
I tried patching the tokenActions table to trap on this, but the token id for E O F is outside of the range of the table. The Python example that you pointed me to is a little different. It overrides scannerError, and explicitly adds a newline token if there is an error at the end of the file. It doesnât actually use the E O F token, but it is probably a pattern that I can steal.
In all honesty, I wasn't thinking about that, but instead to be able to write '<eof>' in the grammar itself to terminate statements. The Python approach is necessary because you may have to emit additional dedent tokens at the end of a file (this is a typical issue of those meaningfull identation whitespace languages: an idea used in the very beginning of programming languages, then considered harmfull, then coming back up again...).
In the meantime, I made the final StatementSeparator (<newline> or ";") optional in all the productions. The grammar is a bit ugly, but the parser is cleaner.
Which is the cleanest way to do it (at least, like that, you have a documented way around that instead of carrying around a grammar + hacks in the scanner)(*)
I also gave up trying to eliminate intermediate parseTree nodes. Instead, I eliminated intermediate productions form the grammar. This makes the grammar more ugly (it has several repetitions where I inlined the intermediate productions), but the tree construction is a lot more straightforward.
Sorry for having been unable to answer your questions on that :( I'm happy to learn you've found a way around it. Thierry (*) Which is still way better than a hand-written, recursive descent parser where any line can hide a hack...
Andrew