On 14 Nov 2017, at 14:07 , Sven Van Caekenberghe <sven@stfx.eu> wrote:
I am curious though, why do you need it ?
I mean, I got away (and try hard) to only use 1-element peeking and managed to implement STON, NeoJSON, NeoCSV, Stomp, PostgresSQL, Redis, and other protocols.
A syntax that really requires multi element peeking is not so common.
I need it in my present situation because Iâm inside a framework that thinks that it owns the Stream (viz, inside SmaCC). I can write a custom action in the scanner for, in my case, <newline> tokens, that deals with indentation. Sometimes that action need to look ahead to do the right thing. For example, if the current line is completely blank, it canât be an indentation error, and does not re-set the current indentation to zero. I need to make such determinations from inside the newline method, and then reset the state of the stream so that the rest of the SmaCC-generated parser is undisturbed. In a framework like a PEG, you get a similar situation. A parser in a PEG should either succeed, and consume input, or fail, and leave the input where it was. This is true not just for simple parsers that read one character, but also for compound parsers that may perform arbitrary actions before they fail. This usually works by saving and resetting the stream state (but also maintaining a cache in the case of a Packrat Parser). In general, this is a modularity issue. Although globally, one may be able to limit lookahead, locally, itâs not so easy. An alternative to saving a restoring state is to allow arbitrary putback, as in OS6 streams. But I think that state-saving is more useful, even if it is less general. Andrew