Confirmed in even a better way: given how convoluted and hacky is writing a full Python Parser, it is probably not even a Context Free Grammar. Let's agree on the fact that you'll be able to parse it using Context Sensitive Grammar (Type 1) for sure ... and if you're very lucky Context Free (Type 2) might be sufficient but PITA to write and maintain. Agreed? :-)
CU, Udo On 23.09.14 14:20, Thierry Goubier wrote:
2014-09-23 13:57 GMT+02:00 Udo Schneider <udo.schneider@homeaddress.de <mailto:udo.schneider@homeaddress.de>>:
> yeap noway I compare Regex with PettitParser. I will probably give a > PettitParser a try, because I try to parse Pharo syntax to Python, I > want now to parse Pharo classes to python class and that will be a > nightmare with regex, so time to give PettitParser a serious try. Without wanting to go too deep into theory... Parsing Pharo or Python would definitely need a Chromsky Type 2 language (Context free grammar). So you'd be out of luck parsing Smalltalk or Python code with Regular Expressions (Chromsky Type 3) only - except maybe for some very simple expressions.
Confirmed in even a better way: given how convoluted and hacky is writing a full Python Parser, it is probably not even a Context Free Grammar.
That is: the Python grammar is LALR which is a subset of Context Free Grammar (but already more than a type 3 Chomsky grammar), but the Lexer is a fair bit more than usual (dedent tokens come to mind; special handling of end of lines as well).
I think you need a fair bit of special magic in PetitParser to parse Python.
Luckily, Python is well documented in that regard.
Thierry
CU,
Udo
On 23.09.14 13:35, kilon alios wrote:
yeap noway I compare Regex with PettitParser. I will probably give a PettitParser a try, because I try to parse Pharo syntax to Python, I want now to parse Pharo classes to python class and that will be a nightmare with regex, so time to give PettitParser a serious try.
On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider <udo.schneider@homeaddress.de <mailto:udo.schneider@homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>> wrote:
> I have not used PettitParser yet, looks powerful but I find it a bit > weird in design. On the other hand regex is quite ugly and understanding > complex regex a pain. I normally encounter two issues with RegExps:
1) The syntax between different apps/libs/frameworks differs sligtly. Esp. for character classes or greediness. This drove me crazy more than one time. 2) Often enough I realize (while developing the RegExp) that I need something more capable than a Chomsky Type 3 parser (which RegExps are in a way). So I have to use "a bigger gun" - e.g. PetitParser.
CU,
Udo
On 23.09.14 10:15, kilon alios wrote:
it reminds a lot of Kent's Beck Smalltalk Practice Patterns where it removes all ifs and replaces them with regular unary messages . It is definitely an elegant way of coding making the code just flow.
I have not used PettitParser yet, looks powerful but I find it a bit weird in design. On the other hand regex is quite ugly and understanding complex regex a pain.
On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider <udo.schneider@homeaddress.de <mailto:udo.schneider@homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>> <mailto:udo.schneider@ <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>>> wrote:
> just as it is black magic for me now :D The nice thing about this approach is the fact that it "just" piggybacks the normal Smalltalk message sending. So you can step through it using the Debugger - it's Smalltalk all the way down.
I still remember my first shock when (having no formal background on parsing theory at that time) I saw the parsing tables generated by T-Gen. Of course it was Smalltalk ... but understanding those tables was a nightmare.
PetitParser is the gold standard here IMHO. It is able to parse arbitrary input (compared to my block expressions only). And you can still use the Debugger to step through the parsing process *and still understand what's going on!*
CU,
Udo
On 23.09.14 09:54, kilon alios wrote:
just as it is black magic for me now :D
At least I get the general feeling. I am new to parsing too, so far I have only played with regex parsing. Not the most smalltalkish way but it works well so far.
On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider
<udo.schneider@homeaddress.de <mailto:udo.schneider@homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>> <mailto:udo.schneider@ <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>> <mailto:udo.schneider@ <mailto:udo.schneider@> <mailto:udo.schneider@ <mailto:udo.schneider@>>__homea__d__dress.de <http://homead__dress.de> <http://homeaddress.de>
<mailto:udo.schneider@ <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>>>> wrote:
Hi Estaban,
I think the first time I saw this pattern was in ReStore on Dolphin Smalltalk. I didn't understand it's implementation back then. I assume that it's similar to what I described though. But having a Smalltalk block automagically creating the equivalent SQL SELECT expression was like black magic at that time :-)
CU,
Udo
On 23.09.14 04:15, Esteban A. Maringolo wrote:
Excellent article.
I think GLORP uses a similar technique to setup its expressions, and also have issues with #and:/#or: selectors due to inlining, so it uses AND:/#OR: instead.
Regards!
Esteban A. Maringolo
pd: Your blog and it's choosen topic made me remember http://use-the-index-luke.com/
2014-09-22 20:48 GMT-03:00 Udo Schneider
<udo.schneider@homeaddress.de <mailto:udo.schneider@homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>> <mailto:udo.schneider@ <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>> <mailto:udo.schneider@ <mailto:udo.schneider@> <mailto:udo.schneider@ <mailto:udo.schneider@>>__homea__d__dress.de <http://homead__dress.de> <http://homeaddress.de> <mailto:udo.schneider@ <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de> <mailto:udo.schneider@__homeaddress.de <mailto:udo.schneider@homeaddress.de>>>>>__:
All,
I just finished a blog entry. It shows how to use Smalltalk blocks as parsers/translators. E.g. translating a Block
[:customer | (customer joinDate year is: Date today year)]
into an SQL-like String
(YEAR(customers.joinDate) = 2014)
The SQL stuff is just an example - you can create nearly any output.
Check out http://readthesourceluke.__blo______gspot.de/2014/09/block-________translato... <http://blo____gspot.de/2014/09/block-______translators-parsing-magic.html>
<http://blo__gspot.de/2014/09/__block-____translators-parsing-__magic.html <http://blo__gspot.de/2014/09/block-____translators-parsing-magic.html>>
<http://blogspot.de/2014/09/____block-__translators-parsing-____magic.html <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html>
<http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>>
<http://readthesourceluke.__bl____ogspot.de/2014/09/block-______translators-p... <http://bl__ogspot.de/2014/09/block-____translators-parsing-magic.html>
<http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>
<http://readthesourceluke.__bl__ogspot.de/2014/09/block-____translators-parsi... <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>
<http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-m... <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>__>__>
Maybe that's old stuff for some of you - but I hope it's interesting for some at least :-)
Comments and feedback appreciated.
CU,
Udo