2014-09-23 15:57 GMT+02:00 Udo Schneider <udo.schneider@homeaddress.de>:
> Confirmed in even a better way: given how convoluted and hacky is
> writing a full Python Parser, it is probably not even a Context Free
> Grammar.
Let's agree on the fact that you'll be able to parse it using Context Sensitive Grammar (Type 1) for sure ... and if you're very lucky Context Free (Type 2) might be sufficient but PITA to write and maintain. Agreed? :-)

Agreed :)

Thierry

CU,

Udo



On 23.09.14 14:20, Thierry Goubier wrote:


2014-09-23 13:57 GMT+02:00 Udo Schneider <udo.schneider@homeaddress.de
<mailto:udo.schneider@homeaddress.de>>:


� � > yeap noway I compare Regex with PettitParser. I will probably give a
� � > PettitParser a try, because I try to parse Pharo syntax to Python, I
� � > want now to parse Pharo classes to python class and that will be a
� � > nightmare with regex, so time to give PettitParser a serious try.
� � Without wanting to go too deep into theory... Parsing Pharo or
� � Python would definitely need a Chromsky Type 2 language (Context
� � free grammar). So you'd be out of luck parsing Smalltalk or Python
� � code with Regular Expressions (Chromsky Type 3) only - except maybe
� � for some very simple expressions.


Confirmed in even a better way: given how convoluted and hacky is
writing a full Python Parser, it is probably not even a Context Free
Grammar.

That is: the Python grammar is LALR which is a subset of Context Free
Grammar (but already more than a type 3 Chomsky grammar), but the Lexer
is a fair bit more than usual (dedent tokens come to mind; special
handling of end of lines as well).

I think you need a fair bit of special magic in PetitParser to parse Python.

Luckily, Python is well documented in that regard.

Thierry


� � CU,

� � Udo


� � On 23.09.14 13:35, kilon alios wrote:

� � � � yeap noway I compare Regex with PettitParser. I will probably give a
� � � � PettitParser a try, because I try to parse Pharo syntax to Python, I
� � � � want now to parse Pharo classes to python class and that will be a
� � � � nightmare with regex, so time to give PettitParser a serious try.

� � � � On Tue, Sep 23, 2014 at 1:10 PM, Udo Schneider
� � � � <udo.schneider@homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>
� � � � <mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>
� � � � wrote:

� � � � � � �> I have not used PettitParser yet, looks powerful but I
� � � � find it a bit
� � � � � � �> weird in design. On the other hand regex is quite ugly
� � � � and understanding
� � � � � � �> complex regex a pain.
� � � � � � �I normally encounter two issues with RegExps:

� � � � � � �1) The syntax between different apps/libs/frameworks differs
� � � � � � �sligtly. Esp. for character classes or greediness. This
� � � � drove me
� � � � � � �crazy more than one time.
� � � � � � �2) Often enough I realize (while developing the RegExp)
� � � � that I need
� � � � � � �something more capable than a Chomsky Type 3 parser (which
� � � � RegExps
� � � � � � �are in a way). So I have to use "a bigger gun" - e.g.
� � � � PetitParser.

� � � � � � �CU,

� � � � � � �Udo


� � � � � � �On 23.09.14 10:15, kilon alios wrote:

� � � � � � � � �it reminds a lot of Kent's Beck Smalltalk Practice
� � � � Patterns where it
� � � � � � � � �removes all ifs and replaces them with regular unary
� � � � messages .
� � � � � � � � �It is
� � � � � � � � �definitely an elegant way of coding making the code
� � � � just flow.

� � � � � � � � �I have not used PettitParser yet, looks powerful but I
� � � � find it a bit
� � � � � � � � �weird in design. On the other hand regex is quite ugly and
� � � � � � � � �understanding
� � � � � � � � �complex regex a pain.

� � � � � � � � �On Tue, Sep 23, 2014 at 11:07 AM, Udo Schneider
� � � � � � � � �<udo.schneider@homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>
� � � � � � � � �<mailto:udo.schneider@
� � � � <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>>
� � � � � � � � �wrote:

� � � � � � � � � � � > just as it is black magic for me now :D
� � � � � � � � � � � The nice thing about this approach is the fact
� � � � that it "just"
� � � � � � � � � � � piggybacks the normal Smalltalk message sending.
� � � � So you can
� � � � � � � � �step
� � � � � � � � � � � through it using the Debugger - it's Smalltalk all
� � � � the way
� � � � � � � � �down.

� � � � � � � � � � � I still remember my first shock when (having no formal
� � � � � � � � �background on
� � � � � � � � � � � parsing theory at that time) I saw the parsing tables
� � � � � � � � �generated by
� � � � � � � � � � � T-Gen. Of course it was Smalltalk ... but
� � � � understanding
� � � � � � � � �those tables
� � � � � � � � � � � was a nightmare.

� � � � � � � � � � � PetitParser is the gold standard here IMHO. It is
� � � � able to parse
� � � � � � � � � � � arbitrary input (compared to my block expressions
� � � � only).
� � � � � � � � �And you can
� � � � � � � � � � � still use the Debugger to step through the parsing
� � � � process *and
� � � � � � � � � � � still understand what's going on!*

� � � � � � � � � � � CU,

� � � � � � � � � � � Udo

� � � � � � � � � � � On 23.09.14 09:54, kilon alios wrote:

� � � � � � � � � � � � � just as it is black magic for me now :D

� � � � � � � � � � � � � At least I get the general feeling. I am new
� � � � to parsing
� � � � � � � � �too, so
� � � � � � � � � � � � � far I
� � � � � � � � � � � � � have only played with regex parsing. Not the most
� � � � � � � � �smalltalkish
� � � � � � � � � � � � � way but
� � � � � � � � � � � � � it works well so far.

� � � � � � � � � � � � � On Tue, Sep 23, 2014 at 9:39 AM, Udo Schneider

� � � � � <udo.schneider@homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>
� � � � � � � � � � � � � <mailto:udo.schneider@
� � � � <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>
� � � � � � � � � � � � � <mailto:udo.schneider@ <mailto:udo.schneider@>
� � � � � � � � �<mailto:udo.schneider@
� � � � <mailto:udo.schneider@>>__homea__d__dress.de
� � � � <http://homead__dress.de> <http://homeaddress.de>



� � � � � � � � � � � � � <mailto:udo.schneider@
� � � � <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>>>
� � � � � � � � � � � � � wrote:

� � � � � � � � � � � � � � � �Hi Estaban,

� � � � � � � � � � � � � � � �I think the first time I saw this pattern
� � � � was in
� � � � � � � � �ReStore on
� � � � � � � � � � � � � Dolphin
� � � � � � � � � � � � � � � �Smalltalk. I didn't understand it's
� � � � implementation
� � � � � � � � �back then. I
� � � � � � � � � � � � � � � �assume that it's similar to what I described
� � � � � � � � �though. But
� � � � � � � � � � � � � having a
� � � � � � � � � � � � � � � �Smalltalk block automagically creating the
� � � � � � � � �equivalent SQL
� � � � � � � � � � � � � SELECT
� � � � � � � � � � � � � � � �expression was like black magic at that
� � � � time :-)

� � � � � � � � � � � � � � � �CU,

� � � � � � � � � � � � � � � �Udo




� � � � � � � � � � � � � � � �On 23.09.14 04:15, Esteban A. Maringolo
� � � � wrote:

� � � � � � � � � � � � � � � � � �Excellent article.

� � � � � � � � � � � � � � � � � �I think GLORP uses a similar technique to
� � � � � � � � �setup its
� � � � � � � � � � � � � expressions, and
� � � � � � � � � � � � � � � � � �also have issues with #and:/#or:
� � � � selectors due to
� � � � � � � � � � � � � inlining, so
� � � � � � � � � � � � � � � � � �it uses
� � � � � � � � � � � � � � � � � �AND:/#OR: instead.

� � � � � � � � � � � � � � � � � �Regards!

� � � � � � � � � � � � � � � � � �Esteban A. Maringolo

� � � � � � � � � � � � � � � � � �pd: Your blog and it's choosen topic
� � � � made me
� � � � � � � � �remember
� � � � http://use-the-index-luke.com/

� � � � � � � � � � � � � � � � � �2014-09-22 20:48 GMT-03:00 Udo Schneider

� � � � <udo.schneider@homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>
� � � � � � � � � � � � � <mailto:udo.schneider@
� � � � <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>
� � � � � � � � � � � � � � � � � �<mailto:udo.schneider@
� � � � <mailto:udo.schneider@>
� � � � � � � � �<mailto:udo.schneider@
� � � � <mailto:udo.schneider@>>__homea__d__dress.de
� � � � <http://homead__dress.de> <http://homeaddress.de>
� � � � � � � � � � � � � <mailto:udo.schneider@
� � � � <mailto:udo.schneider@>__homead__dress.de <http://homeaddress.de>
� � � � � � � � �<mailto:udo.schneider@__homeaddress.de
� � � � <mailto:udo.schneider@homeaddress.de>>>>>__:


� � � � � � � � � � � � � � � � � � � �All,

� � � � � � � � � � � � � � � � � � � �I just finished a blog entry. It
� � � � shows how
� � � � � � � � �to use
� � � � � � � � � � � � � Smalltalk
� � � � � � � � � � � � � � � � � � � �blocks as parsers/translators. E.g.
� � � � � � � � �translating a Block

� � � � � � � � � � � � � � � � � � � � � � � � �[:customer | (customer
� � � � joinDate
� � � � � � � � �year is: Date
� � � � � � � � � � � � � � � � � � � �today year)]

� � � � � � � � � � � � � � � � � � � �into an SQL-like String


� � � � (YEAR(customers.joinDate) = 2014)

� � � � � � � � � � � � � � � � � � � �The SQL stuff is just an example
� � � � - you can
� � � � � � � � �create
� � � � � � � � � � � � � nearly any
� � � � � � � � � � � � � � � � � � � �output.

� � � � � � � � � � � � � � � � � � � �Check out
� � � � http://readthesourceluke.__blo______gspot.de/2014/09/block-________translators-parsing-magic.__html
� � � � <http://blo____gspot.de/2014/09/block-______translators-parsing-magic.html>

� � � � <http://blo__gspot.de/2014/09/__block-____translators-parsing-__magic.html
� � � � <http://blo__gspot.de/2014/09/block-____translators-parsing-magic.html>>


� � � � <http://blogspot.de/2014/09/____block-__translators-parsing-____magic.html
� � � � <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html>

� � � � <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html
� � � � <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>>



� � � � <http://readthesourceluke.__bl____ogspot.de/2014/09/block-______translators-parsing-magic.html
� � � � <http://bl__ogspot.de/2014/09/block-____translators-parsing-magic.html>

� � � � <http://blogspot.de/2014/09/__block-__translators-parsing-__magic.html
� � � � <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>>


� � � � <http://readthesourceluke.__bl__ogspot.de/2014/09/block-____translators-parsing-magic.html
� � � � <http://blogspot.de/2014/09/block-__translators-parsing-magic.html>

� � � � <http://readthesourceluke.__blogspot.de/2014/09/block-__translators-parsing-magic.html
� � � � <http://readthesourceluke.blogspot.de/2014/09/block-translators-parsing-magic.html>__>__>__>

� � � � � � � � � � � � � � � � � � � �Maybe that's old stuff for some
� � � � of you -
� � � � � � � � �but I hope
� � � � � � � � � � � � � it's
� � � � � � � � � � � � � � � � � � � �interesting for some at least :-)

� � � � � � � � � � � � � � � � � � � �Comments and feedback appreciated.

� � � � � � � � � � � � � � � � � � � �CU,

� � � � � � � � � � � � � � � � � � � �Udo