Hey, I also created an indentation sensitive extension of PetitParser, I also included some examples including Markdown. If you load PetitParser, you get the Markdown example as well. There is a short introduction to the indentation: http://scg.unibe.ch/research/indentParsing Cheers Jan On Apr 13, 2015 5:24 PM, "H. Hirzel" <hannes.hirzel@gmail.com> wrote:

Are there any news on this?

--Hannes

On 12/28/14, Norbert Hartl <norbert@hartl.name> wrote:

...

I found markdown parsers in smalltalkhub and in the configuration browser.

http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/ <http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/>

and the one in the configuration browser

PPMarkdown

If I understand it correctly both are rooted into work Camillo did. The one in PharoExtras looks newer to me. So I assume the best would be to copy the ConfigurationOfPetitMarkdown from PharoExtras to MetacelloRepo and remove the ConfigurationOfPPMarkdown. Right? Or are there any objections?

Norbert

+1 PetitMarkdown (at least in the state I left it) had an incomplete and contorted grammar, translated from another implementation (the one used by pandoc, I think). I started something to handle indentation, but meh. Jan's extension of PetitParser is the way to go, and it should make the grammar way more readable. On 14 April 2015 at 11:43, Esteban Lorenzano <estebanlm@gmail.com> wrote:

how complete it is? we would really like to think on:

1) include petitparser in Pharo 2) allow the writing of class docs in MD format :)

of course, we are talking about the not-so-near future, but is an idea we are thinking about… :P

and in any case, I would like to use a good MD parser for my own stuff :)

Esteban

On 14 Apr 2015, at 10:42, Jan Kurš <kurs@iam.unibe.ch> wrote:

Hey,

I also created an indentation sensitive extension of PetitParser, I also included some examples including Markdown. If you load PetitParser, you get the Markdown example as well.

There is a short introduction to the indentation:

http://scg.unibe.ch/research/indentParsing

Cheers Jan On Apr 13, 2015 5:24 PM, "H. Hirzel" <hannes.hirzel@gmail.com> wrote:

...

Are there any news on this?

--Hannes

On 12/28/14, Norbert Hartl <norbert@hartl.name> wrote:

...

I found markdown parsers in smalltalkhub and in the configuration browser.

http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/ <http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/>

and the one in the configuration browser

PPMarkdown

If I understand it correctly both are rooted into work Camillo did. The one in PharoExtras looks newer to me. So I assume the best would be to copy the ConfigurationOfPetitMarkdown from PharoExtras to MetacelloRepo and remove the ConfigurationOfPPMarkdown. Right? Or are there any objections?

Norbert

-- Damien Pollet type less, do more [ | ] http://people.untyped.org/damien.pollet

On Tue, Apr 14, 2015 at 2:19 PM, Christophe Demarey < Christophe.Demarey@inria.fr> wrote:

Le 14 avr. 2015 à 11:43, Esteban Lorenzano a écrit :

...

how complete it is? we would really like to think on:

1) include petitparser in Pharo 2) allow the writing of class docs in MD format :)

I would prefer pillar for class / packages comments

I was quite surprised there are any MD defendants considering the pillar push. But since diversity is (often) a good think maybe having something like gt-inspector there would be cool where you can add this in whatever format you want. (And maybe one day someone will write pillar to morphic/whatever converter and it would be even cooler.) Peter

On Tue, Apr 14, 2015 at 9:21 AM, Jan Kurš <kurs@iam.unibe.ch> wrote:

If there is an interest to use PetitParser/CommonMark in Pharo, I can spend some time on it. I guess, I would be able to parse majority of the examples (I am not sure about all the corner cases).

That would be fantastic! In general, even regardless of the use case of using MD for in-Pharo documentation, the ability to parse Markdown is pretty fundamental to a programming language (behind only JSON, XML, and CSV).

On 14 Apr 2015, at 15:51, Norbert Hartl <norbert@hartl.name> wrote:

...

Am 14.04.2015 um 14:24 schrieb Peter Uhnák <i.uhnak@gmail.com <mailto:i.uhnak@gmail.com>>:

On Tue, Apr 14, 2015 at 2:19 PM, Christophe Demarey <Christophe.Demarey@inria.fr <mailto:Christophe.Demarey@inria.fr>> wrote:

Le 14 avr. 2015 à 11:43, Esteban Lorenzano a écrit :

...

how complete it is? we would really like to think on:

1) include petitparser in Pharo 2) allow the writing of class docs in MD format :)

I would prefer pillar for class / packages comments

I was quite surprised there are any MD defendants considering the pillar push. But since diversity is (often) a good think maybe having something like gt-inspector there would be cool where you can add this in whatever format you want. (And maybe one day someone will write pillar to morphic/whatever converter and it would be even cooler.)

It is a difficult topic. I agree with anyone that MarkDown is not a good format for parsing. Pillar is the right thing to do here. But there is one point of MarkDown that is hard to beat. A MarkDown text is always good to read, eben while writing. In something like a class comment it would be easy to use. What we don't want is to write system documentation in a format that you need to convert first before you can see the result. It is either having a wysiwyg editor for those things with pillar below or a simple format that can both.

my 2 cents,

that’s actually my main point too, yes. Esteban

norbert

Hi, MD is a cool exercise for PetitParser exactly because it is a terrible language. Jan did a great job at pushing parsing limits, but that does not mean that we have to jump on it to use it inside Pharo :). Actually, at this point in time, I really do not quite understand why we are still entertaining the idea of MD inside Pharo given all the effort for documenting using Pillar. In any case, as you might know, there already exists support for editing Pillar from inside Pharo ( http://www.humane-assessment.com/blog/writing-pillar-books-with-the-gtinspec...), and we will likely develop significantly this support in the IDE. Cheers, Doru On Tue, Apr 14, 2015 at 9:08 PM, stepharo <stepharo@free.fr> wrote:

I'm really pissed off. Because nearly nobody tried to write anything with pillar and you just talk about what you do not know. But thanks this is great to see that we are spending our energy for people who will never even try to use what we are doing. Superb!

No need to reply I will not read this thread anymore. And I should not even have because it was so obvious.

And yes I 'm REALLY pissed off. You should also say to cyril that what is is doing is hopeless because as soon as we will have a stupid markdown parser suddenly it will be great. what a shit.

So go and write your documentation in any format and do not expect me to look at it. I'm fed up about people that want doc on the web and when we spend time to migrate from latex to pillar to generate html and latex do not even consider what we did.

Stef

I would prefer pillar for class / packages comments

...

I was quite surprised there are any MD defendants considering the pillar push. But since diversity is (often) a good think maybe having something like gt-inspector there would be cool where you can add this in whatever format you want. (And maybe one day someone will write pillar to morphic/whatever converter and it would be even cooler.)

It is a difficult topic. I agree with anyone that MarkDown is not a good format for parsing. Pillar is the right thing to do here. But there is one point of MarkDown that is hard to beat. A MarkDown text is always good to read, eben while writing. In something like a class comment it would be easy to use. What we don't want is to write system documentation in a format that you need to convert first before you can see the result. It is either having a wysiwyg editor for those things with pillar below or a simple format that can both.

my 2 cents,

that’s actually my main point too, yes.

Esteban

norbert

-- www.tudorgirba.com "Every thing has its own flow"

On 14 Apr 2015, at 22:03, Dmitri Zagidulin <dmitri@zagidulin.net> wrote:

Whoa. I genuinely don't understand the fierce emotions here. Why do Pillar and Markdown have to be opposed? Why is wanting support for better parsing of MD (a commonly used format around the web, and useful in many projects) somehow an insult to the work done on Pillar?

(Incidentally, I don't quite understand why Pillar was created in the first place. Why have a slightly different and incompatible markdown format from what the rest of the world is using? But that's not the point. We have both, and it's easy to support both. What's the problem?)

Communication is always an issue, especially for newcomers. Pillar was chosen because it is our format, something we control and define, something we can extend anyway we want, not somebody else's format that we can only follow. Incidentally, Pillar precedes MarkDown by years. Furthermore, there is NO MarkDown standard, nor will there ever be. There are slight but annoying differences between the main implementors. It is super hard to write a parser for MD. Anyway, IMHO, the key thing is the underlying document model.

On Tue, Apr 14, 2015 at 3:08 PM, stepharo <stepharo@free.fr> wrote: I'm really pissed off. Because nearly nobody tried to write anything with pillar and you just talk about what you do not know. But thanks this is great to see that we are spending our energy for people who will never even try to use what we are doing. Superb!

No need to reply I will not read this thread anymore. And I should not even have because it was so obvious.

And yes I 'm REALLY pissed off. You should also say to cyril that what is is doing is hopeless because as soon as we will have a stupid markdown parser suddenly it will be great. what a shit.

So go and write your documentation in any format and do not expect me to look at it. I'm fed up about people that want doc on the web and when we spend time to migrate from latex to pillar to generate html and latex do not even consider what we did.

Stef

...

...

...

I would prefer pillar for class / packages comments

I was quite surprised there are any MD defendants considering the pillar push. But since diversity is (often) a good think maybe having something like gt-inspector there would be cool where you can add this in whatever format you want. (And maybe one day someone will write pillar to morphic/whatever converter and it would be even cooler.)

It is a difficult topic. I agree with anyone that MarkDown is not a good format for parsing. Pillar is the right thing to do here. But there is one point of MarkDown that is hard to beat. A MarkDown text is always good to read, eben while writing. In something like a class comment it would be easy to use. What we don't want is to write system documentation in a format that you need to convert first before you can see the result. It is either having a wysiwyg editor for those things with pillar below or a simple format that can both.

my 2 cents,

that’s actually my main point too, yes.

Esteban

...

norbert

On 14 April 2015 at 23:11, Dmitri Zagidulin <dmitri@zagidulin.net> wrote:

It is super hard to write a parser for MD.

...

What do you mean? Why?

The syntax depends on indentation and block layout of text. It's very hard to express using the usual grammar tools. In PetitMarkdown, before Jan added indent-aware parsing to PetitParser, I was thinking of doing multi-pass parsing, first recognizing block-level, then inline syntax. The closest thing to a standard is John Gruber's reference implementation, and that is a sequence of regular expression search-and-replace passes. Not exactly what you'd call a grammar :) Anyway… https://twitter.com/willmanduffy/status/588057382225346560 -- Damien Pollet type less, do more [ | ] http://people.untyped.org/damien.pollet

Some should cool down. I am using Pillar to write my (internal) docs. And I use markdown too. Both have their uses. Compared to things like RMarkdown, knitr, and RPubs in the R community, what we do have is passable in terms of output. We have a small community, we have to deal with that fact. I hate the figures not going where I want due to the LaTeX generation in Pillar. As for class comments, markdown would make more sense to use when putting files under source control on git and then stored in github because filetree generates README.md files for such comments. It makes for a nice reading there. In that respect, Pillar class comments will be more of a nuisance than a help. For generating web documentation, it is easier to provide markdown so that any web framework will be able to convert (e;g; Showdown.js and 2 lines of code). Furthermore, a lot of Pillar files embed non pillar markup for special things. For me, Pillar is for the books. And speciality applications that use the visitors to produce other stuff (among which Markdown). As for writing stuff, I do md and pillar files in Vim. So much for specific tools that weren't satisfying when writing. Too slow, too cumbersome, not enough power for editing (compared to Vim, yeah, nothing comes close in terms of speed once you know the tricks). Phil On Tue, Apr 14, 2015 at 10:03 PM, Dmitri Zagidulin <dmitri@zagidulin.net> wrote:

Whoa. I genuinely don't understand the fierce emotions here. Why do Pillar and Markdown have to be opposed? Why is wanting support for better parsing of MD (a commonly used format around the web, and useful in many projects) somehow an insult to the work done on Pillar?

(Incidentally, I don't quite understand why Pillar was created in the first place. Why have a slightly different and incompatible markdown format from what the rest of the world is using? But that's not the point. We have both, and it's easy to support both. What's the problem?)

On Tue, Apr 14, 2015 at 3:08 PM, stepharo <stepharo@free.fr> wrote:

...

I'm really pissed off. Because nearly nobody tried to write anything with pillar and you just talk about what you do not know. But thanks this is great to see that we are spending our energy for people who will never even try to use what we are doing. Superb!

No need to reply I will not read this thread anymore. And I should not even have because it was so obvious.

And yes I 'm REALLY pissed off. You should also say to cyril that what is is doing is hopeless because as soon as we will have a stupid markdown parser suddenly it will be great. what a shit.

So go and write your documentation in any format and do not expect me to look at it. I'm fed up about people that want doc on the web and when we spend time to migrate from latex to pillar to generate html and latex do not even consider what we did.

Stef

I would prefer pillar for class / packages comments

...

I was quite surprised there are any MD defendants considering the pillar push. But since diversity is (often) a good think maybe having something like gt-inspector there would be cool where you can add this in whatever format you want. (And maybe one day someone will write pillar to morphic/whatever converter and it would be even cooler.)

It is a difficult topic. I agree with anyone that MarkDown is not a good format for parsing. Pillar is the right thing to do here. But there is one point of MarkDown that is hard to beat. A MarkDown text is always good to read, eben while writing. In something like a class comment it would be easy to use. What we don't want is to write system documentation in a format that you need to convert first before you can see the result. It is either having a wysiwyg editor for those things with pillar below or a simple format that can both.

my 2 cents,

that’s actually my main point too, yes.

Esteban

norbert

-

Christophe Demarey <Christophe.Demarey@inria.fr> writes:

Le 14 avr. 2015 à 22:51, phil@highoctane.be a écrit :

...

I hate the figures not going where I want due to the LaTeX generation in Pillar.

There should be options to place it where you want. Maybe a pillar expert could answer?

this is not possible and, in my opinion, not desired. The following explains my reasons but requires some LaTeX knowledge to be understood. LaTeX does a really good job a placing figures at nice locations. For example, if your paragraph ends with 2 lines at the top of a page and you place a figure after this paragraph, LaTeX will make sure the figure appears at the top of a page and not after the 2 lines: this is because it would be ugly to have 2 lines, then a figure, then the rest of the text. In LaTeX, you have to make sure to always write things like "as can be seen in Figure 3" instead of "as can be seen in the figure below" because you can never be sure of the exact location of your figures. Please note that if in Pillar you don't specify any label you could reference from your text, then the figure will be placed by LaTeX at the exact location where you put it in Pillar. This is because, in this case, there is no way to write "as can be seen in Figure 3" when there is no label. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill

On 16/04/15 12:07, phil@highoctane.be wrote:

Customer opinion on the output: why are figures not where I expect them to be?

Most of the figures are HD screenshots. Moving them around sucks. I'll try the no label thing. But labels are useful. HTML export is not doing that. LaTeX is an interim tool to get the PDF output. Would it be possible to use Artefact instead?

Not without creating extremely ugly documentation, at the current state of Artefact layouting. Line breaking is primitive, support for other than base-14 fonts is missing, pngs are rendered, not embedded.

LaTeX toolchain is *huge*.

Yeah, it is. The best object model for documentation is that of Framemaker. Current machines should be fast enough to run something like that in Smalltalk instead of c now, 20 years later. Stephan

On Thu, Apr 16, 2015 at 4:40 AM, Damien Cassou <damien.cassou@gmail.com> wrote:

Please note that if in Pillar you don't specify any label you could reference from your text, then the figure will be placed by LaTeX at the exact location where you put it in Pillar. This is because, in this case, there is no way to write "as can be seen in Figure 3" when there is no label.

I tried that out in a recent UPBE chapter (here: https://github.com/SquareBracketAssociates/UpdatedPharoByExample/commit/f0b1... ). Didn't seem to work, the figure with no label was still placed arbitrarily by LaTeX (and not in the exact location). (And, actually, LaTeX placed the figure in the middle of a script block, which is really odd). Is this an issue of the version of Pillar that the book's build tools are using?

Dmitri Zagidulin <dmitri@zagidulin.net> writes:

Is this an issue of the version of Pillar that the book's build tools are using?

I thought I implemented that but you are right, it is not implemented. If you are lucky, Cyril will see this thread and fix it (hint, it's in PRLaTeXWriter>>writeEmbeddedPicture:). -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill

Le 14/4/15 11:43, Esteban Lorenzano a écrit :

how complete it is? we would really like to think on:

1) include petitparser in Pharo 2) allow the writing of class docs in MD format :)

seriously? With all the efforts and that damien and me are spending on pillar! Thanks a lot for your consideration

of course, we are talking about the not-so-near future, but is an idea we are thinking about… :P

and in any case, I would like to use a good MD parser for my own stuff :)

Esteban

...

On 14 Apr 2015, at 10:42, Jan Kurš <kurs@iam.unibe.ch <mailto:kurs@iam.unibe.ch>> wrote:

Hey,

I also created an indentation sensitive extension of PetitParser, I also included some examples including Markdown. If you load PetitParser, you get the Markdown example as well.

There is a short introduction to the indentation:

http://scg.unibe.ch/research/indentParsing

Cheers Jan

On Apr 13, 2015 5:24 PM, "H. Hirzel" <hannes.hirzel@gmail.com <mailto:hannes.hirzel@gmail.com>> wrote:

Are there any news on this?

--Hannes

On 12/28/14, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote: > I found markdown parsers in smalltalkhub and in the configuration browser. > > http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/ <http://smalltalkhub.com/#%21/%7EPharoExtras/PetitMarkdown/> > <http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/ <http://smalltalkhub.com/#%21/%7EPharoExtras/PetitMarkdown/>> > > and the one in the configuration browser > > PPMarkdown > > If I understand it correctly both are rooted into work Camillo did. The one > in PharoExtras looks newer to me. So I assume the best would be to copy the > ConfigurationOfPetitMarkdown from PharoExtras to MetacelloRepo and remove > the ConfigurationOfPPMarkdown. Right? Or are there any objections? > > Norbert

Hi Hannes, I actually did not do any changes to the grammar, I wrote the grammar from scratch. The reason I did so was to have a proof of concept of my PetitParser extension for parsing indentation-sensitive grammars such as Markdown or Python. The most simplest example you can do is to evaluate following: PPMiniCommonMark new parse: ' # Heading - list item - second list item ' As a result, you will get HTML fragment, that corresponds to one heading and list of items. All the supported CommonMark syntax is covered in the tests, so what you don't see in the test, has to be implemented. CommonMark is kind of beast to parse, I had to do one big extension to parse it. It is to introduce the indentation stack (as described in http://scg.unibe.ch/research/indentParsing). You can see the indentation stack manipulation in PPMiniCommonMark rules such as: #prefix, #quoteBegin, #quoteEnd, #itemBullet and #itemEnd. Cheers, Jan On 3 May 2015 at 08:46, H. Hirzel <hannes.hirzel@gmail.com> wrote:

Thank you Jan, for your work in enhancing PetitParser for Markdown. I think I got what you did by loading PetitParser in Pharo 4.0.

I found the examples PPMarkdownGrammarTest, category testing-documents.

I also read http://scg.unibe.ch/research/indentParsing.

Could you please elaborate a bit more about the changes you did to the grammar and give some more usage examples?

Regards

Hannes

On 4/14/15, Jan Kurš <kurs@iam.unibe.ch> wrote:

...

Hey,

I also created an indentation sensitive extension of PetitParser, I also included some examples including Markdown. If you load PetitParser, you get the Markdown example as well.

There is a short introduction to the indentation:

http://scg.unibe.ch/research/indentParsing

Cheers Jan On Apr 13, 2015 5:24 PM, "H. Hirzel" <hannes.hirzel@gmail.com> wrote:

...

Are there any news on this?

--Hannes

On 12/28/14, Norbert Hartl <norbert@hartl.name> wrote:

...

I found markdown parsers in smalltalkhub and in the configuration browser.

http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/ <http://smalltalkhub.com/#!/~PharoExtras/PetitMarkdown/>

and the one in the configuration browser

PPMarkdown

If I understand it correctly both are rooted into work Camillo did. The one in PharoExtras looks newer to me. So I assume the best would be to copy the ConfigurationOfPetitMarkdown from PharoExtras to MetacelloRepo and remove the ConfigurationOfPPMarkdown. Right? Or are there any objections?

Norbert