Tx cyril For class comment I image that we want ! - - *url* and bold [[[ ]]] Did I miss something. Stef On Fri, Aug 11, 2017 at 6:57 PM, Cyril Ferlicot D. <cyril.ferlicot@gmail.com> wrote:

Le 11/08/2017 à 18:52, Peter Uhnak a écrit :

...

Hi,

I would like to propose including Pillar in the Pharo image by default.

My reasoning:

Since we are moving to git, and most people will use github, gitlab, and the likes, it is expected to include a README.md file (or possibly more extensive documentation) alongside the code.

Which means that people will (and are) writing the README mostly by hand, which of course is problematic, as e.g. code snippets can get deprecated, screenshots become outdated, etc.

As Pillar tries to address these problems, it would make sense to me to include Pillar in the image by default, as anyone using git (which eventually should be everyone) will most likely benefit from writing their documentation in Pillar. Similarly using Pillar would open an avenue to provide the documentation in-image, e.g. one exporter for html/markdown, and another one for Pharo's Help system.

I could, of course, install Pillar every time, but considering thats extra effort and in the extra time I can fix the issues by hand, I don't have such an incentive to use Pillar for this.

Questions & Problems:

I don't know by how much would pillar increase the image size. Perhaps there could be (a) "lightweight Pillar" (that would include just pillar & markdown exporter), or (b) we would have different images for different uses.

By different images I mean something along the lines of a) developer image - meant to be directly used by developers to create their software b) production image - as a foundation for running systems / users

Does this make sense?

Peter

Hi,

I know that Stephane want to includes a light version of Pillar in the image.

Currently he is working on a way to remove Magritte dependency from Pillar. Another step would be to get a minimal parser not relying on PetitParser. This parser would parse only a subset of the Pillar syntax that can be used for writing class comments. If this subset is defined (I we have the list of all Document Items we want to understand), I can give a try to make this light parse.

-- Cyril Ferlicot https://ferlicot.fr

http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

Le 11/08/2017 à 19:12, Cyril Ferlicot D. a écrit :

We might want to add some syntax only for class comment and not Pillar itself. For example to be able to reference a class in Pharo and get a link to the class during the rendering.

I think that we might also want the formats (bold, etc) and the ordered list.

I just saw that you said bold after sending… -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

Le 11/08/2017 à 19:19, Stephane Ducasse a écrit :

Cyril

the ** can be used to point to another class for example acting as a ref.

Stef

I though it would be kept for real links to web urls. -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques Prévert 01, 59650 Villeneuve d'ascq France

On 11 Aug 2017, at 21:10, Esteban Lorenzano <estebanlm@gmail.com> wrote:

hi,

...

On 11 Aug 2017, at 18:57, Cyril Ferlicot D. <cyril.ferlicot@gmail.com <mailto:cyril.ferlicot@gmail.com>> wrote:

Another step would be to get a minimal parser not relying on PetitParser.

Let’s think differently: why not to include a tiny PetitParser? Then we can think on:

- pillar sintax (better than just a restricted version) *syntax*… sorry for spanish intromission ;)

btw… not just outside the image (as readme). There was a prototype about rich text (using pillar, of course) for class comments. Esteban

- simplify other “small parsers” that are already on the image. - we provide a tool to o cool stuff (instead relying as always in regexp, etc.)

cheers, Esteban

If we are considering this, might we take the hard decision to actually support proper markdown (or even kramdown if we need more control). Pillar is a weird variant that isn't like everyone else - so i always have to think about how to do formatting. I know it's not nice, and lots of things are done in pillar - but could we convert them to be like everyone else? The readme.md that Peter is talking about is gfm markdown.' Tim Sent from my iPhone

On 11 Aug 2017, at 20:24, Peter Uhnak <i.uhnak@gmail.com> wrote:

...

On Fri, Aug 11, 2017 at 09:14:01PM +0200, Esteban Lorenzano wrote:

...

On 11 Aug 2017, at 21:10, Esteban Lorenzano <estebanlm@gmail.com> wrote:

hi,

...

On 11 Aug 2017, at 18:57, Cyril Ferlicot D. <cyril.ferlicot@gmail.com <mailto:cyril.ferlicot@gmail.com>> wrote:

Another step would be to get a minimal parser not relying on PetitParser.

Let’s think differently: why not to include a tiny PetitParser? Then we can think on:

- pillar sintax (better than just a restricted version) *syntax*… sorry for spanish intromission ;)

btw… not just outside the image (as readme).

Yes, one of the points I want is to have the readme available in Pharo's Help Browser with rich content.

Peter

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well). However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

The readme.md that Peter is talking about is gfm markdown

Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.) Peter

Hi tim I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing. And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer. Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before. [[[test 2+3

...

...

5 ]]]

and being able to verify that the doc is in sync. Stef On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown

Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

Hi Tim, The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books. The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text. Cheers, Doru

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown

Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com “Live like you mean it."

Hi,

On Aug 14, 2017, at 3:51 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:

again, I think this is a discussion for pharo-dev. Please keep it there (is good discussion, btw ;) ).

What about my proposal of including a tiny PetitParser? (it would be “InfimeParser” :P)

I am for it, but there will be some work to repackage PetitParser2. Also, I notice that the French lessons start to pay off :) Cheers, Doru

Esteban

...

On 14 Aug 2017, at 11:10, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi Tim,

The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books.

The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text.

Cheers, Doru

...

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown

Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com

“Live like you mean it."

-- www.tudorgirba.com www.feenk.com "To lead is not to demand things, it is to make them happen."

Jimmie et al. nicely reasoned arguments - and Doru's point about controlling the syntax is an interesting one that I hadn’t thought about. Personally, I find having too many similar syntax’s confusing - contributing to things is hard enough - having to remember that its !! Instead of ## and “” instead of ** is just frustrating for me. My vote would be what Peter suggested - use http://spec.commonmark.org/0.28/ <http://spec.commonmark.org/0.28/> and put our Pillar extensions back on top for things that Stef was mentioning. (I think that’s what I’ve understood gfm markdown is). Sure, maybe we were first with Pillar, but for me, lots of programming is in other languages, and I use Smalltalk where I can, and a hybrid of multiple languages and projects is often the reality - so a lowest common denominator of Markdown is just easier. The fact that we are quite close to what our colleagues in other languages use (regardless of what Python has chosen), is quite interesting. That said, if the community wants to stick to its gun’s thats fine - I will probably still investigate how to use Commonmark for myself, and will still contribute to Pillar docs where I can (and curse history) - but I think we are long better off trying to join emerging standards where we can particularly if they aren’t our core language thing. And it just makes it less frictionless for ourselves and newcomers. Of course, if we were to move, we would need to translate a lot of quality docs to a new format - but I would be up for contributing to that if that was a deciding factor. Tim

On 14 Aug 2017, at 16:41, Jimmie Houchin <jlhouchin@gmail.com> wrote:

TL;DR

Main points: Their is no universally accepted markup language. Other communities use their own markup and tools and their markup and tools choice is not determine by other communities decisions. We need a language and tool chain that we can control and maintain which accomplishes our goals. Our language and tools already exist and have existed for longer than most of the other markup languages. Of course they existed in various different forms over the years and have evolved into what they currently are. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects.

I just want to comment on the fact that there is no universal markup language that every development community has settled upon. Making Markdown or some variant the markup language for Pharo only aligns us with a certain part of the development community. Even Markdown is not unified as is evident by the discussion.

It is true that GitHub uses their variant of Markdown. And as long as we use GitHub we will need to use their variant for documents that reside on their system.

However as a significant counter example to lets all use gfm Markdown, is the Python community and their documentation.

https://docs.python.org/devguide/documenting.html """ 7. Documenting Python The Python language has a substantial body of documentation, much of it contributed by various authors. The markup used for the Python documentation is reStructuredText, developed by the docutils project, amended by custom directives and using a toolset named Sphinx to post-process the HTML output.

This document describes the style guide for our documentation as well as the custom reStructuredText markup introduced by Sphinx to support Python documentation and how it should be used.

The documentation in HTML, PDF or EPUB format is generated from text files written using the reStructuredText format and contained in the CPython Git repository. """

So the Python community uses their own markup language and their own tool chain. So therefore, it is not wrong for a community to go their own way, for their own reasons. Even within the conventional file based languages such as Python.

The fact that you have tools such as Pandoc, suggest that there is not true uniformity or unanimity among developers as to the best markup language or tool chain.

I believe that a language that we can control and maintain is better than adopting some other foreign markup language that is neither better, nor unanimously used by all. That would ultimately potentially require extensions to accomplish our goals. Then we would be maintaining someone else's language with our extensions that may or may not be accepted by the larger community. This does not prevent but rather encourages fragmentation of the existing Markdown.

Regardless, Pillar markup already exists. The tools in Pharo already understand it. Should someone desire to use Pharo which is far more different from Python/Ruby/etc. than Pillar syntax is from Markdown. Then it should be worth their effort to learn our tools.

Pillar markup is older than Markdown, etc. It's history begins in SmallWiki. It isn't as if we jumped up and decided to create something new in order to be different. Our markup and tools are older. They (and others) are the ones that decided to do their own markup and tools. And it is okay that they did so. Nothing wrong with doing so. Every community has the right to what they believe is best for their community. Even if other communities disagree.

The ability to control and maintain is highly valuable. We can understand what our requirements are for today. But we can not know what the requirements are in the future. Nor can we know that Markdown or whomever will have such requirements when they appear. It is easy to see in the beginning with the Squeak Wiki syntax to the now Pillar syntax, changes that have been made to accommodate new requirements as they became known. We need to maintain that ability. Sure we would reserve the right to do so in any language we adopt. But the then current standard bearer of said language would determine whether what we do is acceptable and incorporate or whether we are then in fact adding to their fragmentation. Pillar is ours. There is not fragmentation when we evolve.

However, since we have made a decision to use GitHub and GitHub has made a decision to use their own GFM Markdown. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects. This way we can use our own tools and markup language to accomplish whatever we want to accomplish. Including generating a Readme.md for our GitHub projects.

Just wanted to toss out this simple opinion and facts about the situation.

Jimmie

On 08/14/2017 04:10 AM, Tudor Girba wrote:

...

Hi Tim,

The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books.

The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text.

Cheers, Doru

...

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com

“Live like you mean it."

Hi, While I appreciate the historical perspective, I continue to be with Tim on this (and I consider myself part of the Pharo Community). I have also personal preferences for other light markup languages (like txt2tags[1] and dokuwiki's[2]), but I will stick with Pandoc's Markdown, because is support everything Stephan or any professional author wants to do and in fact you can create full books with it, including references, tables, images, and bibliographic references (which are not supported by Pillar AFAIK), but also because is an emerging standard beyond the programmers community, including academicians and researchers with the Scholarly Markdown[3] initiative and with possibilities to import and export from/to several markup languages[4]. [1] http://txt2tags.org/ [2] https://www.dokuwiki.org/wiki:syntax [3] http://scholmd.org/ [4] http://pandoc.org/ I don't share the vision of particular communities choosing particular markup languages as default, because, if you already payed the price of learning that particular language/environment, you are willing to pay the price with whatever that community choose in other fronts like DVCS or markup languages. Python community supports reST, but also markdown and several other markup languages and Jupyter notebooks[5] user Markdown by default. In fact, the Iceberg Git support shows the increasing concern to bridge the Pharo world with the stuff you already know and I think that a similar approach should be taken in the documentation/markup front, even if this implies breaking compatibility with the canonical Smalltalk way (TM) (I really like that critical approach from Pharo to the past). [5] http://jupyter.org/ That being said, I don't think that should be exclusively one way or another. We can have Pillar and (Pandoc's) Markdown, if the community doesn't reach and agreement on only one. I plan to explore the Brick editor once I have time and will try to add Pandoc's Markdown support. Unfortunately, in the past I have not had many luck testing and giving feedback on Moose alpha releases of tools and my questions/comments on them remain largely unanswered or simply ignored for long time (or just forever), so my priority on testing these tools have just decreased, but once Brick editor become more well supported, Pandoc's Markdown support for it will be in my target and concerns. Cheers, Offray On 14/08/17 12:48, Jimmie Houchin wrote:

Thank Tim,

My primary reason to submit the message was not to necessarily persuade you per se. But to provide something historical for the mailing list as this can be a recurring subject. Why use Pillar markup instead of ???(insert personal favorite).

If Pharo were to decide on a different markup language. The question would still be which one, why and then how do we proceed. Then our extensions may not be accepted by the greater body of users of said markup. We would still be contributing to the fragmentation of markup. As far as familiarity, I don't know. And familiarity with what. I do not find that reStructuredText to be similar to Markdown.

It would stop people from asking why we aren't using Markdown. But it wouldn't prevent others. Why aren't we using GFM Markdown, or Kramdown or Commonmark or ...? Why aren't we using YAML or reST or AsciiDoc or insert latest greatest creation markup or current flavor of the moment. Which is why I wanted to point out that there is no consensus among users of markup languages. At least I do not see one. Nor do I believe that we have seen the end of creation of new markup languages.

I understand the difficulty, though I do not suffer from it as I have not mastered any of those other languages. I have been using Squeak/Pharo for a long time. I struggle when I look at those other languages. To me they are the foreign ones.

And I do not see these emerging standards you refer to. When we see Python, Ruby, Perl, C++, various projects, etc. communities having consensus on a common markup for documentation. Then I see an emerging standard. Until then it seems to possibly be an emerging standard for a particular markup language which is among the set of markup languages.

If we were the only language and development environment doing our own thing. Then we might have a very good reason to talk. But we are not. Python with its enormous community does its own thing. I don't know that other languages have a consensus for markup for documentation except for Python and Pharo.

While writing this email I went and discovered that even GitHub is not dogmatic about the subject. Obviously they have an opinion. But they permit multiple markup languages. Quite possibly someone could write a Pillar tool for GitHub to use and then we could just submit Readme.pillar for our projects. :)

https://github.com/github/markup

Shows that GitHub allows for .markdown, .mdown, .mkdn, .md; .textile; .rdoc; .org; .creole; .mediawiki, .wiki; .rst; .asciidoc, .adoc, .asc; .pod. So it seems that there are many communities on GitHub who prefer their own markup and tools.

We could possibly write the Pillar tool for GitHub or an exporter to the preferred markup language of the above.

This author provides arguments for using reStructuredText over Markdown for GitHub documents. Citing deficiencies in Markdown and expressiveness in reST.

https://gist.github.com/dupuy/1855764

So again. I am just not seeing a consensus around any emerging standard for "the markup language".

At the same time if you are desirous of writing in Commonmark in your text editor. Can you not write conversion software that goes from Commonmark to Pillar? Thus, meeting want you want and what we require? If you were to do so, you would definitely have a good understanding of the differences in philosophy and capabilities of each. Just a thought.

Any way, thanks for engaging in the conversation. I wasn't targeting you personally, but rather the topic. You are not alone in your thinking. The Pharo community is not alone in its thinking either.

Thanks.

Jimmie

On 08/14/2017 11:34 AM, Tim Mackinnon wrote:

...

Jimmie et al. nicely reasoned arguments - and Doru's point about controlling the syntax is an interesting one that I hadn’t thought about.

Personally, I find having too many similar syntax’s confusing - contributing to things is hard enough - having to remember that its !! Instead of ## and “” instead of ** is just frustrating for me.

My vote would be what Peter suggested - use http://spec.commonmark.org/0.28/ and put our Pillar extensions back on top for things that Stef was mentioning. (I think that’s what I’ve understood gfm markdown is).

Sure, maybe we were first with Pillar, but for me, lots of programming is in other languages, and I use Smalltalk where I can, and a hybrid of multiple languages and projects is often the reality - so a lowest common denominator of Markdown is just easier. The fact that we are quite close to what our colleagues in other languages use (regardless of what Python has chosen), is quite interesting.

That said, if the community wants to stick to its gun’s thats fine - I will probably still investigate how to use Commonmark for myself, and will still contribute to Pillar docs where I can (and curse history) - but I think we are long better off trying to join emerging standards where we can particularly if they aren’t our core language thing. And it just makes it less frictionless for ourselves and newcomers.

Of course, if we were to move, we would need to translate a lot of quality docs to a new format - but I would be up for contributing to that if that was a deciding factor.

Tim

...

On 14 Aug 2017, at 16:41, Jimmie Houchin <jlhouchin@gmail.com <mailto:jlhouchin@gmail.com>> wrote:

TL;DR

Main points: Their is no universally accepted markup language. Other communities use their own markup and tools and their markup and tools choice is not determine by other communities decisions. We need a language and tool chain that we can control and maintain which accomplishes our goals. Our language and tools already exist and have existed for longer than most of the other markup languages. Of course they existed in various different forms over the years and have evolved into what they currently are. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects.

I just want to comment on the fact that there is no universal markup language that every development community has settled upon. Making Markdown or some variant the markup language for Pharo only aligns us with a certain part of the development community. Even Markdown is not unified as is evident by the discussion.

It is true that GitHub uses their variant of Markdown. And as long as we use GitHub we will need to use their variant for documents that reside on their system.

However as a significant counter example to lets all use gfm Markdown, is the Python community and their documentation.

https://docs.python.org/devguide/documenting.html """ 7. Documenting Python The Python language has a substantial body of documentation, much of it contributed by various authors. The markup used for the Python documentation is reStructuredText, developed by the docutils project, amended by custom directives and using a toolset named Sphinx to post-process the HTML output.

This document describes the style guide for our documentation as well as the custom reStructuredText markup introduced by Sphinx to support Python documentation and how it should be used.

The documentation in HTML, PDF or EPUB format is generated from text files written using the reStructuredText format and contained in the CPython Git repository. """

So the Python community uses their own markup language and their own tool chain. So therefore, it is not wrong for a community to go their own way, for their own reasons. Even within the conventional file based languages such as Python.

The fact that you have tools such as Pandoc, suggest that there is not true uniformity or unanimity among developers as to the best markup language or tool chain.

I believe that a language that we can control and maintain is better than adopting some other foreign markup language that is neither better, nor unanimously used by all. That would ultimately potentially require extensions to accomplish our goals. Then we would be maintaining someone else's language with our extensions that may or may not be accepted by the larger community. This does not prevent but rather encourages fragmentation of the existing Markdown.

Regardless, Pillar markup already exists. The tools in Pharo already understand it. Should someone desire to use Pharo which is far more different from Python/Ruby/etc. than Pillar syntax is from Markdown. Then it should be worth their effort to learn our tools.

Pillar markup is older than Markdown, etc. It's history begins in SmallWiki. It isn't as if we jumped up and decided to create something new in order to be different. Our markup and tools are older. They (and others) are the ones that decided to do their own markup and tools. And it is okay that they did so. Nothing wrong with doing so. Every community has the right to what they believe is best for their community. Even if other communities disagree.

The ability to control and maintain is highly valuable. We can understand what our requirements are for today. But we can not know what the requirements are in the future. Nor can we know that Markdown or whomever will have such requirements when they appear. It is easy to see in the beginning with the Squeak Wiki syntax to the now Pillar syntax, changes that have been made to accommodate new requirements as they became known. We need to maintain that ability. Sure we would reserve the right to do so in any language we adopt. But the then current standard bearer of said language would determine whether what we do is acceptable and incorporate or whether we are then in fact adding to their fragmentation. Pillar is ours. There is not fragmentation when we evolve.

However, since we have made a decision to use GitHub and GitHub has made a decision to use their own GFM Markdown. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects. This way we can use our own tools and markup language to accomplish whatever we want to accomplish. Including generating a Readme.md for our GitHub projects.

Just wanted to toss out this simple opinion and facts about the situation.

Jimmie

On 08/14/2017 04:10 AM, Tudor Girba wrote:

...

Hi Tim,

The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books.

The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text.

Cheers, Doru

...

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com

“Live like you mean it."

P.S. +1 for including pillar in the 6.2 image , then start working on the document tree and see how it can be adapted. On 8/15/17, H. Hirzel <hannes.hirzel@gmail.com> wrote:

Offray,

thanks for the nice write-up about the general usefulness of Markdown for writing papers.

Stephen D. wrote yesterday in a terse way

"We can change the syntax or propose an alternate one as soon as it uses the same internal structure.

Stef"

This means that Pillars tagging system may be changed to a more generally known tagging system later.

I assume with 'internal structure' he refers to the AST/Document Object Model of Pillar which would need to be aligned with one of another markup system.

Maybe the gap is not all that large.

Personally I think as well that if Pharo could also be used as a markdown (commonmark) editor with its tool-chain to generate different types of documents would expand the area of application considerably.

--Hannes

On 8/15/17, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:

...

Hi,

While I appreciate the historical perspective, I continue to be with Tim on this (and I consider myself part of the Pharo Community). I have also personal preferences for other light markup languages (like txt2tags[1] and dokuwiki's[2]), but I will stick with Pandoc's Markdown, because is support everything Stephan or any professional author wants to do and in fact you can create full books with it, including references, tables, images, and bibliographic references (which are not supported by Pillar AFAIK), but also because is an emerging standard beyond the programmers community, including academicians and researchers with the Scholarly Markdown[3] initiative and with possibilities to import and export from/to several markup languages[4].

[1] http://txt2tags.org/ [2] https://www.dokuwiki.org/wiki:syntax [3] http://scholmd.org/ [4] http://pandoc.org/

I don't share the vision of particular communities choosing particular markup languages as default, because, if you already payed the price of learning that particular language/environment, you are willing to pay the price with whatever that community choose in other fronts like DVCS or markup languages. Python community supports reST, but also markdown and several other markup languages and Jupyter notebooks[5] user Markdown by default. In fact, the Iceberg Git support shows the increasing concern to bridge the Pharo world with the stuff you already know and I think that a similar approach should be taken in the documentation/markup front, even if this implies breaking compatibility with the canonical Smalltalk way (TM) (I really like that critical approach from Pharo to the past).

[5] http://jupyter.org/

That being said, I don't think that should be exclusively one way or another. We can have Pillar and (Pandoc's) Markdown, if the community doesn't reach and agreement on only one.

I plan to explore the Brick editor once I have time and will try to add Pandoc's Markdown support. Unfortunately, in the past I have not had many luck testing and giving feedback on Moose alpha releases of tools and my questions/comments on them remain largely unanswered or simply ignored for long time (or just forever), so my priority on testing these tools have just decreased, but once Brick editor become more well supported, Pandoc's Markdown support for it will be in my target and concerns.

Cheers,

Offray

On 14/08/17 12:48, Jimmie Houchin wrote:

...

Thank Tim,

My primary reason to submit the message was not to necessarily persuade you per se. But to provide something historical for the mailing list as this can be a recurring subject. Why use Pillar markup instead of ???(insert personal favorite).

If Pharo were to decide on a different markup language. The question would still be which one, why and then how do we proceed. Then our extensions may not be accepted by the greater body of users of said markup. We would still be contributing to the fragmentation of markup. As far as familiarity, I don't know. And familiarity with what. I do not find that reStructuredText to be similar to Markdown.

It would stop people from asking why we aren't using Markdown. But it wouldn't prevent others. Why aren't we using GFM Markdown, or Kramdown or Commonmark or ...? Why aren't we using YAML or reST or AsciiDoc or insert latest greatest creation markup or current flavor of the moment. Which is why I wanted to point out that there is no consensus among users of markup languages. At least I do not see one. Nor do I believe that we have seen the end of creation of new markup languages.

I understand the difficulty, though I do not suffer from it as I have not mastered any of those other languages. I have been using Squeak/Pharo for a long time. I struggle when I look at those other languages. To me they are the foreign ones.

And I do not see these emerging standards you refer to. When we see Python, Ruby, Perl, C++, various projects, etc. communities having consensus on a common markup for documentation. Then I see an emerging standard. Until then it seems to possibly be an emerging standard for a particular markup language which is among the set of markup languages.

If we were the only language and development environment doing our own thing. Then we might have a very good reason to talk. But we are not. Python with its enormous community does its own thing. I don't know that other languages have a consensus for markup for documentation except for Python and Pharo.

While writing this email I went and discovered that even GitHub is not dogmatic about the subject. Obviously they have an opinion. But they permit multiple markup languages. Quite possibly someone could write a Pillar tool for GitHub to use and then we could just submit Readme.pillar for our projects. :)

https://github.com/github/markup

Shows that GitHub allows for .markdown, .mdown, .mkdn, .md; .textile; .rdoc; .org; .creole; .mediawiki, .wiki; .rst; .asciidoc, .adoc, .asc; .pod. So it seems that there are many communities on GitHub who prefer their own markup and tools.

We could possibly write the Pillar tool for GitHub or an exporter to the preferred markup language of the above.

This author provides arguments for using reStructuredText over Markdown for GitHub documents. Citing deficiencies in Markdown and expressiveness in reST.

https://gist.github.com/dupuy/1855764

So again. I am just not seeing a consensus around any emerging standard for "the markup language".

At the same time if you are desirous of writing in Commonmark in your text editor. Can you not write conversion software that goes from Commonmark to Pillar? Thus, meeting want you want and what we require? If you were to do so, you would definitely have a good understanding of the differences in philosophy and capabilities of each. Just a thought.

Any way, thanks for engaging in the conversation. I wasn't targeting you personally, but rather the topic. You are not alone in your thinking. The Pharo community is not alone in its thinking either.

Thanks.

Jimmie

On 08/14/2017 11:34 AM, Tim Mackinnon wrote:

...

Jimmie et al. nicely reasoned arguments - and Doru's point about controlling the syntax is an interesting one that I hadn’t thought about.

Personally, I find having too many similar syntax’s confusing - contributing to things is hard enough - having to remember that its !! Instead of ## and “” instead of ** is just frustrating for me.

My vote would be what Peter suggested - use http://spec.commonmark.org/0.28/ and put our Pillar extensions back on top for things that Stef was mentioning. (I think that’s what I’ve understood gfm markdown is).

Sure, maybe we were first with Pillar, but for me, lots of programming is in other languages, and I use Smalltalk where I can, and a hybrid of multiple languages and projects is often the reality - so a lowest common denominator of Markdown is just easier. The fact that we are quite close to what our colleagues in other languages use (regardless of what Python has chosen), is quite interesting.

That said, if the community wants to stick to its gun’s thats fine - I will probably still investigate how to use Commonmark for myself, and will still contribute to Pillar docs where I can (and curse history) - but I think we are long better off trying to join emerging standards where we can particularly if they aren’t our core language thing. And it just makes it less frictionless for ourselves and newcomers.

Of course, if we were to move, we would need to translate a lot of quality docs to a new format - but I would be up for contributing to that if that was a deciding factor.

Tim

...

On 14 Aug 2017, at 16:41, Jimmie Houchin <jlhouchin@gmail.com <mailto:jlhouchin@gmail.com>> wrote:

TL;DR

Main points: Their is no universally accepted markup language. Other communities use their own markup and tools and their markup and tools choice is not determine by other communities decisions. We need a language and tool chain that we can control and maintain which accomplishes our goals. Our language and tools already exist and have existed for longer than most of the other markup languages. Of course they existed in various different forms over the years and have evolved into what they currently are. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects.

I just want to comment on the fact that there is no universal markup language that every development community has settled upon. Making Markdown or some variant the markup language for Pharo only aligns us with a certain part of the development community. Even Markdown is not unified as is evident by the discussion.

It is true that GitHub uses their variant of Markdown. And as long as we use GitHub we will need to use their variant for documents that reside on their system.

However as a significant counter example to lets all use gfm Markdown, is the Python community and their documentation.

https://docs.python.org/devguide/documenting.html """ 7. Documenting Python The Python language has a substantial body of documentation, much of it contributed by various authors. The markup used for the Python documentation is reStructuredText, developed by the docutils project, amended by custom directives and using a toolset named Sphinx to post-process the HTML output.

This document describes the style guide for our documentation as well as the custom reStructuredText markup introduced by Sphinx to support Python documentation and how it should be used.

The documentation in HTML, PDF or EPUB format is generated from text files written using the reStructuredText format and contained in the CPython Git repository. """

So the Python community uses their own markup language and their own tool chain. So therefore, it is not wrong for a community to go their own way, for their own reasons. Even within the conventional file based languages such as Python.

The fact that you have tools such as Pandoc, suggest that there is not true uniformity or unanimity among developers as to the best markup language or tool chain.

I believe that a language that we can control and maintain is better than adopting some other foreign markup language that is neither better, nor unanimously used by all. That would ultimately potentially require extensions to accomplish our goals. Then we would be maintaining someone else's language with our extensions that may or may not be accepted by the larger community. This does not prevent but rather encourages fragmentation of the existing Markdown.

Regardless, Pillar markup already exists. The tools in Pharo already understand it. Should someone desire to use Pharo which is far more different from Python/Ruby/etc. than Pillar syntax is from Markdown. Then it should be worth their effort to learn our tools.

Pillar markup is older than Markdown, etc. It's history begins in SmallWiki. It isn't as if we jumped up and decided to create something new in order to be different. Our markup and tools are older. They (and others) are the ones that decided to do their own markup and tools. And it is okay that they did so. Nothing wrong with doing so. Every community has the right to what they believe is best for their community. Even if other communities disagree.

The ability to control and maintain is highly valuable. We can understand what our requirements are for today. But we can not know what the requirements are in the future. Nor can we know that Markdown or whomever will have such requirements when they appear. It is easy to see in the beginning with the Squeak Wiki syntax to the now Pillar syntax, changes that have been made to accommodate new requirements as they became known. We need to maintain that ability. Sure we would reserve the right to do so in any language we adopt. But the then current standard bearer of said language would determine whether what we do is acceptable and incorporate or whether we are then in fact adding to their fragmentation. Pillar is ours. There is not fragmentation when we evolve.

However, since we have made a decision to use GitHub and GitHub has made a decision to use their own GFM Markdown. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects. This way we can use our own tools and markup language to accomplish whatever we want to accomplish. Including generating a Readme.md for our GitHub projects.

Just wanted to toss out this simple opinion and facts about the situation.

Jimmie

On 08/14/2017 04:10 AM, Tudor Girba wrote:

...

Hi Tim,

The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books.

The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text.

Cheers, Doru

...

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com

“Live like you mean it."

P.S. 3 And I realize that there is an pillar issue "Create a small parser independant of PetitParser for a light pillar" https://github.com/pillar-markup/pillar/issues/174 AFAIK this parser already exists. The hand written parser from in the Pier CMS. Some small adaptations are probably necessary. On 8/15/17, H. Hirzel <hannes.hirzel@gmail.com> wrote:

P.S. 2

And yes the Pillar syntax which goes back to the Pier syntax [1] has been around for quite some time

2008 at least according to

Pier http://wiki.squeak.org/squeak/3700

http://www.piercms.com/doc/syntax

[1] Pharo 6.1 catalog entry for Pillar

Pillar is a wiki-like syntax, its document model, a parser for it, and a set of exporters (e.g., HTML, LaTeX, Markdown...). Pillar is primarily used as the wiki syntax behind the *Pier CMS>http://piercms.com*. Pillar is also being used to write books: e.g., *the Enterprise Pharo book>http://books.pharo.org/*.

The original creator of Pillar (formerly known as ''the syntax behind the Pier CMS'') is Lukas Renggli. Nevertheless, *Damien Cassou>damien.cassou@inria.fr* is now the maintainer. The website is at *http://www.smalltalkhub.com/#!/~Pier/Pillar*. Issues should be reported to *https://github.com/pillar-markup/pillar/issues*

On 8/15/17, H. Hirzel <hannes.hirzel@gmail.com> wrote:

...

P.S. +1 for including pillar in the 6.2 image , then start working on the document tree and see how it can be adapted.

On 8/15/17, H. Hirzel <hannes.hirzel@gmail.com> wrote:

...

Offray,

thanks for the nice write-up about the general usefulness of Markdown for writing papers.

Stephen D. wrote yesterday in a terse way

"We can change the syntax or propose an alternate one as soon as it uses the same internal structure.

Stef"

This means that Pillars tagging system may be changed to a more generally known tagging system later.

I assume with 'internal structure' he refers to the AST/Document Object Model of Pillar which would need to be aligned with one of another markup system.

Maybe the gap is not all that large.

Personally I think as well that if Pharo could also be used as a markdown (commonmark) editor with its tool-chain to generate different types of documents would expand the area of application considerably.

--Hannes

On 8/15/17, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:

...

Hi,

While I appreciate the historical perspective, I continue to be with Tim on this (and I consider myself part of the Pharo Community). I have also personal preferences for other light markup languages (like txt2tags[1] and dokuwiki's[2]), but I will stick with Pandoc's Markdown, because is support everything Stephan or any professional author wants to do and in fact you can create full books with it, including references, tables, images, and bibliographic references (which are not supported by Pillar AFAIK), but also because is an emerging standard beyond the programmers community, including academicians and researchers with the Scholarly Markdown[3] initiative and with possibilities to import and export from/to several markup languages[4].

[1] http://txt2tags.org/ [2] https://www.dokuwiki.org/wiki:syntax [3] http://scholmd.org/ [4] http://pandoc.org/

I don't share the vision of particular communities choosing particular markup languages as default, because, if you already payed the price of learning that particular language/environment, you are willing to pay the price with whatever that community choose in other fronts like DVCS or markup languages. Python community supports reST, but also markdown and several other markup languages and Jupyter notebooks[5] user Markdown by default. In fact, the Iceberg Git support shows the increasing concern to bridge the Pharo world with the stuff you already know and I think that a similar approach should be taken in the documentation/markup front, even if this implies breaking compatibility with the canonical Smalltalk way (TM) (I really like that critical approach from Pharo to the past).

[5] http://jupyter.org/

That being said, I don't think that should be exclusively one way or another. We can have Pillar and (Pandoc's) Markdown, if the community doesn't reach and agreement on only one.

I plan to explore the Brick editor once I have time and will try to add Pandoc's Markdown support. Unfortunately, in the past I have not had many luck testing and giving feedback on Moose alpha releases of tools and my questions/comments on them remain largely unanswered or simply ignored for long time (or just forever), so my priority on testing these tools have just decreased, but once Brick editor become more well supported, Pandoc's Markdown support for it will be in my target and concerns.

Cheers,

Offray

On 14/08/17 12:48, Jimmie Houchin wrote:

...

Thank Tim,

My primary reason to submit the message was not to necessarily persuade you per se. But to provide something historical for the mailing list as this can be a recurring subject. Why use Pillar markup instead of ???(insert personal favorite).

If Pharo were to decide on a different markup language. The question would still be which one, why and then how do we proceed. Then our extensions may not be accepted by the greater body of users of said markup. We would still be contributing to the fragmentation of markup. As far as familiarity, I don't know. And familiarity with what. I do not find that reStructuredText to be similar to Markdown.

It would stop people from asking why we aren't using Markdown. But it wouldn't prevent others. Why aren't we using GFM Markdown, or Kramdown or Commonmark or ...? Why aren't we using YAML or reST or AsciiDoc or insert latest greatest creation markup or current flavor of the moment. Which is why I wanted to point out that there is no consensus among users of markup languages. At least I do not see one. Nor do I believe that we have seen the end of creation of new markup languages.

I understand the difficulty, though I do not suffer from it as I have not mastered any of those other languages. I have been using Squeak/Pharo for a long time. I struggle when I look at those other languages. To me they are the foreign ones.

And I do not see these emerging standards you refer to. When we see Python, Ruby, Perl, C++, various projects, etc. communities having consensus on a common markup for documentation. Then I see an emerging standard. Until then it seems to possibly be an emerging standard for a particular markup language which is among the set of markup languages.

If we were the only language and development environment doing our own thing. Then we might have a very good reason to talk. But we are not. Python with its enormous community does its own thing. I don't know that other languages have a consensus for markup for documentation except for Python and Pharo.

While writing this email I went and discovered that even GitHub is not dogmatic about the subject. Obviously they have an opinion. But they permit multiple markup languages. Quite possibly someone could write a Pillar tool for GitHub to use and then we could just submit Readme.pillar for our projects. :)

https://github.com/github/markup

Shows that GitHub allows for .markdown, .mdown, .mkdn, .md; .textile; .rdoc; .org; .creole; .mediawiki, .wiki; .rst; .asciidoc, .adoc, .asc; .pod. So it seems that there are many communities on GitHub who prefer their own markup and tools.

We could possibly write the Pillar tool for GitHub or an exporter to the preferred markup language of the above.

This author provides arguments for using reStructuredText over Markdown for GitHub documents. Citing deficiencies in Markdown and expressiveness in reST.

https://gist.github.com/dupuy/1855764

So again. I am just not seeing a consensus around any emerging standard for "the markup language".

At the same time if you are desirous of writing in Commonmark in your text editor. Can you not write conversion software that goes from Commonmark to Pillar? Thus, meeting want you want and what we require? If you were to do so, you would definitely have a good understanding of the differences in philosophy and capabilities of each. Just a thought.

Any way, thanks for engaging in the conversation. I wasn't targeting you personally, but rather the topic. You are not alone in your thinking. The Pharo community is not alone in its thinking either.

Thanks.

Jimmie

On 08/14/2017 11:34 AM, Tim Mackinnon wrote:

...

Jimmie et al. nicely reasoned arguments - and Doru's point about controlling the syntax is an interesting one that I hadn’t thought about.

Personally, I find having too many similar syntax’s confusing - contributing to things is hard enough - having to remember that its !! Instead of ## and “” instead of ** is just frustrating for me.

My vote would be what Peter suggested - use http://spec.commonmark.org/0.28/ and put our Pillar extensions back on top for things that Stef was mentioning. (I think that’s what I’ve understood gfm markdown is).

Sure, maybe we were first with Pillar, but for me, lots of programming is in other languages, and I use Smalltalk where I can, and a hybrid of multiple languages and projects is often the reality - so a lowest common denominator of Markdown is just easier. The fact that we are quite close to what our colleagues in other languages use (regardless of what Python has chosen), is quite interesting.

That said, if the community wants to stick to its gun’s thats fine - I will probably still investigate how to use Commonmark for myself, and will still contribute to Pillar docs where I can (and curse history) - but I think we are long better off trying to join emerging standards where we can particularly if they aren’t our core language thing. And it just makes it less frictionless for ourselves and newcomers.

Of course, if we were to move, we would need to translate a lot of quality docs to a new format - but I would be up for contributing to that if that was a deciding factor.

Tim

...

On 14 Aug 2017, at 16:41, Jimmie Houchin <jlhouchin@gmail.com <mailto:jlhouchin@gmail.com>> wrote:

TL;DR

Main points: Their is no universally accepted markup language. Other communities use their own markup and tools and their markup and tools choice is not determine by other communities decisions. We need a language and tool chain that we can control and maintain which accomplishes our goals. Our language and tools already exist and have existed for longer than most of the other markup languages. Of course they existed in various different forms over the years and have evolved into what they currently are. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects.

I just want to comment on the fact that there is no universal markup language that every development community has settled upon. Making Markdown or some variant the markup language for Pharo only aligns us with a certain part of the development community. Even Markdown is not unified as is evident by the discussion.

It is true that GitHub uses their variant of Markdown. And as long as we use GitHub we will need to use their variant for documents that reside on their system.

However as a significant counter example to lets all use gfm Markdown, is the Python community and their documentation.

https://docs.python.org/devguide/documenting.html """ 7. Documenting Python The Python language has a substantial body of documentation, much of it contributed by various authors. The markup used for the Python documentation is reStructuredText, developed by the docutils project, amended by custom directives and using a toolset named Sphinx to post-process the HTML output.

This document describes the style guide for our documentation as well as the custom reStructuredText markup introduced by Sphinx to support Python documentation and how it should be used.

The documentation in HTML, PDF or EPUB format is generated from text files written using the reStructuredText format and contained in the CPython Git repository. """

So the Python community uses their own markup language and their own tool chain. So therefore, it is not wrong for a community to go their own way, for their own reasons. Even within the conventional file based languages such as Python.

The fact that you have tools such as Pandoc, suggest that there is not true uniformity or unanimity among developers as to the best markup language or tool chain.

I believe that a language that we can control and maintain is better than adopting some other foreign markup language that is neither better, nor unanimously used by all. That would ultimately potentially require extensions to accomplish our goals. Then we would be maintaining someone else's language with our extensions that may or may not be accepted by the larger community. This does not prevent but rather encourages fragmentation of the existing Markdown.

Regardless, Pillar markup already exists. The tools in Pharo already understand it. Should someone desire to use Pharo which is far more different from Python/Ruby/etc. than Pillar syntax is from Markdown. Then it should be worth their effort to learn our tools.

Pillar markup is older than Markdown, etc. It's history begins in SmallWiki. It isn't as if we jumped up and decided to create something new in order to be different. Our markup and tools are older. They (and others) are the ones that decided to do their own markup and tools. And it is okay that they did so. Nothing wrong with doing so. Every community has the right to what they believe is best for their community. Even if other communities disagree.

The ability to control and maintain is highly valuable. We can understand what our requirements are for today. But we can not know what the requirements are in the future. Nor can we know that Markdown or whomever will have such requirements when they appear. It is easy to see in the beginning with the Squeak Wiki syntax to the now Pillar syntax, changes that have been made to accommodate new requirements as they became known. We need to maintain that ability. Sure we would reserve the right to do so in any language we adopt. But the then current standard bearer of said language would determine whether what we do is acceptable and incorporate or whether we are then in fact adding to their fragmentation. Pillar is ours. There is not fragmentation when we evolve.

However, since we have made a decision to use GitHub and GitHub has made a decision to use their own GFM Markdown. It might be nice to have a GFM Markdown exporter from Pillar for GitHub projects. This way we can use our own tools and markup language to accomplish whatever we want to accomplish. Including generating a Readme.md for our GitHub projects.

Just wanted to toss out this simple opinion and facts about the situation.

Jimmie

On 08/14/2017 04:10 AM, Tudor Girba wrote:

...

Hi Tim,

The main benefit of relying on Pillar is that we control its syntax and can easily extend it for our purposes. Also, there was quite a bit of engineering invested in it, and even though we still need to improve it, there exists a pipeline that allows people to quickly publish books.

The figure embedding problem is one example of the need to customize the syntax and behavior, but this extensibility will become even more important for supporting the idea of moving the documentation inside the image. For example, the ability to refer to a class, method or other artifacts will be quite relevant soon especially that the editor will be able to embed advanced elements inside the text.

Cheers, Doru

...

On Aug 14, 2017, at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

-- www.tudorgirba.com www.feenk.com

“Live like you mean it."

This is in our plan. Have a look at Ecstatic because I want to be able to - use plain pillar - generate plain HTML There is also a little web server to display your ecstatic web site. All this is rudimentary. We use mustache for the templating. Stef On Mon, Aug 14, 2017 at 10:46 AM, Tim Mackinnon <tim@testit.works> wrote:

Hi Stef - I think your’s is a fair requirement (in fact I hit something similar when doing a static website using a JS markdown framework - and this is why I mentioned Kramdown which adds a few extras to regular markdown - but it feels like it goes a bit too far).

My next item on my learning todo list was to try and replace that JS generator with something from Smalltalk - so I think we can possibly come up with something that ticks all the right boxes (I’d like to try anyway).

I’ll keep working away on it and compare notes with you. I think with Pillar, it was more that things like headers, bold and italics are similar concepts but just use different characters - so I keep typing the wrong thing and getting frustrated particularly when we embrace Git and readme.md is in markdown.

Tim

...

On 13 Aug 2017, at 20:08, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi tim

I personally do not care much about the syntax but I care about what I can do with it (ref, cite, ... ) I cannot write books in markdown because reference to figures!!!!!! were missing.

And of course a parser because markdown is not really nice to parse and I will not write a parser because I have something else to do. I want to make pillar smaller, simpler, nicer.

Now if someone come up with a parser that parse for REAL a markdown that can be extended with decent behavior (figure reference, section reference, cite) and can be extended because there are many things that can be nice to have (for example I want to be able to write the example below) and emit a PillarModel (AST) we can talk to have another syntax for Pillar but not before.

[[[test 2+3

...

...

...

5 ]]]

and being able to verify that the doc is in sync.

Stef

On Sat, Aug 12, 2017 at 12:37 AM, Tim Mackinnon <tim@testit.works> wrote:

...

Of course, I/we recognise and appreciate all the work that's gone into docs in pillar - but I think it should be reasonably straightforward to write a converter as it is pretty closely related from what I have seen.

So I don't make the suggestion flippantly, and would want to help write a converter and get us to a common ground where we can differentiate on the aspects where we can excel.

Tim

Sent from my iPhone

...

On 11 Aug 2017, at 23:21, Peter Uhnak <i.uhnak@gmail.com> wrote:

A long time issue with Markdown was that there was no standardization (and when I used Pillar's MD export ~2 years ago it didn't work well).

However CommonMark ( http://spec.commonmark.org/0.28/ ) has become the de-facto standard, so it would make sense to support it bidirectionally with Pillar.

...

The readme.md that Peter is talking about is gfm markdown

Well, technically it is just a CommonMark, as I am not using any github extensions. (Github uses CommonMarks and adds just couple small extensions.)

Peter

On 12 Aug 2017, at 12:46, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi,

I would very much like this path. What would constitute tiny?

no idea, we need to think on it :) but I see no point in create a new (even small) parser/scanner each time we need one. At the end you have several small parsers that repeat always same process when PetitParser is good for most of our problems (and if you need to improve it. you can always load PP compiler, etc.). Esteban

Cheers, Doru

...

On Aug 11, 2017, at 9:10 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:

hi,

...

On 11 Aug 2017, at 18:57, Cyril Ferlicot D. <cyril.ferlicot@gmail.com> wrote:

Another step would be to get a minimal parser not relying on PetitParser.

Let’s think differently: why not to include a tiny PetitParser? Then we can think on:

- pillar sintax (better than just a restricted version) - simplify other “small parsers” that are already on the image. - we provide a tool to o cool stuff (instead relying as always in regexp, etc.)

cheers, Esteban

-- www.tudorgirba.com www.feenk.com

"Beauty is where we see it."

Esteban, I think that when Stef says "a mini version of pillar" he does not mean a mini version of the parser or the AST, but a version that does not have the exporting to html/latex/... On Sat, Aug 12, 2017 at 4:37 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:

and btw, this kind of threads belongs to pharo-dev, not to pharo-users :)

Esteban

...

On 12 Aug 2017, at 16:35, Esteban Lorenzano <estebanlm@gmail.com> wrote:

...

On 12 Aug 2017, at 12:46, Tudor Girba <tudor@tudorgirba.com> wrote:

Hi,

I would very much like this path. What would constitute tiny?

no idea, we need to think on it :) but I see no point in create a new (even small) parser/scanner each time we need one. At the end you have several small parsers that repeat always same process when PetitParser is good for most of our problems (and if you need to improve it. you can always load PP compiler, etc.).

Esteban

...

Cheers, Doru

...

On Aug 11, 2017, at 9:10 PM, Esteban Lorenzano <estebanlm@gmail.com>

wrote:

...

...

hi,

...

On 11 Aug 2017, at 18:57, Cyril Ferlicot D. <cyril.ferlicot@gmail.com>

wrote:

...

...

Another step would be to get a minimal parser not relying on PetitParser.

Let’s think differently: why not to include a tiny PetitParser? Then we can think on:

- pillar sintax (better than just a restricted version) - simplify other “small parsers” that are already on the image. - we provide a tool to o cool stuff (instead relying as always in regexp, etc.)

cheers, Esteban

-- www.tudorgirba.com www.feenk.com

"Beauty is where we see it."

-- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Hi,

Am 11.08.2017 um 21:10 schrieb Esteban Lorenzano <estebanlm@gmail.com>:

hi,

...

On 11 Aug 2017, at 18:57, Cyril Ferlicot D. <cyril.ferlicot@gmail.com <mailto:cyril.ferlicot@gmail.com>> wrote:

Another step would be to get a minimal parser not relying on PetitParser.

Let’s think differently: why not to include a tiny PetitParser? Then we can think on:

- pillar sintax (better than just a restricted version) - simplify other “small parsers” that are already on the image. - we provide a tool to o cool stuff (instead relying as always in regexp, etc.)

+1 I like that. I always feel bad if I use regex in pharo because I think I can do better. But then installing petit parser just to do some parsing sounds heavy weight. And parsing is such central functionality that it would be good to have a small parser near the core. For pillar I would like to see that class comments are a layer on top of the image core. It is metadata that can be optional and pillar could be a dependency to that layer. This would make it possible to strip out on deployment, too. Because I don't need class comments on my server :) Norbert

What I meant is that I have a readme, e.g. https://github.com/OpenPonk/xmi , which contains * Pharo code examples * images * References to Pharo code (class names, etc.) But if the code changes (renames, API changes, different UI), I have to manually update the README. There is also no way for me to validate the examples, etc and see if anything is broken. Peter On Fri, Aug 11, 2017 at 06:58:50PM +0200, Stephane Ducasse wrote:

Let us rephrase it:

- I would like to have a mini pillar with a simplified model and visitor to display class comments.

- then think about Pharo 70 as the core and birth of a new generation of imageS

I will restart to revisit Pillar once I'm done with the Lecture at Prague.

Stef

On Fri, Aug 11, 2017 at 6:52 PM, Peter Uhnak <i.uhnak@gmail.com> wrote:

...

Hi,

I would like to propose including Pillar in the Pharo image by default.

My reasoning:

Since we are moving to git, and most people will use github, gitlab, and the likes, it is expected to include a README.md file (or possibly more extensive documentation) alongside the code.

Which means that people will (and are) writing the README mostly by hand, which of course is problematic, as e.g. code snippets can get deprecated, screenshots become outdated, etc.

As Pillar tries to address these problems, it would make sense to me to include Pillar in the image by default, as anyone using git (which eventually should be everyone) will most likely benefit from writing their documentation in Pillar. Similarly using Pillar would open an avenue to provide the documentation in-image, e.g. one exporter for html/markdown, and another one for Pharo's Help system.

I could, of course, install Pillar every time, but considering thats extra effort and in the extra time I can fix the issues by hand, I don't have such an incentive to use Pillar for this.

Questions & Problems:

I don't know by how much would pillar increase the image size. Perhaps there could be (a) "lightweight Pillar" (that would include just pillar & markdown exporter), or (b) we would have different images for different uses.

By different images I mean something along the lines of a) developer image - meant to be directly used by developers to create their software b) production image - as a foundation for running systems / users

Does this make sense?

Peter