February 2015 - Pharo-users

Re: [Pharo-users] recover lost changes

by Stephan Eggermont

Norbert wrote: >Yes but resources are still sparse. I'm usually arguing the same way like you. >But at the moment I have the feeling we postpone the integration of too much too long. >If the release is postponed until july I'd vote for integrating it now and iron it out. >Otherwise it might be never integrated and that is our poor history of >good projects dying because. I don't know the current state of epicea. To me, it sounds like something that should have some time in real use before the code freeze. I agree that a postponed release would make that time available if we can integrate real soon now. Stephan

Feb. 11, 2015

Re: [Pharo-users] SmaCC: First steps

by kilon alios

"What you see is that, with the {{}}, I create PyAtomNode instances for all productions, even if it isn't appropriate. Maybe this should be changed like that for lists : | <lbrack> <rbrack> {{List}} | <lbrack> listmaker 'list' <rbrack> {{List}}" Both approaches you described a) adding instance methods to PyAtomNode that provide checks for the type b) Creating separate nodes for diffirent types work for me. Solution (b) seemed more smalltalky to me. Maybe a best compromise would be to have for example PyListNode as a subclass of PyAtomNode ? If I can create something that others find useful too, I certainly would prefer it. My needs are not very specific, I think I want pretty much what anyone would want for importing data from python to pharo . One way or another I will satisfy my needs this is not my worry. "And the last one is about the visitor. For complex processing like the transformations you intend, I would see two strategies: a builder inside the visitor with a stack/context strategy, so that you can recurse in your visit of the ast and add elements to the right collection, or a simple recurse and merge the result of the lower visits (when in a List node, collect the visit of all the children as an array or as an OrderedCollection)." Yes that was my way of thinking too. A collection of methods that consume the AST , walk the tree and build a more simplified structure. The problem I was having was two side, from one side PyAtomNode is used for several diffirent things. From the other side not only lists, dictionaries, tupples can be multidimensional but also can act as containers for each other. So a list can contain a dictionary which can contain a list which can contain a tupple and as you imagine the rabbit whole can go very deep. Of course nothing of this is surprising for any language. Generally speaking this is not such a big problem right now for me because I prefer dealing with simple types with a bit of multidimensionality. Most of the types Blender uses is like that. But it may become a problem later on for example if the user wants to have access to the node system of blender. Nodes can easily contain other nodes and it can create a nightmare scenario in that case but I will leave that for when the time comes. "I believe you are on the right path, if my explanations made sense :)" Your explanation not only made sense but you pretty much described what I was considering doing. "If you start changing the grammar as suggested above, make a fork and pull requests on github :)" Will do, my focus is in latest python 3 because its what Blender uses, but on types should not make any diffirence.

Feb. 11, 2015

Re: [Pharo-users] Changes file size

by Sven Van Caekenberghe

> On 11 Feb 2015, at 09:35, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > I will try to do what you did and commit a version of that method with a normal time stamp. I committed: === Name: Glorp-SvenVanCaekenberghe.96 Author: SvenVanCaekenberghe Time: 11 February 2015, 9:59:31.16347 am UUID: 1e1961df-8d15-44a7-bcef-9ec7dd17aab6 Ancestors: Glorp-SvenVanCaekenberghe.95 Try editing GlorpSession>>#dropTables: to get rid of the enormous timestamp === And then tried loading in a fresh 4.0 image: ConfigurationOfGlorp project bleedingEdge load. And image/changes sizes remain normal: $ ls -lah current/ total 137336 drwxr-xr-x 8 sven staff 272B Feb 11 10:27 . drwxr-xr-x 11 sven staff 374B Feb 11 10:23 .. -rw-rw-r-- 1 sven staff 27M Feb 11 10:30 Pharo.changes -rw-rw-r--@ 1 sven staff 40M Feb 11 10:30 Pharo.image -rw-r--r--@ 1 sven staff 12K Feb 11 10:27 PharoDebug.log drwxr-xr-x 6 sven staff 204B Feb 11 10:25 package-cache drwxr-xr-x 4 sven staff 136B Feb 11 10:26 play-cache -rw-r--r-- 1 sven staff 29B Feb 11 10:23 updates40.staged $ ls -lah current/package-cache/ total 1640 drwxr-xr-x 6 sven staff 204B Feb 11 10:25 . drwxr-xr-x 8 sven staff 272B Feb 11 10:27 .. -rw-r--r--@ 1 sven staff 7.5K Feb 11 10:24 ConfigurationOfGlorp-TorstenBergmann.42.mcz -rw-r--r--@ 1 sven staff 541K Feb 11 10:25 Glorp-SvenVanCaekenberghe.96.mcz -rw-r--r--@ 1 sven staff 2.8K Feb 11 10:25 GlorpPharoCompatibility-GuillermoPolito.6.mcz -rw-r--r--@ 1 sven staff 261K Feb 11 10:25 GlorpTests-FranckWarlouzet.41.mcz Let's hope it now stays like that ;-) Sven

Feb. 11, 2015

GSOC 2015 application

by Serge Stinckwich

Dear all, with Uko, we are considering submitting the Pharo project as a mentoring organization for GSOC 2015: http://www.google-melange.com/gsoc/homepage/google/gsoc2015 We will need your help in order to find ideas of projects. Stay tune for more information soon. Thank you. Cheers, -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Every DSL ends up being Smalltalk http://www.doesnotunderstand.org/

Feb. 11, 2015

Re: [Pharo-users] SmaCC: First steps

by Thierry Goubier

Hi Kilon, 2015-02-11 8:24 GMT+01:00 kilon alios <kilon.alios(a)gmail.com>: > Ok so after rereading the tutorial and testing again and again , I think I > have finally managed to understand how SmaCC really works and I was > succesful into converting simple python litsts to pharo arrays and ordered > collections. > > The tricky part now is to apply this knowledge to complex python types > like multi dimensional lists, tuples and dictionaries. I understand that > the visitor allows me to visit a specific object instances each time they > are found in the AST . But because I want to walk the AST in order to build > multi dimensional ordered collections I need something more, or maybe my > understanding of the visitor pattern is flawed. > > The problem I am having here is that each time I parse a python type that > python type is not necessarily represented by different kind of node. For > example whether its a list or a tuple or a dictionary the same class is > used PyAtomNode. In order to differentiate between those diffirent python > types PyAtomNode has instance variables for right and left bracket, > parantheses, and curly. So my initial thinking is to check those instance > variables to see if they are nil and from that I can conclude which python > type I am parsing. > > So I can preform simple ifs that check that the instance variable is Nil > or not but the question is if my strategy is a good one or a bad one. > Well, I see three things in what you wwant to achieve. The first one is that checking the node (it has parentheses, brackets, etc...) is a good way of determining its type during the visitor traversal. You may extend directly PyAtomNode with selectors such as isArray, isDictionary, etc... so as to cope with the fact that the parser puts different objects under the umbrella PyAtomNode. The second one is that I did most of the AST generation as a crude, get rid of warnings approach (and it took me long enough). Now that you have a better understanding of your requirements, it may be a good idea to revisit the grammar and ensure that the right type of node is generated. For example, if you rewrite the atom productions : atom: <lparen> <rparen> {{}} | <lparen> yield_expr 'list' <rparen> {{}} | <lparen> testlist_comp 'list' <rparen> {{}} | <lbrack> <rbrack> {{}} | <lbrack> listmaker 'list' <rbrack> {{}} | <lcurly> dictorsetmaker 'list' <rcurly> {{}} | <lcurly> <rcurly> {{}} | "`" testlist1 'list' "`" {{BackTick}} | <name> {{Symbol}} | <number> {{Number}} | strings ; What you see is that, with the {{}}, I create PyAtomNode instances for all productions, even if it isn't appropriate. Maybe this should be changed like that for lists : | <lbrack> <rbrack> {{List}} | <lbrack> listmaker 'list' <rbrack> {{List}} Like that I get PyListNode when I parse '[ ]' . I just have to tune the AST generation code so that it gives me the nodes I need, and, at the moment, you are the right person to do so since you're molding it to your needs. And the last one is about the visitor. For complex processing like the transformations you intend, I would see two strategies: a builder inside the visitor with a stack/context strategy, so that you can recurse in your visit of the ast and add elements to the right collection, or a simple recurse and merge the result of the lower visits (when in a List node, collect the visit of all the children as an array or as an OrderedCollection). > > I could define my own syntax to simplify the AST tree including different > nodes for different python types , because from the looks of it , it seems > it is a bit too verbose for my needs but On the other hand I am not so sure > because in the future my needs may become more verbose too. > I believe this is approach two above: you can mold the grammar to your needs. As John Brant told me: an AST is not a parse tree. So you can, and should, adapt the AST generation in the way that makes sense to you. > > So I am very close and ready to create my full python types converter for > pharo but I wanted some good advice before wasting time on something that > is not efficient. > I believe you are on the right path, if my explanations made sense :) > > > By the way Thierry I have to agree with you Smacc is a very capable > parser, also I like the use of regex syntax, makes it uglier compared > Pettit Parser but I prefer the compact regex syntax to having to define and > browse tons of classes and send tons of messages. Also the Python support > is very good and I am impressed how easily SmaCC can parse whole python > applications since some of the test are very complex. Well done great work! > Thank to the help of all who have worked on Python parsing before and with me: having a large test base as I inherited from the previous parser is a huge benefit. Happy that you like it: implementing the ident/dedent tokens was the most interesting part in there. If you start changing the grammar as suggested above, make a fork and pull requests on github :) Thierry

Feb. 11, 2015

Re: [Pharo-users] recover lost changes

by Norbert Hartl

> Am 11.02.2015 um 09:33 schrieb Stephan Eggermont <stephan(a)stack.nl>: > > Norbert wrote: >>> Am 10.02.2015 um 20:59 schrieb stepharo <[hidden email]>: >>> >>> Yes this should be fixed. >>> >> You mean fix the current implementation or swap it for epicea? > > The current implementation. It is a significant barrier to entry for > new smalltalkers and gives a very bad impression. > The epicea swap is a high-impact change, that is likely to > affect stability for some time. Yes but resources are still sparse. I'm usually arguing the same way like you. But at the moment I have the feeling we postpone the integration of too much too long. If the release is postponed until july I'd vote for integrating it now and iron it out. Otherwise it might be never integrated and that is our poor history of good projects dying because. my 2 cents, Norbert

Feb. 11, 2015

Re: [Pharo-users] Changes file size

by Sven Van Caekenberghe

> On 10 Feb 2015, at 09:13, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Esteban, > > You might be on something important there. > > I always found it strange what effect loading Glorp had on an image, blowing it up. > > I can't say I understand though, best make it into an issue. I did some further digging. Basically, the GlorpSession>>#dropTables:'s MCMethodDefinitions are totally broken. I loaded Glorp in a clean 4.0 image (and did a merge) and I have two such definition instances, each with a #timeStamp of MEGABYTES, one ~ 60 Mb, one ~ 130 Mb, the contents is 'am From VisualWorks' and then lots of garbage: No wonder the image grows by 200 Mb ! I will try to do what you did and commit a version of that method with a normal time stamp. Sven > Sven > >> On 10 Feb 2015, at 00:22, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: >> >> Just when I found a way to reduce the total size of the changes file, I >> noticed the latest versions in the repository doesn't have this issue, >> what a good way of wasting one's time learning about the guts of >> Monticello :) >> >> But just for the record: >> The problem happened after installing into GlorpSession a >> MCMethodDefinition (#dropTables:), I logged every single method >> installation, and the point where things went bananas was after >> installing that culprit method: >> >> After a MCMethodDefinition(dropTables): 1201686 bytes (~1.2Mb) >> After a MCMethodDefinition(dropTables:): 135419895 bytes (~135Mb) >> >> I did a fileOut, deleted the method, and then file it in again, and >> after versioning (and reloading) the changes increased ~1Mb after >> installing Glorp. >> >> I didn't find anything particularly suspicious about it, not in the >> source code nor in the bytecodes. >> >> Now after running PharoChangesCondenser condense the changes file >> effectively shrinks. >> >> Regards! >> >> >> Esteban A. Maringolo >> >> >> 2014-10-28 12:31 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>> >>>> On 28 Oct 2014, at 14:58, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: >>>> >>>> 2014-10-28 5:41 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>>>> Esteban, >>>>> >>>>> The Reddit example's CI Job (https://ci.inria.fr/pharo-contribution/job/Reddit/) also loads Seaside and Glorp (two big packages) in Pharo 3 and it too results in a 110 Mb image and 142 Mb changes file. After condensing, that goes to 276 Mb ! >>>> >>>>> I would say something is wrong here ;-) >>>> >>>> Either something is wrong, or we feature it and rename >>>> PharoChangesCondenser>>#condense to PharoChangesExpander>>#expand :) >>>> >>>>> Note that condensing changes on newly loaded code should not make much difference (in essence, all multiple versions of methods are reduced to 1). >>>> >>>>> I think we should create some issues out of this. >>>> >>>> This is true, however I find a changes of 140 megs to be MASSIVE. >>> >>> I totally agree, this is unacceptable. However, I did some tests, and the problem is with Glorp (or any of its sub packages). I tried building both my Reddit and HP35 examples from scratch in Pharo 3 on Linux. >>> >>> $ ./pharo reddit.image config http://smalltalkhub.com/mc/SvenVanCaekenberghe/Reddit/main ConfigurationOfReddit --install=stable >>> >>> $ ./pharo hp35.image config http://smalltalkhub.com/mc/SvenVanCaekenberghe/HP35/main ConfigurationOfHP35 --install=stable --group=Web-UI >>> >>> Both load Seaside and some other stuff, but only Reddit loads Glorp and PostgresV2. >>> >>> I did a condense changes on the HP35 image, the one on the Reddit image ran out of physical memory (granted is was a small machine) ! >>> >>> Here are the resulting sizes: >>> >>> $ ls -lah >>> total 337M >>> drwxr-xr-x 4 root root 4.0K Oct 28 15:09 . >>> drwx------ 12 root root 4.0K Oct 28 12:28 .. >>> -rw-r--r-- 1 root root 5.8M Oct 28 13:17 hp35.changes >>> -rw-r--r-- 1 root root 5.6M Oct 28 14:38 hp35-condense-test.changes >>> -rw-r--r-- 1 root root 5.8M Oct 28 14:27 hp35-condense-test.changes.bak >>> -rw-r--r-- 1 root root 29M Oct 28 14:38 hp35-condense-test.image >>> -rw-r--r-- 1 root root 29M Oct 28 13:17 hp35.image >>> drwxr-xr-x 2 root root 4.0K Oct 28 13:12 package-cache >>> -rwxr-xr-x 1 root root 367 Oct 28 12:29 pharo >>> -rw-rw-r-- 1 root root 265K Oct 24 14:42 Pharo.changes >>> -rw-rw-r-- 1 root root 21M Oct 24 14:42 Pharo.image >>> -rwxr-xr-x 1 root root 354 Oct 28 12:29 pharo-ui >>> drwxr-xr-x 3 root root 4.0K Oct 28 12:29 pharo-vm >>> -rw-r--r-- 1 root root 136M Oct 28 12:48 reddit.changes >>> -rw-r--r-- 1 root root 105M Oct 28 12:48 reddit.image >>> >>> Note how the HP35 image and changes sizes are totally acceptable/normal, while the Reddit one explodes, with the difference being Glorp. >>> >>> But I have no idea what causes this. >>> >>> There is no way Glorp can generate 100 Mb changes, when Seaside is OK with 6 Mb. >>> >>>> I remember loading ~4000 classes in the order of 10^6 LOC in Dolphin, and >>>> changes never got half that size. Sizes were ~28/55MB image/changes. >>>> The same in VAST, any image beyond the 20MB was an alert of something >>>> being leaked. >>>> >>>>> Hmm, we need more tests and data points, I will try on Linux command line later on. >>>> >>>> I ran in it in Linux (Ubuntu) through the command line. >>>> >>>> >>>> Regards! >>>> >>> >>> >> >

Feb. 11, 2015

Re: [Pharo-users] get them while they are young

by Martin BÃ¤hr

Excerpts from kilon alios's message of 2015-02-11 09:19:27 +0100: > Well done Martin, personally I would not recommend Pharo to beginner > coders, but with Updated Pharo By Example near completion I may change my > mind ;) well, they are not beginner coders, at least i hope not. they should have taken some programming classes already. greetings, martin. -- eKita - the online platform for your entire academic life -- chief engineer eKita.co pike programmer pike.lysator.liu.se caudium.net societyserver.org secretary beijinglug.org mentor fossasia.org foresight developer foresightlinux.org realss.com unix sysadmin Martin BÃ¤hr working in china http://societyserver.org/mbaehr/

Feb. 11, 2015

Re: [Pharo-users] recover lost changes

by Stephan Eggermont

Norbert wrote: >> Am 10.02.2015 um 20:59 schrieb stepharo <[hidden email]>: >> >> Yes this should be fixed. >> > You mean fix the current implementation or swap it for epicea? The current implementation. It is a significant barrier to entry for new smalltalkers and gives a very bad impression. The epicea swap is a high-impact change, that is likely to affect stability for some time. Stephan

Feb. 11, 2015

Re: [Pharo-users] get them while they are young

by kilon alios

Well done Martin, personally I would not recommend Pharo to beginner coders, but with Updated Pharo By Example near completion I may change my mind ;) On Wed, Feb 11, 2015 at 9:58 AM, Martin BÃ¤hr < mbaehr(a)email.archlab.tuwien.ac.at> wrote: > here is the reaction of a student preparing for a GSoC project for which > the > student needs to learn smalltalk: > > "I have installed Pharo v4.0 and started the book Pharo By Example as > suggested by you .... I just wanted to say that I am in the initial > stages > and loving it already :)" > > greetings, martin. > > -- > eKita - the online platform for your entire academic > life > -- > chief engineer > eKita.co > pike programmer pike.lysator.liu.se caudium.net > societyserver.org > secretary > beijinglug.org > mentor > fossasia.org > foresight developer foresightlinux.org > realss.com > unix sysadmin > Martin BÃ¤hr working in china > http://societyserver.org/mbaehr/ > >

Feb. 11, 2015