July 2017 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Creating the smallest server runtime footprint

by Sven Van Caekenberghe

> On 19 Jul 2017, at 14:55, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: > > I don't know how "mainstream" solutions perform on AWS Lambda or EC2, > but this seems really fast to me. 50 ms is great, assuming it bills by > every 100ms, you still have room to perform your computation. Yes, it seems incredibly fast. I'll have to try this myself to check, but I have no time now. > Thank you for pursuing this path, it could open a new territory for > using Pharo at big scale. > > Esteban A. Maringolo > > > 2017-07-17 8:32 GMT-03:00 Tim Mackinnon <tim(a)testit.works>: >> Well Iâve been shooting in the dark a bit - but I also left out the sound >> and display soâs (e.g. -x execlude the following and add back the null so's >> >> zip -r --symlinks ../deploy/$LAMBDA_NAME.zip * -x pharo-local/\* \*.sources >> \*.changes \*.st \*.log >> */libgit2.* */libSDL2* */B3DAccelerator* */JPEGRead* */vm-sound* >> */vm-display* tmp/\* */__MACOSX\* >> - zip -uR ../deploy/$LAMBDA_NAME.zip *-null.so >> >> >> And everything seems to run clean. (Would be useful to get some feedback >> from those in the know - does just leaving out soâs incurred a penalty if >> you donât recompile the VM? Presumably something would get written to std >> error or pharodebug if it was an issue). >> >> In fact my run times on EC2 are pretty impressive: >> >> PharoLambdaMin]$ time ./pharo Pharo.image exec "Lambda processJSON: '{}'" >> {"outputSpeech":{"text":"Good Morning, it's eleven >> twenty-six","type":"PlainText"},"shouldEndSession":true,"card":{"content":"Operation >> Successful","title":"Pharo Lambda","type":"Simple"}} >> >> real 0m0.039s >> user 0m0.028s >> sys 0m0.000s >> [ec2-user@ip-172-31-44-73 PharoLambdaMin]$ time ./pharo Pharo.image exec >> "Lambda processJSON: '{}'" >> {"outputSpeech":{"text":"Good Morning, it's eleven >> twenty-six","type":"PlainText"},"shouldEndSession":true,"card":{"content":"Operation >> Successful","title":"Pharo Lambda","type":"Simple"}} >> >> real 0m0.039s >> user 0m0.020s >> sys 0m0.008s >> >> >> Not bad eh? >> >> Tim >> >> On 17 Jul 2017, at 07:00, Tim Mackinnon <tim(a)testit.works> wrote: >> >> Thanks again Pavel - I'll try the 6.0 step 4 or possibly step 5 with sunit >> (as many libraries don't separate out their tests). >> >> I've also tried leaving out libgit and libsdl2 .so's on my server build and >> that seems fine too - making me wonder what others I can safely leave out? >> Sound is a candidate (but small fry in size but do you need the null >> variant?). >> >> Libcrypto is big - but I wonder if https routines would use that (and it >> sounds server processing'y so maybe best left). >> >> I was hoping to find a list explaining them somewhere - but it remains >> rather mysterious. >> >> However, at this point, I think I may have hit the sweet spot in size where >> AWS seems to load efficiently below a zip of 10mb? >> >> Tim >> >> Sent from my iPhone >> >> On 15 Jul 2017, at 09:35, Pavel Krivanek <pavel.krivanek(a)gmail.com> wrote: >> >> If you want to stay with Pharo 6 image, you can try the bootstrapped version >> of the minimal image: >> https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configu… >> >> -- Pavel >> >> 2017-07-15 10:33 GMT+02:00 Pavel Krivanek <pavel.krivanek(a)gmail.com>: >>> >>> Try the Pharo 7 metacello image (=Pharo 7 minimal image that the CI is >>> already converting to 64bit). There should be no problem with STON because >>> whole Pharo is loaded into it using metacello and filetree. Pharo 6 minimal >>> image is done differently (by shrinking) and not so well tested. >>> >>> For the conversion of 32-bit image to 64-bit image you need a VMMaker >>> image: >>> >>> https://ci.inria.fr/pharo/job/Spur-Git-Tracker/lastSuccessfulBuild/artifact… >>> and then evaluate: >>> ./pharo generator.image eval "[Spur32to64BitBootstrap new bootstrapImage: >>> 'conversion.image'] on: AssertionFailure do: [ :fail | fail resumeUnchecked: >>> nil ]" >>> >>> -- Pavel >>> >>> >>> >>> 2017-07-15 10:19 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>> >>>> Hi Pavel - thanks for getting me to the point where I could even have a >>>> minimal image. As Iâm on the edge of my Pharo knowledge here, Iâll try and >>>> run with this as best I can. >>>> >>>> Iâd been using the 6.0 image you suggested to me - but maybe I could use >>>> a 70 image with Pharo 6 for a while (until the VM diverges) right? >>>> >>>> The bit I havenât quite understood however, is how the 64bit image is >>>> created - as your reference is to a 32bit version? Is the 64bit one >>>> converted from 32 in a later stage? (For AWS Lambda I need 64bit) - am I >>>> right in thinking the pipeline stage after this one is the one you sent me - >>>> and the travis.yml file shows me what it does? But I canât see a trivis.yml >>>> in the conversion stage so Iâm not sure how it does that. (Question - how do >>>> I see what the pipelines do to answer my own questions?) >>>> >>>> I was hoping that there was a basic image that got me up to metacello >>>> baseline level to load git file tree packages/baselines in my own repo as >>>> well baselines on the internet. The one you sent me is fairly close to that >>>> (its just missing STON in the image and seems to have an issue with >>>> resolving undeclared classes that get loaded in - should do a fogbugz on >>>> that?) >>>> >>>> The follow-on from a metacello image is how we can get people to create >>>> better baselines that give you more minimal loading options (e.g. >>>> conditionally leave out the test cases perhaps) >>>> >>>> Tim >>>> >>>> On 15 Jul 2017, at 08:24, Pavel Krivanek <pavel.krivanek(a)gmail.com> >>>> wrote: >>>> >>>> Hi Tim, >>>> >>>> you can base the your work on the bootstrapped image, see >>>> https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit/, file >>>> Pharo7.0-core-*.zip >>>> >>>> This image does not have a lot of basic components like Monticello or >>>> network but it has a compiler so the code can be imported as *.st files. >>>> Then we have Pharo7.0-monticello-*.zip which will be easier to use and >>>> probably can fit your needs. Monticello and network support are included. >>>> But you cannot use baselines nor configurations to load your code. >>>> >>>> -- Pavel >>>> >>>> 2017-07-14 9:59 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>> >>>>> Hi - buoyed by the success of a minimal image (thanks Pavel), I'm >>>>> wondering if I can get even smaller. >>>>> >>>>> There are lots of .so's in the vm which wouldn't make sense on a server >>>>> once deployed - sound, maybe libgit ... >>>>> >>>>> Is there a list of the essential ones, or tips on what I can strip out >>>>> of the Linux deployment? I also recall that i can leave out .sources and >>>>> .changes as well right? >>>>> >>>>> Tim >>>>> >>>>> Sent from my iPhone >>>>> >>>> >>>> >>> >> >> >

July 19, 2017

Re: [Pharo-users] Creating the smallest server runtime footprint

by Esteban A. Maringolo

I don't know how "mainstream" solutions perform on AWS Lambda or EC2, but this seems really fast to me. 50 ms is great, assuming it bills by every 100ms, you still have room to perform your computation. Thank you for pursuing this path, it could open a new territory for using Pharo at big scale. Esteban A. Maringolo 2017-07-17 8:32 GMT-03:00 Tim Mackinnon <tim(a)testit.works>: > Well Iâve been shooting in the dark a bit - but I also left out the sound > and display soâs (e.g. -x execlude the following and add back the null so's > > zip -r --symlinks ../deploy/$LAMBDA_NAME.zip * -x pharo-local/\* \*.sources > \*.changes \*.st \*.log > */libgit2.* */libSDL2* */B3DAccelerator* */JPEGRead* */vm-sound* > */vm-display* tmp/\* */__MACOSX\* > - zip -uR ../deploy/$LAMBDA_NAME.zip *-null.so > > > And everything seems to run clean. (Would be useful to get some feedback > from those in the know - does just leaving out soâs incurred a penalty if > you donât recompile the VM? Presumably something would get written to std > error or pharodebug if it was an issue). > > In fact my run times on EC2 are pretty impressive: > > PharoLambdaMin]$ time ./pharo Pharo.image exec "Lambda processJSON: '{}'" > {"outputSpeech":{"text":"Good Morning, it's eleven > twenty-six","type":"PlainText"},"shouldEndSession":true,"card":{"content":"Operation > Successful","title":"Pharo Lambda","type":"Simple"}} > > real 0m0.039s > user 0m0.028s > sys 0m0.000s > [ec2-user@ip-172-31-44-73 PharoLambdaMin]$ time ./pharo Pharo.image exec > "Lambda processJSON: '{}'" > {"outputSpeech":{"text":"Good Morning, it's eleven > twenty-six","type":"PlainText"},"shouldEndSession":true,"card":{"content":"Operation > Successful","title":"Pharo Lambda","type":"Simple"}} > > real 0m0.039s > user 0m0.020s > sys 0m0.008s > > > Not bad eh? > > Tim > > On 17 Jul 2017, at 07:00, Tim Mackinnon <tim(a)testit.works> wrote: > > Thanks again Pavel - I'll try the 6.0 step 4 or possibly step 5 with sunit > (as many libraries don't separate out their tests). > > I've also tried leaving out libgit and libsdl2 .so's on my server build and > that seems fine too - making me wonder what others I can safely leave out? > Sound is a candidate (but small fry in size but do you need the null > variant?). > > Libcrypto is big - but I wonder if https routines would use that (and it > sounds server processing'y so maybe best left). > > I was hoping to find a list explaining them somewhere - but it remains > rather mysterious. > > However, at this point, I think I may have hit the sweet spot in size where > AWS seems to load efficiently below a zip of 10mb? > > Tim > > Sent from my iPhone > > On 15 Jul 2017, at 09:35, Pavel Krivanek <pavel.krivanek(a)gmail.com> wrote: > > If you want to stay with Pharo 6 image, you can try the bootstrapped version > of the minimal image: > https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configu… > > -- Pavel > > 2017-07-15 10:33 GMT+02:00 Pavel Krivanek <pavel.krivanek(a)gmail.com>: >> >> Try the Pharo 7 metacello image (=Pharo 7 minimal image that the CI is >> already converting to 64bit). There should be no problem with STON because >> whole Pharo is loaded into it using metacello and filetree. Pharo 6 minimal >> image is done differently (by shrinking) and not so well tested. >> >> For the conversion of 32-bit image to 64-bit image you need a VMMaker >> image: >> >> https://ci.inria.fr/pharo/job/Spur-Git-Tracker/lastSuccessfulBuild/artifact… >> and then evaluate: >> ./pharo generator.image eval "[Spur32to64BitBootstrap new bootstrapImage: >> 'conversion.image'] on: AssertionFailure do: [ :fail | fail resumeUnchecked: >> nil ]" >> >> -- Pavel >> >> >> >> 2017-07-15 10:19 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>> >>> Hi Pavel - thanks for getting me to the point where I could even have a >>> minimal image. As Iâm on the edge of my Pharo knowledge here, Iâll try and >>> run with this as best I can. >>> >>> Iâd been using the 6.0 image you suggested to me - but maybe I could use >>> a 70 image with Pharo 6 for a while (until the VM diverges) right? >>> >>> The bit I havenât quite understood however, is how the 64bit image is >>> created - as your reference is to a 32bit version? Is the 64bit one >>> converted from 32 in a later stage? (For AWS Lambda I need 64bit) - am I >>> right in thinking the pipeline stage after this one is the one you sent me - >>> and the travis.yml file shows me what it does? But I canât see a trivis.yml >>> in the conversion stage so Iâm not sure how it does that. (Question - how do >>> I see what the pipelines do to answer my own questions?) >>> >>> I was hoping that there was a basic image that got me up to metacello >>> baseline level to load git file tree packages/baselines in my own repo as >>> well baselines on the internet. The one you sent me is fairly close to that >>> (its just missing STON in the image and seems to have an issue with >>> resolving undeclared classes that get loaded in - should do a fogbugz on >>> that?) >>> >>> The follow-on from a metacello image is how we can get people to create >>> better baselines that give you more minimal loading options (e.g. >>> conditionally leave out the test cases perhaps) >>> >>> Tim >>> >>> On 15 Jul 2017, at 08:24, Pavel Krivanek <pavel.krivanek(a)gmail.com> >>> wrote: >>> >>> Hi Tim, >>> >>> you can base the your work on the bootstrapped image, see >>> https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit/, file >>> Pharo7.0-core-*.zip >>> >>> This image does not have a lot of basic components like Monticello or >>> network but it has a compiler so the code can be imported as *.st files. >>> Then we have Pharo7.0-monticello-*.zip which will be easier to use and >>> probably can fit your needs. Monticello and network support are included. >>> But you cannot use baselines nor configurations to load your code. >>> >>> -- Pavel >>> >>> 2017-07-14 9:59 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>> >>>> Hi - buoyed by the success of a minimal image (thanks Pavel), I'm >>>> wondering if I can get even smaller. >>>> >>>> There are lots of .so's in the vm which wouldn't make sense on a server >>>> once deployed - sound, maybe libgit ... >>>> >>>> Is there a list of the essential ones, or tips on what I can strip out >>>> of the Linux deployment? I also recall that i can leave out .sources and >>>> .changes as well right? >>>> >>>> Tim >>>> >>>> Sent from my iPhone >>>> >>> >>> >> > >

July 19, 2017

Re: [Pharo-users] Problem to access the "book compilation farm"

by Cyril Ferlicot

On Wed, Jul 19, 2017 at 2:24 PM, Matteo via Pharo-users <pharo-users(a)lists.pharo.org> wrote: > > > ---------- Forwarded message ---------- > From: Matteo <matteob8(a)yahoo.it> > To: "Pharo is welcome (ML)" <pharo-users(a)lists.pharo.org> > Cc: > Bcc: > Date: Wed, 19 Jul 2017 14:24:18 +0200 > Subject: Problem to access the "book compilation farm" > Dears, > > I cannot accesses the site: > https://ci.inria.fr/pharo-contribution/view/Books/ > > Is there some problem with the server? > > Hi, The CI is down for an update. It should come back tomorrow or later this week. > Further, I saw serval "strange" messages on the main page of the > Pharo forum, http://forum.world.st/Pharo-f1294836.html. > > Should these messages be removed? > > > Thanks, > > Matteo. > > > -- Cyril Ferlicot https://ferlicot.fr http://www.synectique.eu 2 rue Jacques PrÃ©vert 01, 59650 Villeneuve d'ascq France

July 19, 2017

Problem to access the "book compilation farm"

by Matteo

Dears, I cannot accesses the site: https://ci.inria.fr/pharo-contribution/view/Books/ Is there some problem with the server? Further, I saw serval "strange" messages on the main page of the Pharo forum, http://forum.world.st/Pharo-f1294836.html. Should these messages be removed? Thanks, Matteo.

July 19, 2017

Re: [Pharo-users] Performance Testing Tools

by Mariano Martinez Peck

The ones I remember are Smark [1] and CalipeL [2] Cheers, [1] http://www.smalltalkhub.com/#!/~StefanMarr/SMark [2] https://bitbucket.org/janvrany/jv-calipel On Wed, Jul 19, 2017 at 4:17 AM, Luke Gorrie <luke(a)snabb.co> wrote: > Hi Evan, > > I am also really interesting in this topic and have been doing a bunch of > work on automating statistical benchmarks. I don't have a background in > statistics or formal QA but I am learning as I go along :). > > The tools I'm building are outside Smalltalk. Our full performance test > suite takes about a week of machine time to run because tests ~15,000 QEMU > VMs with different software versions / configurations / workloads. There is > a CI server that runs all those tests, getting pretty fast turnarounds by > distributing across a cluster of servers and reusing results from > unmodified software branches, and spits out a CSV with one row per test > result (giving the benchmark score and the parameters of the test.) > > Then what to do with that ~15,000 line CSV file? Just now I run Rmarkdown > to make a report on the distribution of results and then manually inspect > that to check for interesting differences. I lump all of the different > configurations in together and treat them as one population at the moment. > Here is an example report: > https://hydra.snabb.co/build/1604171/download/2/report.html > > It's a bit primitive but it is getting the job done for release > engineering. I'm reasonably confident that new software releases don't > break or slow down in obscure configurations. We are building network > equipment and performance regressions are generally not acceptable. > > I'm looking into more clever ways to automatically interpret the results, > e.g. fumbling around at https://stats.stackexchange. > com/questions/288416/non-parametric-test-if-two- > samples-are-drawn-from-the-same-distribution. > > Could relate to your ambitions somehow? > > > On 19 July 2017 at 02:00, Evan Donahue <emdonahu(a)gmail.com> wrote: > >> Hi, >> >> I've been doing a lot of performance testing lately, and I've found >> myself wanting to upgrade my methods from ad hoc use of bench and message >> tally. Is there any kind of framework for like, statistically comparing >> improvements in performance benchmarks across different versions of code, >> or anything that generally helps manage the test-tweak-test loop? Just >> curious what's out there before I go writing something. Too many useful >> little libraries to keep track of! >> >> Evan >> > > -- Mariano http://marianopeck.wordpress.com

July 19, 2017

[Pharo News] Free Ephemeric Cloud for Association Members

by Marcus Denker

Free Ephemeric Cloud for Association Members http://pharo.org/news/FreeEphemeric online: now.

July 19, 2017

Re: [Pharo-users] could not find module vm-display-X11

by Hilaire

Hi Alistair, Indeed installing this dependency makes Dr. Geo to work again, and the "could not find module vm-display-X11" error is gone. Thanks guys Hilaire Le 19/07/2017 Ã 10:22, Alistair Grant a Ã©crit : > > I'm in a bit of a rush, but as Nicolai says, it looks like LibGL.so.1 > is the problem. A google search should help, but maybe try: > > sudo apt-get install libgl1-mesa-glx:i386 > > Cheers, > Alistair > > -- Dr. Geo http://drgeo.eu

July 19, 2017

Re: [Pharo-users] could not find module vm-display-X11

by Alistair Grant

Hi Hilaire, On 19 July 2017 at 09:46, Hilaire <hilaire(a)drgeo.eu> wrote: > Hi, > > It does not work on a new installed system. It is related to the installed > system, other report it to me with the same system > > > Le 19/07/2017 Ã 00:01, Alistair Grant a Ã©crit : > > Is this something that was working and has suddenly stopped, or a new > install that doesn't work? > > Assuming that it is installed correctly, can you provide the output of: > > cd /to/where/the/pharo/exe/is > ldd pharo > > ldd pharo > linux-gate.so.1 => (0xf7777000) > libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7705000) > libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf7700000) > libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf76e2000) > libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf752c000) > /lib/ld-linux.so.2 (0x5655a000) > > ldd vm-display-X11 > > ldd vm-display-X11 > linux-gate.so.1 => (0xf773a000) > libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf76e4000) > libGL.so.1 => not found > libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf7598000) > libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf73e2000) > /lib/ld-linux.so.2 (0x56649000) > libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf73bc000) > libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf73b7000) > libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf73b3000) > libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf73ab000) I'm in a bit of a rush, but as Nicolai says, it looks like LibGL.so.1 is the problem. A google search should help, but maybe try: sudo apt-get install libgl1-mesa-glx:i386 Cheers, Alistair

July 19, 2017

Re: [Pharo-users] get output of a forked process on windows

by Christophe Demarey

Hi Thierry, > Le 18 juil. 2017 Ã 15:40, Thierry Goubier <thierry.goubier(a)gmail.com> a Ã©crit : > > Hi Christophe, > > You have to use ProcessWrapper. > > Metacello new > configuration: 'ProcessWrapper'; > repository: 'http://smalltalkhub.com/mc/Pharo/MetaRepoForPharo40/main <http://smalltalkhub.com/mc/Pharo/MetaRepoForPharo40/main>'; > load Thanks, it is working with the given example from http://smalltalkhub.com/#!/~hernan/ProcessWrapper/. Best regards, Christophe

July 19, 2017

Re: [Pharo-users] could not find module vm-display-X11

by Nicolai Hess

Looks like libgl is the issue Am 19.07.2017 9:47 vorm. schrieb "Hilaire" <hilaire(a)drgeo.eu>: > Hi, > > It does not work on a new installed system. It is related to the installed > system, other report it to me with the same system > > Le 19/07/2017 Ã 00:01, Alistair Grant a Ã©crit : > > Is this something that was working and has suddenly stopped, or a new > install that doesn't work? > > Assuming that it is installed correctly, can you provide the output of: > > cd /to/where/the/pharo/exe/is > ldd pharo > > ldd pharo > linux-gate.so.1 => (0xf7777000) > libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xf7705000) > libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf7700000) > libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf76e2000) > libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf752c000) > /lib/ld-linux.so.2 (0x5655a000) > > ldd vm-display-X11 > > ldd vm-display-X11 > linux-gate.so.1 => (0xf773a000) > libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xf76e4000) > libGL.so.1 => not found > libX11.so.6 => /usr/lib/i386-linux-gnu/libX11.so.6 (0xf7598000) > libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf73e2000) > /lib/ld-linux.so.2 (0x56649000) > libxcb.so.1 => /usr/lib/i386-linux-gnu/libxcb.so.1 (0xf73bc000) > libdl.so.2 => /lib/i386-linux-gnu/libdl.so.2 (0xf73b7000) > libXau.so.6 => /usr/lib/i386-linux-gnu/libXau.so.6 (0xf73b3000) > libXdmcp.so.6 => /usr/lib/i386-linux-gnu/libXdmcp.so.6 (0xf73ab000) > > > > > Cheers, > Alistair > > > -- > Dr. Geohttp://drgeo.eu > >

July 19, 2017