- Pharo-users - lists.pharo.org

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Guillermo Polito

Be careful because some of those aggressive cleanups may turn some parts of your image unstable. For example: This is dangerous: MCMethodDefinition allInstances do: [:each | each become: String new ]. MCClassDefinition allInstances do: [:each | each become: String new ]. MCVersionInfo allInstances do: [:each | each become: String new ]. And this may break some code if you're using non-ascii characters: Unicode classPool at: #GeneralCategory put: nil. Unicode classPool at: #DecimalProperty put: nil. On Wed, Aug 16, 2017 at 10:07 AM, Tim Mackinnon <tim(a)testit.works> wrote: > This is very encouraging, and also very instructive (your post on snapshot > is also one in this area too. I have a queue of things to try for you on > Friday). > > I'll try adding your ideas to my script and see if it squeezes some more. > > And then I guess we need to to decide which parts go into your minimum > build steps and which are an external script (or possibly a > HeadlessImageCleaner class we keep loaded so it's easier to maintain?) > > Tim > > Sent from my iPhone > > On 16 Aug 2017, at 08:52, Guillermo Polito <guillermopolito(a)gmail.com> > wrote: > > This means it would be healthy to do a cleanup (at least the non > aggressive one, ChangeSets and MC stuff) on each of the images we produce > and not just the latest one. > > On Wed, Aug 16, 2017 at 8:35 AM, Tim Mackinnon <tim(a)testit.works> wrote: > >> Yes you were on to something there (and at the same time, by poking >> around with #pointersTo I noticed some chains of objects too). So I ran the >> following script (partially borrowed from ImageCleaner) and this has got me >> down to a 14mb image (instance sizes listed below, which is looking much >> healthier - and those MethodChangeRecords are gone too) !!! >> >> I suspect there are more monti/metacello things that are still lurking >> around. >> >> I also wonder if I need some the character sorting strings too. >> >> Tim >> >> "CmdLine script to debug the initial minimal image" >> >> | logger repo version | >> >> logger := FileStream stderr. >> logger cr; nextPutAll: 'Starting Minimal Cleanup Script...'. >> >> logger cr; nextPutAll: '>Resetting Class Comments'. >> Smalltalk allClasses do: [ :c | c classComment: '' stamp: '' ]. >> >> logger cr; nextPutAll: '>Removing MC holders'. >> MCMethodDefinition allInstances do: [:each | each become: String new ]. >> MCClassDefinition allInstances do: [:each | each become: String new ]. >> MCVersionInfo allInstances do: [:each | each become: String new ]. >> >> logger cr; nextPutAll: '>ImageCleaner release routines'. >> Smalltalk organization removeEmptyCategories. >> Smalltalk >> allClassesAndTraitsDo: [ :class | >> [ :each | >> each >> removeEmptyCategories; >> sortCategories ] >> value: class organization; >> value: class class organization ]. >> >> (RPackageOrganizer default packages select: #isEmpty) >> do: #unregister. >> >> Smalltalk organization sortCategories. >> Smalltalk garbageCollect. >> Smalltalk cleanOutUndeclared. >> Smalltalk fixObsoleteReferences. >> Smalltalk cleanUp: true except: #() confirming: false. >> >> logger cr; nextPutAll: '>GC'. >> 3 timesRepeat: [ >> Smalltalk garbageCollect. >> Smalltalk cleanOutUndeclared. >> Smalltalk fixObsoleteReferences]. >> >> logger cr; nextPutAll: 'Finished Script.'; cr; cr. >> >> >> My top instances are now: >> >> Class code space # instances >> inst space percent inst average size >> CompiledMethod 19159 30481 >> 2912968 21.60 95.57 >> Array 3742 36495 >> 2852448 21.10 78.16 >> ByteString 2640 24018 >> 2517168 18.60 104.80 >> ByteSymbol 1698 20722 >> 759208 5.60 36.64 >> Association 1148 19786 >> 633152 4.70 32.00 >> IdentitySet 408 15452 >> 494464 3.70 32.00 >> MethodDictionary 3310 3520 >> 350192 2.60 99.49 >> Protocol 1679 8382 >> 268224 2.00 32.00 >> WeakArray 1758 265 >> 232304 1.70 876.62 >> OrderedCollection 6555 5043 >> 201720 1.50 40.00 >> ClassOrganization 5281 3520 >> 168960 1.30 48.00 >> Metaclass 7184 1748 >> 153824 1.10 88.00 >> >> >> >> >> On 15 Aug 2017, at 23:00, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> >> Just a hunch: could you inspect ur MethodChangeRecord instances ? >> >> Le mar. 15 aoÃ»t 2017 Ã 23:55, Tim Mackinnon <tim(a)testit.works> a Ã©crit : >> >>> A weird observation - is it possible that source code is being stored in >>> the image as strings somehow? When I do >>> >>> ./pharo PharoLambda.image eval "ByteString allInstances inject: >>> (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" >>> >>> I see to see reams of what looks like method source - but I thought >>> source code was stored in the .sources file and the .changes file (and I >>> havenât been bundling those in my deployed image). >>> >>> Iâm trying to figure out how you find references to a string object, to >>> chase down what is pointing to these strings as maybe there is a quick 4mb >>> win by simply nilâing out some obvious things. (This doesnât of course help >>> with a default minimal image - but maybe a few tricks for packaging and >>> deploying something). >>> >>> Tim >>> >>> On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: >>> >>> Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it >>> seems very convoluted to load a single package into the image, I was trying >>> to avoid having to create a baselineOf for something so simple - I ended up >>> with: >>> >>> repo := MCFileTreeRepository new directory: './bootstrap' >>> asFileReference. >>> >>> version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. >>> version load. >>> >>> >>> Anyway - in my minimal image, like in the fat image there seems to be a >>> surprising amount of bytestrings (4mb worth?). I think that might need some >>> digging into? It seems like a lot somehow. Although Benâs neat experiment >>> of zipping strings shows thatâs not a real route. >>> >>> In a deployed minimal image - maybe I can get rid of some other things >>> like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins >>> - but noticeable) >>> >>> Class code space # instances >>> inst space percent inst average size >>> ByteString 2640 37365 >>> 4823848 21.50 129.10 >>> Array 3742 53002 >>> 3961944 17.60 74.75 >>> CompiledMethod 19159 30481 >>> 2912968 13.00 95.57 >>> Association 1148 58348 >>> 1867136 8.30 32.00 >>> MethodChangeRecord 431 34312 >>> 1097984 4.90 32.00 >>> ByteArray 4605 290 >>> 908728 4.00 3133.54 >>> ByteSymbol 1698 22689 >>> 840168 3.70 37.03 >>> IdentitySet 408 19076 >>> 610432 2.70 32.00 >>> MethodDictionary 3310 3520 >>> 608688 2.70 172.92 >>> WeakArray 1758 3024 >>> 597824 2.70 197.69 >>> MCMethodDefinition 4318 6659 >>> 426176 1.90 64.00 >>> Protocol 1679 8382 >>> 268224 1.20 32.00 >>> OrderedCollection 6555 5509 >>> 220360 1.00 40.00 >>> >>> As an aside - my Gitlab project is public, the scripts that load things >>> up are in ./scripts (build.sh, and minimal.st and loadlocal.st) >>> >>> Tim >>> >>> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> >>> wrote: >>> >>> >>> >>> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: >>> >>>> Hi Guille - just running SpaceTally on my dev image to get a feel for >>>> it. It turns out that in the minimal images youâve been creating, its not >>>> loaded (makes sense). >>>> >>> >>> Yup, it's loaded afterwards. >>> >>> All packages are loaded through metacello baselines. We should start >>> refactoring and making standalone projects, each one with a baseline for >>> himself, and his own dependencies described. >>> >>> I was checking on your gitlab and I have probably no access: how are you >>> finally loading packages in the bootstrap image? Can you share that with us >>> in text? I'd like to improve that situation. >>> >>> >>>> Iâm wondering if there is an easy way to import it in (I guess that >>>> package should be in the Pharo git tree I cloned to get Fuel loaded right? >>>> Or is there a separate standalone source?). >>>> >>> >>> Yes it is, you can get the package programatically doing >>> >>> SpaceTally package name >>> >>> And furthermore, get the baseline that currently is loading by doing >>> >>> package := SpaceTally package name. >>> BaselineOf subclasses select: [ :e | >>> e project version packages anySatisfy: [ :p | p name = package ]]. >>> >>> >>>> >>>> Thanks for all the support, and your email about why the contexts stack >>>> up is very well received (I will comment over there). >>>> >>>> By the way - it looks like Martin Fowler picked up on this announcement >>>> - so maybe we might get some interest from his mass of followers. >>>> >>>> Tim >>>> >>>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> >>>> wrote: >>>> >>>> Hi Tim, >>>> >>>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> w >>>> rote: >>>> >>>>> Hey guys, thanks for your enthusiasm around this - and I cannot stress >>>>> enough how this was only possible because of the work that has gone into >>>>> making Pharo (in particular the 64bit image, as well as having a minimal >>>>> image, and some great blog posts on serialising contexts) as well as the >>>>> patience from everyone in answering questions and helping me get it all >>>>> working. >>>>> >>>>> Iâm still quite keen to get my execution time back down under 800ms >>>>> and Iâd like to actually get back to writing a few skills to automate a few >>>>> things around my house. >>>>> >>>>> To Answer Denisâ question - >>>>> >>>>> My final footprint is 30.4mb - thats composed of a 22mb image (with a >>>>> simple example that pulls in Fuel, ZTimestamp and the S3 Library which >>>>> depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>>>> >>>>> In my original experiments with a 6.0 minimal image - I did manage to >>>>> get to a 13.4mb image (which started out as 12mb original size, and then >>>>> loaded in STON and had only a simple clock example). I think the sweet spot >>>>> is around 20mb total footprint as that seems to get me into the 450ms-900ms >>>>> range. >>>>> >>>>> The 7.0 min image now starts out at 15mb and then Iâm not sure why >>>>> loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not >>>>> dug into that). >>>>> >>>> >>>> You can do further space analysis using the following expression >>>> >>>> SpaceTally new printSpaceAnalysis >>>> >>>> You can do that in an eval and check what's taking space. With measures >>>> we can iterate and improve :). >>>> >>>> >>>>> Iâve also found (and this on the back of unserialising the context in >>>>> my example) that the way we build images has 15+ saved stack sessions that >>>>> have saved on top of each other from the way we build up the images. I >>>>> donât yet know the implications of size/speed of these - but we need a >>>>> better way of folding executions when we snapshot headless images. Iâm also >>>>> not clear if there are any other startup tasks that take precious time >>>>> (this also has implications for our fat development images as they take >>>>> much longer to appear than they really should). >>>>> >>>> >>>> I'm working on this as I'm writing this mail ;) >>>> >>>> https://pharo.fogbugz.com/f/cases/20309 >>>> https://github.com/pharo-project/pharo/pull/196 >>>> >>>> I'll write down the implications further in a different thread. >>>> >>>> >>>>> Iâll be exploring some of these size/speed tradeoffâs in follow on >>>>> messages. >>>>> >>>>> But once again, a big thanks - Iâve not enjoyed programming like this >>>>> for ages. >>>>> >>>>> Tim >>>>> >>>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>>>> >>>>> hi Tim, >>>>> >>>>> That is..... AWESOME! >>>>> >>>>> Very nice delivery - it flowed well with great narration. >>>>> >>>>> I loved @2:17 "this is the interesting piece, because PharoLambda has >>>>> serialized the execution context of its application and saved it into [my >>>>> S3 bucket] ... [then on the local machine] rematerializes a debugger [on >>>>> that context]." >>>>> >>>>> There is a clarity in your video presentation that really may intrigue >>>>> outsiders. As a community we should push this on the usual hacker forums - >>>>> ycombinator could be a good starting point (but I'm locked out of my >>>>> account there). >>>>> An enticing title could be... >>>>> "Debugging Lambdas by re-materializing saved execution contexts on >>>>> your local machine." >>>>> >>>>> cheers -ben >>>>> >>>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.c >>>>> om> wrote: >>>>> >>>>>> This is cool Tim. >>>>>> >>>>>> So what image size you deployed at the end? >>>>>> >>>>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>>> >>>>>>> I just wanted to thank everyone for their help in getting my pet >>>>>>> project further along, so that now I can announce that PharoLambda is now >>>>>>> working with the V7 minimal image and also supports post mortem debugging >>>>>>> by saving a zipped fuel context onto S3. >>>>>>> >>>>>>> This latter item is particularly satisfying as at a recent >>>>>>> serverless conference (JeffConf) there was a panel where poor development >>>>>>> tools on serverless platforms was highlighted as a real problem. >>>>>>> >>>>>>> In our community weâve had these kinds of tools at our fingertips >>>>>>> for ages - but I donât think the wider development community has really >>>>>>> noticed. Debugging something short lived like a Lambda execution is quite >>>>>>> startling, as the current answer is âadd more loggingâ, and we all know >>>>>>> that sucks. To this end, Iâve created a little screencast showing this in >>>>>>> action - and it was pretty cool because it was a real example I encountered >>>>>>> when I got everything working and was trying my test application out. >>>>>>> >>>>>>> Iâve also put a bit of work into tuning the excellent GitLab CI >>>>>>> tools, so that I can cache many of the artefacts used between different >>>>>>> build runs (this might also be of interest to others using CI systems). >>>>>>> >>>>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> >>>>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>> >>>>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS >>>>>>> Lambda. Itâs early days, but I though it might be interesting to share what >>>>>>> Iâve learned so far. >>>>>>> >>>>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>>>> >>>>>>> With help from many of the folks here, Iâve been able to get a >>>>>>> simple example to run in 500ms-1200ms with a minimal Pharo 6 image. You can >>>>>>> easily try it out yourself. This seems slightly better than what the GoLang >>>>>>> folks have been able to do. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Guille Polito >>>> >>>> Research Engineer >>>> French National Center for Scientific Research - *http://www.cnrs.fr* >>>> <http://www.cnrs.fr/> >>>> >>>> >>>> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >>>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>>> >>>> >>>> >>> >>> >>> -- >>> >>> Guille Polito >>> >>> Research Engineer >>> French National Center for Scientific Research - *http://www.cnrs.fr* >>> <http://www.cnrs.fr/> >>> >>> >>> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>> >>> >>> >>> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - *http://www.cnrs.fr* >> <http://www.cnrs.fr/> >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> >> >> > > > -- > > > > Guille Polito > > > Research Engineer > > French National Center for Scientific Research - *http://www.cnrs.fr* > <http://www.cnrs.fr> > > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > > -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Aug. 16, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Tim Mackinnon

This is very encouraging, and also very instructive (your post on snapshot is also one in this area too. I have a queue of things to try for you on Friday). I'll try adding your ideas to my script and see if it squeezes some more. And then I guess we need to to decide which parts go into your minimum build steps and which are an external script (or possibly a HeadlessImageCleaner class we keep loaded so it's easier to maintain?) Tim Sent from my iPhone > On 16 Aug 2017, at 08:52, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > This means it would be healthy to do a cleanup (at least the non aggressive one, ChangeSets and MC stuff) on each of the images we produce and not just the latest one. > >> On Wed, Aug 16, 2017 at 8:35 AM, Tim Mackinnon <tim(a)testit.works> wrote: >> Yes you were on to something there (and at the same time, by poking around with #pointersTo I noticed some chains of objects too). So I ran the following script (partially borrowed from ImageCleaner) and this has got me down to a 14mb image (instance sizes listed below, which is looking much healthier - and those MethodChangeRecords are gone too) !!! >> >> I suspect there are more monti/metacello things that are still lurking around. >> >> I also wonder if I need some the character sorting strings too. >> >> Tim >> >> "CmdLine script to debug the initial minimal image" >> >> | logger repo version | >> >> logger := FileStream stderr. >> logger cr; nextPutAll: 'Starting Minimal Cleanup Script...'. >> >> logger cr; nextPutAll: '>Resetting Class Comments'. >> Smalltalk allClasses do: [ :c | c classComment: '' stamp: '' ]. >> >> logger cr; nextPutAll: '>Removing MC holders'. >> MCMethodDefinition allInstances do: [:each | each become: String new ]. >> MCClassDefinition allInstances do: [:each | each become: String new ]. >> MCVersionInfo allInstances do: [:each | each become: String new ]. >> >> logger cr; nextPutAll: '>ImageCleaner release routines'. >> Smalltalk organization removeEmptyCategories. >> Smalltalk >> allClassesAndTraitsDo: [ :class | >> [ :each | >> each >> removeEmptyCategories; >> sortCategories ] >> value: class organization; >> value: class class organization ]. >> >> (RPackageOrganizer default packages select: #isEmpty) >> do: #unregister. >> >> Smalltalk organization sortCategories. >> Smalltalk garbageCollect. >> Smalltalk cleanOutUndeclared. >> Smalltalk fixObsoleteReferences. >> Smalltalk cleanUp: true except: #() confirming: false. >> >> logger cr; nextPutAll: '>GC'. >> 3 timesRepeat: [ >> Smalltalk garbageCollect. >> Smalltalk cleanOutUndeclared. >> Smalltalk fixObsoleteReferences]. >> >> logger cr; nextPutAll: 'Finished Script.'; cr; cr. >> >> My top instances are now: >> >> Class code space # instances inst space percent inst average size >> CompiledMethod 19159 30481 2912968 21.60 95.57 >> Array 3742 36495 2852448 21.10 78.16 >> ByteString 2640 24018 2517168 18.60 104.80 >> ByteSymbol 1698 20722 759208 5.60 36.64 >> Association 1148 19786 633152 4.70 32.00 >> IdentitySet 408 15452 494464 3.70 32.00 >> MethodDictionary 3310 3520 350192 2.60 99.49 >> Protocol 1679 8382 268224 2.00 32.00 >> WeakArray 1758 265 232304 1.70 876.62 >> OrderedCollection 6555 5043 201720 1.50 40.00 >> ClassOrganization 5281 3520 168960 1.30 48.00 >> Metaclass 7184 1748 153824 1.10 88.00 >> >> >> >> >>> On 15 Aug 2017, at 23:00, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >>> >>> Just a hunch: could you inspect ur MethodChangeRecord instances ? >>> >>> Le mar. 15 aoÃ»t 2017 Ã 23:55, Tim Mackinnon <tim(a)testit.works> a Ã©crit : >>>> A weird observation - is it possible that source code is being stored in the image as strings somehow? When I do >>>> >>>> ./pharo PharoLambda.image eval "ByteString allInstances inject: (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" >>>> >>>> I see to see reams of what looks like method source - but I thought source code was stored in the .sources file and the .changes file (and I havenât been bundling those in my deployed image). >>>> >>>> Iâm trying to figure out how you find references to a string object, to chase down what is pointing to these strings as maybe there is a quick 4mb win by simply nilâing out some obvious things. (This doesnât of course help with a default minimal image - but maybe a few tricks for packaging and deploying something). >>>> >>>> Tim >>>> >>>>> On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: >>>>> >>>>> Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it seems very convoluted to load a single package into the image, I was trying to avoid having to create a baselineOf for something so simple - I ended up with: >>>>> >>>>> repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. >>>>> version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. >>>>> version load. >>>>> >>>>> Anyway - in my minimal image, like in the fat image there seems to be a surprising amount of bytestrings (4mb worth?). I think that might need some digging into? It seems like a lot somehow. Although Benâs neat experiment of zipping strings shows thatâs not a real route. >>>>> >>>>> In a deployed minimal image - maybe I can get rid of some other things like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins - but noticeable) >>>>> >>>>> Class code space # instances inst space percent inst average size >>>>> ByteString 2640 37365 4823848 21.50 129.10 >>>>> Array 3742 53002 3961944 17.60 74.75 >>>>> CompiledMethod 19159 30481 2912968 13.00 95.57 >>>>> Association 1148 58348 1867136 8.30 32.00 >>>>> MethodChangeRecord 431 34312 1097984 4.90 32.00 >>>>> ByteArray 4605 290 908728 4.00 3133.54 >>>>> ByteSymbol 1698 22689 840168 3.70 37.03 >>>>> IdentitySet 408 19076 610432 2.70 32.00 >>>>> MethodDictionary 3310 3520 608688 2.70 172.92 >>>>> WeakArray 1758 3024 597824 2.70 197.69 >>>>> MCMethodDefinition 4318 6659 426176 1.90 64.00 >>>>> Protocol 1679 8382 268224 1.20 32.00 >>>>> OrderedCollection 6555 5509 220360 1.00 40.00 >>>>> >>>>> As an aside - my Gitlab project is public, the scripts that load things up are in ./scripts (build.sh, and minimal.st and loadlocal.st) >>>>> >>>>> Tim >>>>> >>>>>> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>> Hi Guille - just running SpaceTally on my dev image to get a feel for it. It turns out that in the minimal images youâve been creating, its not loaded (makes sense). >>>>>> >>>>>> Yup, it's loaded afterwards. >>>>>> >>>>>> All packages are loaded through metacello baselines. We should start refactoring and making standalone projects, each one with a baseline for himself, and his own dependencies described. >>>>>> >>>>>> I was checking on your gitlab and I have probably no access: how are you finally loading packages in the bootstrap image? Can you share that with us in text? I'd like to improve that situation. >>>>>> >>>>>>> Iâm wondering if there is an easy way to import it in (I guess that package should be in the Pharo git tree I cloned to get Fuel loaded right? Or is there a separate standalone source?). >>>>>> >>>>>> Yes it is, you can get the package programatically doing >>>>>> >>>>>> SpaceTally package name >>>>>> >>>>>> And furthermore, get the baseline that currently is loading by doing >>>>>> >>>>>> package := SpaceTally package name. >>>>>> BaselineOf subclasses select: [ :e | >>>>>> e project version packages anySatisfy: [ :p | p name = package ]]. >>>>>> >>>>>>> >>>>>>> Thanks for all the support, and your email about why the contexts stack up is very well received (I will comment over there). >>>>>>> >>>>>>> By the way - it looks like Martin Fowler picked up on this announcement - so maybe we might get some interest from his mass of followers. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >>>>>>>> >>>>>>>> Hi Tim, >>>>>>>> >>>>>>>>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>>>> Hey guys, thanks for your enthusiasm around this - and I cannot stress enough how this was only possible because of the work that has gone into making Pharo (in particular the 64bit image, as well as having a minimal image, and some great blog posts on serialising contexts) as well as the patience from everyone in answering questions and helping me get it all working. >>>>>>>>> >>>>>>>>> Iâm still quite keen to get my execution time back down under 800ms and Iâd like to actually get back to writing a few skills to automate a few things around my house. >>>>>>>>> >>>>>>>>> To Answer Denisâ question - >>>>>>>>> >>>>>>>>> My final footprint is 30.4mb - thats composed of a 22mb image (with a simple example that pulls in Fuel, ZTimestamp and the S3 Library which depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>>>>>>>> >>>>>>>>> In my original experiments with a 6.0 minimal image - I did manage to get to a 13.4mb image (which started out as 12mb original size, and then loaded in STON and had only a simple clock example). I think the sweet spot is around 20mb total footprint as that seems to get me into the 450ms-900ms range. >>>>>>>>> >>>>>>>>> The 7.0 min image now starts out at 15mb and then Iâm not sure why loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not dug into that). >>>>>>>> >>>>>>>> You can do further space analysis using the following expression >>>>>>>> >>>>>>>> SpaceTally new printSpaceAnalysis >>>>>>>> >>>>>>>> You can do that in an eval and check what's taking space. With measures we can iterate and improve :). >>>>>>>> >>>>>>>>> Iâve also found (and this on the back of unserialising the context in my example) that the way we build images has 15+ saved stack sessions that have saved on top of each other from the way we build up the images. I donât yet know the implications of size/speed of these - but we need a better way of folding executions when we snapshot headless images. Iâm also not clear if there are any other startup tasks that take precious time (this also has implications for our fat development images as they take much longer to appear than they really should). >>>>>>>> >>>>>>>> I'm working on this as I'm writing this mail ;) >>>>>>>> >>>>>>>> https://pharo.fogbugz.com/f/cases/20309 >>>>>>>> https://github.com/pharo-project/pharo/pull/196 >>>>>>>> >>>>>>>> I'll write down the implications further in a different thread. >>>>>>>> >>>>>>>>> >>>>>>>>> Iâll be exploring some of these size/speed tradeoffâs in follow on messages. >>>>>>>>> >>>>>>>>> But once again, a big thanks - Iâve not enjoyed programming like this for ages. >>>>>>>>> >>>>>>>>> Tim >>>>>>>>> >>>>>>>>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>>>>>>>>> >>>>>>>>>> hi Tim, >>>>>>>>>> >>>>>>>>>> That is..... AWESOME! >>>>>>>>>> >>>>>>>>>> Very nice delivery - it flowed well with great narration. >>>>>>>>>> >>>>>>>>>> I loved @2:17 "this is the interesting piece, because PharoLambda has serialized the execution context of its application and saved it into [my S3 bucket] ... [then on the local machine] rematerializes a debugger [on that context]." >>>>>>>>>> >>>>>>>>>> There is a clarity in your video presentation that really may intrigue outsiders. As a community we should push this on the usual hacker forums - ycombinator could be a good starting point (but I'm locked out of my account there). >>>>>>>>>> An enticing title could be... >>>>>>>>>> "Debugging Lambdas by re-materializing saved execution contexts on your local machine." >>>>>>>>>> >>>>>>>>>> cheers -ben >>>>>>>>>> >>>>>>>>>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>>>>>>>>>> This is cool Tim. >>>>>>>>>>> >>>>>>>>>>> So what image size you deployed at the end? >>>>>>>>>>> >>>>>>>>>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>>>>>>>>> I just wanted to thank everyone for their help in getting my pet project further along, so that now I can announce that PharoLambda is now working with the V7 minimal image and also supports post mortem debugging by saving a zipped fuel context onto S3. >>>>>>>>>>>> >>>>>>>>>>>> This latter item is particularly satisfying as at a recent serverless conference (JeffConf) there was a panel where poor development tools on serverless platforms was highlighted as a real problem. >>>>>>>>>>>> >>>>>>>>>>>> In our community weâve had these kinds of tools at our fingertips for ages - but I donât think the wider development community has really noticed. Debugging something short lived like a Lambda execution is quite startling, as the current answer is âadd more loggingâ, and we all know that sucks. To this end, Iâve created a little screencast showing this in action - and it was pretty cool because it was a real example I encountered when I got everything working and was trying my test application out. >>>>>>>>>>>> >>>>>>>>>>>> Iâve also put a bit of work into tuning the excellent GitLab CI tools, so that I can cache many of the artefacts used between different build runs (this might also be of interest to others using CI systems). >>>>>>>>>>>> >>>>>>>>>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>>>>>>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>>>>>>>>> >>>>>>>>>>>> Tim >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS Lambda. Itâs early days, but I though it might be interesting to share what Iâve learned so far. >>>>>>>>>>>>> >>>>>>>>>>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>>>>>>>>>> >>>>>>>>>>>>> With help from many of the folks here, Iâve been able to get a simple example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily try it out yourself. This seems slightly better than what the GoLang folks have been able to do. >>>>>>>>>>>>> >>>>>>>>>>>>> Tim >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> Guille Polito >>>>>>>> >>>>>>>> Research Engineer >>>>>>>> French National Center for Scientific Research - http://www.cnrs.fr >>>>>>>> >>>>>>>> >>>>>>>> Web: http://guillep.github.io >>>>>>>> Phone: +33 06 52 70 66 13 >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Guille Polito >>>>>> >>>>>> Research Engineer >>>>>> French National Center for Scientific Research - http://www.cnrs.fr >>>>>> >>>>>> >>>>>> Web: http://guillep.github.io >>>>>> Phone: +33 06 52 70 66 13 >>>>> >>>> >>> >>> -- >>> >>> Guille Polito >>> >>> Research Engineer >>> French National Center for Scientific Research - http://www.cnrs.fr >>> >>> >>> Web: http://guillep.github.io >>> Phone: +33 06 52 70 66 13 >> > > > > -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - http://www.cnrs.fr > > > Web: http://guillep.github.io > Phone: +33 06 52 70 66 13

Aug. 16, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Guillermo Polito

This means it would be healthy to do a cleanup (at least the non aggressive one, ChangeSets and MC stuff) on each of the images we produce and not just the latest one. On Wed, Aug 16, 2017 at 8:35 AM, Tim Mackinnon <tim(a)testit.works> wrote: > Yes you were on to something there (and at the same time, by poking around > with #pointersTo I noticed some chains of objects too). So I ran the > following script (partially borrowed from ImageCleaner) and this has got me > down to a 14mb image (instance sizes listed below, which is looking much > healthier - and those MethodChangeRecords are gone too) !!! > > I suspect there are more monti/metacello things that are still lurking > around. > > I also wonder if I need some the character sorting strings too. > > Tim > > "CmdLine script to debug the initial minimal image" > > | logger repo version | > > logger := FileStream stderr. > logger cr; nextPutAll: 'Starting Minimal Cleanup Script...'. > > logger cr; nextPutAll: '>Resetting Class Comments'. > Smalltalk allClasses do: [ :c | c classComment: '' stamp: '' ]. > > logger cr; nextPutAll: '>Removing MC holders'. > MCMethodDefinition allInstances do: [:each | each become: String new ]. > MCClassDefinition allInstances do: [:each | each become: String new ]. > MCVersionInfo allInstances do: [:each | each become: String new ]. > > logger cr; nextPutAll: '>ImageCleaner release routines'. > Smalltalk organization removeEmptyCategories. > Smalltalk > allClassesAndTraitsDo: [ :class | > [ :each | > each > removeEmptyCategories; > sortCategories ] > value: class organization; > value: class class organization ]. > > (RPackageOrganizer default packages select: #isEmpty) > do: #unregister. > > Smalltalk organization sortCategories. > Smalltalk garbageCollect. > Smalltalk cleanOutUndeclared. > Smalltalk fixObsoleteReferences. > Smalltalk cleanUp: true except: #() confirming: false. > > logger cr; nextPutAll: '>GC'. > 3 timesRepeat: [ > Smalltalk garbageCollect. > Smalltalk cleanOutUndeclared. > Smalltalk fixObsoleteReferences]. > > logger cr; nextPutAll: 'Finished Script.'; cr; cr. > > > My top instances are now: > > Class code space # instances > inst space percent inst average size > CompiledMethod 19159 30481 > 2912968 21.60 95.57 > Array 3742 36495 > 2852448 21.10 78.16 > ByteString 2640 24018 > 2517168 18.60 104.80 > ByteSymbol 1698 20722 > 759208 5.60 36.64 > Association 1148 19786 > 633152 4.70 32.00 > IdentitySet 408 15452 > 494464 3.70 32.00 > MethodDictionary 3310 3520 > 350192 2.60 99.49 > Protocol 1679 8382 > 268224 2.00 32.00 > WeakArray 1758 265 > 232304 1.70 876.62 > OrderedCollection 6555 5043 > 201720 1.50 40.00 > ClassOrganization 5281 3520 > 168960 1.30 48.00 > Metaclass 7184 1748 > 153824 1.10 88.00 > > > > > On 15 Aug 2017, at 23:00, Guillermo Polito <guillermopolito(a)gmail.com> > wrote: > > Just a hunch: could you inspect ur MethodChangeRecord instances ? > > Le mar. 15 aoÃ»t 2017 Ã 23:55, Tim Mackinnon <tim(a)testit.works> a Ã©crit : > >> A weird observation - is it possible that source code is being stored in >> the image as strings somehow? When I do >> >> ./pharo PharoLambda.image eval "ByteString allInstances inject: >> (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" >> >> I see to see reams of what looks like method source - but I thought >> source code was stored in the .sources file and the .changes file (and I >> havenât been bundling those in my deployed image). >> >> Iâm trying to figure out how you find references to a string object, to >> chase down what is pointing to these strings as maybe there is a quick 4mb >> win by simply nilâing out some obvious things. (This doesnât of course help >> with a default minimal image - but maybe a few tricks for packaging and >> deploying something). >> >> Tim >> >> On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: >> >> Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it >> seems very convoluted to load a single package into the image, I was trying >> to avoid having to create a baselineOf for something so simple - I ended up >> with: >> >> repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. >> >> version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. >> version load. >> >> >> Anyway - in my minimal image, like in the fat image there seems to be a >> surprising amount of bytestrings (4mb worth?). I think that might need some >> digging into? It seems like a lot somehow. Although Benâs neat experiment >> of zipping strings shows thatâs not a real route. >> >> In a deployed minimal image - maybe I can get rid of some other things >> like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins >> - but noticeable) >> >> Class code space # instances >> inst space percent inst average size >> ByteString 2640 37365 >> 4823848 21.50 129.10 >> Array 3742 53002 >> 3961944 17.60 74.75 >> CompiledMethod 19159 30481 >> 2912968 13.00 95.57 >> Association 1148 58348 >> 1867136 8.30 32.00 >> MethodChangeRecord 431 34312 >> 1097984 4.90 32.00 >> ByteArray 4605 290 >> 908728 4.00 3133.54 >> ByteSymbol 1698 22689 >> 840168 3.70 37.03 >> IdentitySet 408 19076 >> 610432 2.70 32.00 >> MethodDictionary 3310 3520 >> 608688 2.70 172.92 >> WeakArray 1758 3024 >> 597824 2.70 197.69 >> MCMethodDefinition 4318 6659 >> 426176 1.90 64.00 >> Protocol 1679 8382 >> 268224 1.20 32.00 >> OrderedCollection 6555 5509 >> 220360 1.00 40.00 >> >> As an aside - my Gitlab project is public, the scripts that load things >> up are in ./scripts (build.sh, and minimal.st and loadlocal.st) >> >> Tim >> >> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> >> >> >> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: >> >>> Hi Guille - just running SpaceTally on my dev image to get a feel for >>> it. It turns out that in the minimal images youâve been creating, its not >>> loaded (makes sense). >>> >> >> Yup, it's loaded afterwards. >> >> All packages are loaded through metacello baselines. We should start >> refactoring and making standalone projects, each one with a baseline for >> himself, and his own dependencies described. >> >> I was checking on your gitlab and I have probably no access: how are you >> finally loading packages in the bootstrap image? Can you share that with us >> in text? I'd like to improve that situation. >> >> >>> Iâm wondering if there is an easy way to import it in (I guess that >>> package should be in the Pharo git tree I cloned to get Fuel loaded right? >>> Or is there a separate standalone source?). >>> >> >> Yes it is, you can get the package programatically doing >> >> SpaceTally package name >> >> And furthermore, get the baseline that currently is loading by doing >> >> package := SpaceTally package name. >> BaselineOf subclasses select: [ :e | >> e project version packages anySatisfy: [ :p | p name = package ]]. >> >> >>> >>> Thanks for all the support, and your email about why the contexts stack >>> up is very well received (I will comment over there). >>> >>> By the way - it looks like Martin Fowler picked up on this announcement >>> - so maybe we might get some interest from his mass of followers. >>> >>> Tim >>> >>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> >>> wrote: >>> >>> Hi Tim, >>> >>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> w >>> rote: >>> >>>> Hey guys, thanks for your enthusiasm around this - and I cannot stress >>>> enough how this was only possible because of the work that has gone into >>>> making Pharo (in particular the 64bit image, as well as having a minimal >>>> image, and some great blog posts on serialising contexts) as well as the >>>> patience from everyone in answering questions and helping me get it all >>>> working. >>>> >>>> Iâm still quite keen to get my execution time back down under 800ms and >>>> Iâd like to actually get back to writing a few skills to automate a few >>>> things around my house. >>>> >>>> To Answer Denisâ question - >>>> >>>> My final footprint is 30.4mb - thats composed of a 22mb image (with a >>>> simple example that pulls in Fuel, ZTimestamp and the S3 Library which >>>> depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>>> >>>> In my original experiments with a 6.0 minimal image - I did manage to >>>> get to a 13.4mb image (which started out as 12mb original size, and then >>>> loaded in STON and had only a simple clock example). I think the sweet spot >>>> is around 20mb total footprint as that seems to get me into the 450ms-900ms >>>> range. >>>> >>>> The 7.0 min image now starts out at 15mb and then Iâm not sure why >>>> loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not >>>> dug into that). >>>> >>> >>> You can do further space analysis using the following expression >>> >>> SpaceTally new printSpaceAnalysis >>> >>> You can do that in an eval and check what's taking space. With measures >>> we can iterate and improve :). >>> >>> >>>> Iâve also found (and this on the back of unserialising the context in >>>> my example) that the way we build images has 15+ saved stack sessions that >>>> have saved on top of each other from the way we build up the images. I >>>> donât yet know the implications of size/speed of these - but we need a >>>> better way of folding executions when we snapshot headless images. Iâm also >>>> not clear if there are any other startup tasks that take precious time >>>> (this also has implications for our fat development images as they take >>>> much longer to appear than they really should). >>>> >>> >>> I'm working on this as I'm writing this mail ;) >>> >>> https://pharo.fogbugz.com/f/cases/20309 >>> https://github.com/pharo-project/pharo/pull/196 >>> >>> I'll write down the implications further in a different thread. >>> >>> >>>> Iâll be exploring some of these size/speed tradeoffâs in follow on >>>> messages. >>>> >>>> But once again, a big thanks - Iâve not enjoyed programming like this >>>> for ages. >>>> >>>> Tim >>>> >>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>>> >>>> hi Tim, >>>> >>>> That is..... AWESOME! >>>> >>>> Very nice delivery - it flowed well with great narration. >>>> >>>> I loved @2:17 "this is the interesting piece, because PharoLambda has >>>> serialized the execution context of its application and saved it into [my >>>> S3 bucket] ... [then on the local machine] rematerializes a debugger [on >>>> that context]." >>>> >>>> There is a clarity in your video presentation that really may intrigue >>>> outsiders. As a community we should push this on the usual hacker forums - >>>> ycombinator could be a good starting point (but I'm locked out of my >>>> account there). >>>> An enticing title could be... >>>> "Debugging Lambdas by re-materializing saved execution contexts on your >>>> local machine." >>>> >>>> cheers -ben >>>> >>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com >>>> > wrote: >>>> >>>>> This is cool Tim. >>>>> >>>>> So what image size you deployed at the end? >>>>> >>>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>> >>>>>> I just wanted to thank everyone for their help in getting my pet >>>>>> project further along, so that now I can announce that PharoLambda is now >>>>>> working with the V7 minimal image and also supports post mortem debugging >>>>>> by saving a zipped fuel context onto S3. >>>>>> >>>>>> This latter item is particularly satisfying as at a recent serverless >>>>>> conference (JeffConf) there was a panel where poor development tools on >>>>>> serverless platforms was highlighted as a real problem. >>>>>> >>>>>> In our community weâve had these kinds of tools at our fingertips for >>>>>> ages - but I donât think the wider development community has really >>>>>> noticed. Debugging something short lived like a Lambda execution is quite >>>>>> startling, as the current answer is âadd more loggingâ, and we all know >>>>>> that sucks. To this end, Iâve created a little screencast showing this in >>>>>> action - and it was pretty cool because it was a real example I encountered >>>>>> when I got everything working and was trying my test application out. >>>>>> >>>>>> Iâve also put a bit of work into tuning the excellent GitLab CI >>>>>> tools, so that I can cache many of the artefacts used between different >>>>>> build runs (this might also be of interest to others using CI systems). >>>>>> >>>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>> >>>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS >>>>>> Lambda. Itâs early days, but I though it might be interesting to share what >>>>>> Iâve learned so far. >>>>>> >>>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>>> >>>>>> With help from many of the folks here, Iâve been able to get a simple >>>>>> example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily >>>>>> try it out yourself. This seems slightly better than what the GoLang folks >>>>>> have been able to do. >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >>> -- >>> >>> Guille Polito >>> >>> Research Engineer >>> French National Center for Scientific Research - *http://www.cnrs.fr* >>> <http://www.cnrs.fr/> >>> >>> >>> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>> >>> >>> >> >> >> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - *http://www.cnrs.fr* >> <http://www.cnrs.fr/> >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> >> >> >> -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - *http://www.cnrs.fr* > <http://www.cnrs.fr/> > > > *Web:* *http://guillep.github.io* <http://guillep.github.io/> > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > > > -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Aug. 16, 2017

Re: [Pharo-users] Glamour: update presenter with new text

by Tudor Girba

Hi Hilaire, Sorry for the late reply. Indeed, it should be possible to do this, but we have to look closer at your concrete code. Here is an example: browser := GLMTabulator new. browser column: #one; column: #two. browser transmit to: #one; andShow: [ :a | a list display: [ :x | 1 to: x ] ]. browser transmit from: #one; to: #two; andShow: [ :a | a text ]. browser transmit fromOutsidePort: #explicitSelection; to: #one port: #selection. browser openOn: 42. (browser pane port: #explicitSelection) value: 41 Essentially, you need to transmit the selection from the outside to the inside pane. Please note that once you pass the #explicitSelection to the #selection of pane #one, you also trigger the transmission from #one to #two, and this is why the 41 also appears in the text on the right hand side. Does this help? In the meantime, have you read this chapter: http://www.themoosebook.org/book/index.html#h1buildingbrowserswithglamour ? Cheers, Doru > On Jul 15, 2017, at 10:38 AM, Hilaire <hilaire(a)drgeo.eu> wrote: > > Thanks for the updates on both the internal dependencies and the ListPresentation. > > From this browser, I deduced a second simpler browser for only one script. It was easy to do so thanks to your tips (attached pic). > > With this simpler script editor, when I open a script, I like it opens on the compute method, because it is a mandatory method in a script. > > I was doing it like > > browser := Nautilus openOnClass: scriptClass selector: #compute. > > I guess, there is a way to do so with the Glamour browser ? > > Thanks > > Hilaire > > -- > Dr. Geo > > http://drgeo.eu > <Edit script: Marque sur segment.gif> -- www.tudorgirba.com www.feenk.com "Every now and then stop and ask yourself if the war you're fighting is the right one."

Aug. 16, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Tim Mackinnon

Yes you were on to something there (and at the same time, by poking around with #pointersTo I noticed some chains of objects too). So I ran the following script (partially borrowed from ImageCleaner) and this has got me down to a 14mb image (instance sizes listed below, which is looking much healthier - and those MethodChangeRecords are gone too) !!! I suspect there are more monti/metacello things that are still lurking around. I also wonder if I need some the character sorting strings too. Tim "CmdLine script to debug the initial minimal image" | logger repo version | logger := FileStream stderr. logger cr; nextPutAll: 'Starting Minimal Cleanup Script...'. logger cr; nextPutAll: '>Resetting Class Comments'. Smalltalk allClasses do: [ :c | c classComment: '' stamp: '' ]. logger cr; nextPutAll: '>Removing MC holders'. MCMethodDefinition allInstances do: [:each | each become: String new ]. MCClassDefinition allInstances do: [:each | each become: String new ]. MCVersionInfo allInstances do: [:each | each become: String new ]. logger cr; nextPutAll: '>ImageCleaner release routines'. Smalltalk organization removeEmptyCategories. Smalltalk allClassesAndTraitsDo: [ :class | [ :each | each removeEmptyCategories; sortCategories ] value: class organization; value: class class organization ]. (RPackageOrganizer default packages select: #isEmpty) do: #unregister. Smalltalk organization sortCategories. Smalltalk garbageCollect. Smalltalk cleanOutUndeclared. Smalltalk fixObsoleteReferences. Smalltalk cleanUp: true except: #() confirming: false. logger cr; nextPutAll: '>GC'. 3 timesRepeat: [ Smalltalk garbageCollect. Smalltalk cleanOutUndeclared. Smalltalk fixObsoleteReferences]. logger cr; nextPutAll: 'Finished Script.'; cr; cr. My top instances are now: Class code space # instances inst space percent inst average size CompiledMethod 19159 30481 2912968 21.60 95.57 Array 3742 36495 2852448 21.10 78.16 ByteString 2640 24018 2517168 18.60 104.80 ByteSymbol 1698 20722 759208 5.60 36.64 Association 1148 19786 633152 4.70 32.00 IdentitySet 408 15452 494464 3.70 32.00 MethodDictionary 3310 3520 350192 2.60 99.49 Protocol 1679 8382 268224 2.00 32.00 WeakArray 1758 265 232304 1.70 876.62 OrderedCollection 6555 5043 201720 1.50 40.00 ClassOrganization 5281 3520 168960 1.30 48.00 Metaclass 7184 1748 153824 1.10 88.00 > On 15 Aug 2017, at 23:00, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > Just a hunch: could you inspect ur MethodChangeRecord instances ? > > Le mar. 15 aoÃ»t 2017 Ã 23:55, Tim Mackinnon <tim(a)testit.works> a Ã©crit : >> A weird observation - is it possible that source code is being stored in the image as strings somehow? When I do >> >> ./pharo PharoLambda.image eval "ByteString allInstances inject: (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" >> >> I see to see reams of what looks like method source - but I thought source code was stored in the .sources file and the .changes file (and I havenât been bundling those in my deployed image). >> >> Iâm trying to figure out how you find references to a string object, to chase down what is pointing to these strings as maybe there is a quick 4mb win by simply nilâing out some obvious things. (This doesnât of course help with a default minimal image - but maybe a few tricks for packaging and deploying something). >> >> Tim >> >>> On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: >>> >>> Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it seems very convoluted to load a single package into the image, I was trying to avoid having to create a baselineOf for something so simple - I ended up with: >>> >>> repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. >>> version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. >>> version load. >>> >>> Anyway - in my minimal image, like in the fat image there seems to be a surprising amount of bytestrings (4mb worth?). I think that might need some digging into? It seems like a lot somehow. Although Benâs neat experiment of zipping strings shows thatâs not a real route. >>> >>> In a deployed minimal image - maybe I can get rid of some other things like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins - but noticeable) >>> >>> Class code space # instances inst space percent inst average size >>> ByteString 2640 37365 4823848 21.50 129.10 >>> Array 3742 53002 3961944 17.60 74.75 >>> CompiledMethod 19159 30481 2912968 13.00 95.57 >>> Association 1148 58348 1867136 8.30 32.00 >>> MethodChangeRecord 431 34312 1097984 4.90 32.00 >>> ByteArray 4605 290 908728 4.00 3133.54 >>> ByteSymbol 1698 22689 840168 3.70 37.03 >>> IdentitySet 408 19076 610432 2.70 32.00 >>> MethodDictionary 3310 3520 608688 2.70 172.92 >>> WeakArray 1758 3024 597824 2.70 197.69 >>> MCMethodDefinition 4318 6659 426176 1.90 64.00 >>> Protocol 1679 8382 268224 1.20 32.00 >>> OrderedCollection 6555 5509 220360 1.00 40.00 >>> >>> As an aside - my Gitlab project is public, the scripts that load things up are in ./scripts (build.sh, and minimal.st and loadlocal.st) >>> >>> Tim >>> >>>> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >>>> >>>> >>>> >>>> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: >>>>> Hi Guille - just running SpaceTally on my dev image to get a feel for it. It turns out that in the minimal images youâve been creating, its not loaded (makes sense). >>>> >>>> Yup, it's loaded afterwards. >>>> >>>> All packages are loaded through metacello baselines. We should start refactoring and making standalone projects, each one with a baseline for himself, and his own dependencies described. >>>> >>>> I was checking on your gitlab and I have probably no access: how are you finally loading packages in the bootstrap image? Can you share that with us in text? I'd like to improve that situation. >>>> >>>>> Iâm wondering if there is an easy way to import it in (I guess that package should be in the Pharo git tree I cloned to get Fuel loaded right? Or is there a separate standalone source?). >>>> >>>> Yes it is, you can get the package programatically doing >>>> >>>> SpaceTally package name >>>> >>>> And furthermore, get the baseline that currently is loading by doing >>>> >>>> package := SpaceTally package name. >>>> BaselineOf subclasses select: [ :e | >>>> e project version packages anySatisfy: [ :p | p name = package ]]. >>>> >>>>> >>>>> Thanks for all the support, and your email about why the contexts stack up is very well received (I will comment over there). >>>>> >>>>> By the way - it looks like Martin Fowler picked up on this announcement - so maybe we might get some interest from his mass of followers. >>>>> >>>>> Tim >>>>> >>>>>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >>>>>> >>>>>> Hi Tim, >>>>>> >>>>>>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>> Hey guys, thanks for your enthusiasm around this - and I cannot stress enough how this was only possible because of the work that has gone into making Pharo (in particular the 64bit image, as well as having a minimal image, and some great blog posts on serialising contexts) as well as the patience from everyone in answering questions and helping me get it all working. >>>>>>> >>>>>>> Iâm still quite keen to get my execution time back down under 800ms and Iâd like to actually get back to writing a few skills to automate a few things around my house. >>>>>>> >>>>>>> To Answer Denisâ question - >>>>>>> >>>>>>> My final footprint is 30.4mb - thats composed of a 22mb image (with a simple example that pulls in Fuel, ZTimestamp and the S3 Library which depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>>>>>> >>>>>>> In my original experiments with a 6.0 minimal image - I did manage to get to a 13.4mb image (which started out as 12mb original size, and then loaded in STON and had only a simple clock example). I think the sweet spot is around 20mb total footprint as that seems to get me into the 450ms-900ms range. >>>>>>> >>>>>>> The 7.0 min image now starts out at 15mb and then Iâm not sure why loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not dug into that). >>>>>> >>>>>> You can do further space analysis using the following expression >>>>>> >>>>>> SpaceTally new printSpaceAnalysis >>>>>> >>>>>> You can do that in an eval and check what's taking space. With measures we can iterate and improve :). >>>>>> >>>>>>> Iâve also found (and this on the back of unserialising the context in my example) that the way we build images has 15+ saved stack sessions that have saved on top of each other from the way we build up the images. I donât yet know the implications of size/speed of these - but we need a better way of folding executions when we snapshot headless images. Iâm also not clear if there are any other startup tasks that take precious time (this also has implications for our fat development images as they take much longer to appear than they really should). >>>>>> >>>>>> I'm working on this as I'm writing this mail ;) >>>>>> >>>>>> https://pharo.fogbugz.com/f/cases/20309 >>>>>> https://github.com/pharo-project/pharo/pull/196 >>>>>> >>>>>> I'll write down the implications further in a different thread. >>>>>> >>>>>>> >>>>>>> Iâll be exploring some of these size/speed tradeoffâs in follow on messages. >>>>>>> >>>>>>> But once again, a big thanks - Iâve not enjoyed programming like this for ages. >>>>>>> >>>>>>> Tim >>>>>>> >>>>>>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>>>>>>> >>>>>>>> hi Tim, >>>>>>>> >>>>>>>> That is..... AWESOME! >>>>>>>> >>>>>>>> Very nice delivery - it flowed well with great narration. >>>>>>>> >>>>>>>> I loved @2:17 "this is the interesting piece, because PharoLambda has serialized the execution context of its application and saved it into [my S3 bucket] ... [then on the local machine] rematerializes a debugger [on that context]." >>>>>>>> >>>>>>>> There is a clarity in your video presentation that really may intrigue outsiders. As a community we should push this on the usual hacker forums - ycombinator could be a good starting point (but I'm locked out of my account there). >>>>>>>> An enticing title could be... >>>>>>>> "Debugging Lambdas by re-materializing saved execution contexts on your local machine." >>>>>>>> >>>>>>>> cheers -ben >>>>>>>> >>>>>>>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: >>>>>>>>> This is cool Tim. >>>>>>>>> >>>>>>>>> So what image size you deployed at the end? >>>>>>>>> >>>>>>>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>>>>>>> I just wanted to thank everyone for their help in getting my pet project further along, so that now I can announce that PharoLambda is now working with the V7 minimal image and also supports post mortem debugging by saving a zipped fuel context onto S3. >>>>>>>>>> >>>>>>>>>> This latter item is particularly satisfying as at a recent serverless conference (JeffConf) there was a panel where poor development tools on serverless platforms was highlighted as a real problem. >>>>>>>>>> >>>>>>>>>> In our community weâve had these kinds of tools at our fingertips for ages - but I donât think the wider development community has really noticed. Debugging something short lived like a Lambda execution is quite startling, as the current answer is âadd more loggingâ, and we all know that sucks. To this end, Iâve created a little screencast showing this in action - and it was pretty cool because it was a real example I encountered when I got everything working and was trying my test application out. >>>>>>>>>> >>>>>>>>>> Iâve also put a bit of work into tuning the excellent GitLab CI tools, so that I can cache many of the artefacts used between different build runs (this might also be of interest to others using CI systems). >>>>>>>>>> >>>>>>>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>>>>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>>>>>>> >>>>>>>>>> Tim >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>>>>>>> >>>>>>>>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS Lambda. Itâs early days, but I though it might be interesting to share what Iâve learned so far. >>>>>>>>>>> >>>>>>>>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>>>>>>>> >>>>>>>>>>> With help from many of the folks here, Iâve been able to get a simple example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily try it out yourself. This seems slightly better than what the GoLang folks have been able to do. >>>>>>>>>>> >>>>>>>>>>> Tim >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Guille Polito >>>>>> >>>>>> Research Engineer >>>>>> French National Center for Scientific Research - http://www.cnrs.fr >>>>>> >>>>>> >>>>>> Web: http://guillep.github.io >>>>>> Phone: +33 06 52 70 66 13 >>>> >>>> >>>> >>>> -- >>>> >>>> Guille Polito >>>> >>>> Research Engineer >>>> French National Center for Scientific Research - http://www.cnrs.fr >>>> >>>> >>>> Web: http://guillep.github.io >>>> Phone: +33 06 52 70 66 13 > > -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - http://www.cnrs.fr > > > Web: http://guillep.github.io > Phone: +33 06 52 70 66 13

Aug. 16, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Guillermo Polito

Actually it happens first that monticello is "nicely" coupled with the changeset system and logs all the source code loaded in change sets :D :/ Â¬Â¬. Also, the first two strings in terms of size are related to unicode tables (we should put them in files instead of in the image and load them on demand), and the two biggest arrays also to unicode. I just tried the following in a clean bootstrapped "minimal" image (metacello): "Careful, this will make that #isLetter, #isUppercase #isLowercase, #toLowercase and #toUppercase only work on ascii" Character characterSet: nil. Unicode classPool at: #GeneralCategory put: nil. Unicode classPool at: #DecimalProperty put: nil. UnicodeDefinition removeFromSystem. ChangeSet removeChangeSetsNamedSuchThat: [ :each | true ]. ChangeSet resetCurrentToNewUnnamedChangeSet. MCDefinition clearInstances. Undeclared removeUnreferencedKeys. Smalltalk garbageCollect. like this: ./vm/pharo Pharo7.0-metacello-32bit-fa236b7.image eval --save "Character characterSet: nil. Unicode classPool at: #GeneralCategory put: nil. Unicode classPool at: #DecimalProperty put: nil. UnicodeDefinitions removeFromSystem. ChangeSet removeChangeSetsNamedSuchThat: [ :each | true ]. ChangeSet resetCurrentToNewUnnamedChangeSet. MCDefinition clearInstances. Undeclared removeUnreferencedKeys. Smalltalk garbageCollect." and my image went down from 11MB to 6.6MB (7.0 MB if I don't change back to ascii with the first three lines) Then I tried a tally: ./vm/pharo Pharo7.0-metacello-32bit-fa236b7.image save spacetally ./vm/pharo spacetally.image eval --save "repo := MCFileTreeRepository new directory: '../src' asFileReference. version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. version load." re-clean since i loaded some packages ./vm/pharo spacetally.image eval --save "ChangeSet removeChangeSetsNamedSuchThat: [ :each | true ]. ChangeSet resetCurrentToNewUnnamedChangeSet. MCDefinition clearInstances. Undeclared removeUnreferencedKeys. Smalltalk garbageCollect." This image is now 6.6MB (7.1MB with the unicode large arrays), 4.1% of strings (274k) what seems reasonable. Remaining big strings are Pharo's licence, the buffer of the changes file and then some class comments (shouldn't they be fetched from disk as any other method source code?). Making again a tally shows that ~30% of the space is taken by Arrays and 21.9% by compiled methods. But, BUT! :) I have ~30k arrays and lots of collections also: "MethodDictionary" 2872 + "IdentitySet" 12781 + "OrderedCollection" 4398 + "Set" 2959 + "Dictionary" 1997 + "IdentityDictionary" 454 ----------------------------------------------- 25461 So there are ~5k arrays that are used outside collections. Worth exploring a bit more I think. On Wed, Aug 16, 2017 at 1:23 AM, Guillermo Polito <guillermopolito(a)gmail.com > wrote: > > > On Tue, Aug 15, 2017 at 11:26 PM, Tim Mackinnon <tim(a)testit.works> wrote: > >> Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it >> seems very convoluted to load a single package into the image, I was trying >> to avoid having to create a baselineOf for something so simple - I ended up >> with: >> > > I know, I also believe we have to simplify this. In any case, baselines > are healthy as they allow to also express dependencies. Otherwise you'll > end up loading dependencies by hand. We'll fix this soon I hope. > > >> >> repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. >> >> version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. >> version load. >> >> >> Anyway - in my minimal image, like in the fat image there seems to be a >> surprising amount of bytestrings (4mb worth?). I think that might need some >> digging into? It seems like a lot somehow. Although Benâs neat experiment >> of zipping strings shows thatâs not a real route. >> >> In a deployed minimal image - maybe I can get rid of some other things >> like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins >> - but noticeable) >> >> Class code space # instances >> inst space percent inst average size >> ByteString 2640 37365 >> 4823848 21.50 129.10 >> Array 3742 53002 >> 3961944 17.60 74.75 >> CompiledMethod 19159 30481 >> 2912968 13.00 95.57 >> Association 1148 58348 >> 1867136 8.30 32.00 >> MethodChangeRecord 431 34312 >> 1097984 4.90 32.00 >> ByteArray 4605 290 >> 908728 4.00 3133.54 >> ByteSymbol 1698 22689 >> 840168 3.70 37.03 >> IdentitySet 408 19076 >> 610432 2.70 32.00 >> MethodDictionary 3310 3520 >> 608688 2.70 172.92 >> WeakArray 1758 3024 >> 597824 2.70 197.69 >> MCMethodDefinition 4318 6659 >> 426176 1.90 64.00 >> Protocol 1679 8382 >> 268224 1.20 32.00 >> OrderedCollection 6555 5509 >> 220360 1.00 40.00 >> >> As an aside - my Gitlab project is public, the scripts that load things >> up are in ./scripts (build.sh, and minimal.st and loadlocal.st) >> >> Tim >> >> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> >> >> >> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: >> >>> Hi Guille - just running SpaceTally on my dev image to get a feel for >>> it. It turns out that in the minimal images youâve been creating, its not >>> loaded (makes sense). >>> >> >> Yup, it's loaded afterwards. >> >> All packages are loaded through metacello baselines. We should start >> refactoring and making standalone projects, each one with a baseline for >> himself, and his own dependencies described. >> >> I was checking on your gitlab and I have probably no access: how are you >> finally loading packages in the bootstrap image? Can you share that with us >> in text? I'd like to improve that situation. >> >> >>> Iâm wondering if there is an easy way to import it in (I guess that >>> package should be in the Pharo git tree I cloned to get Fuel loaded right? >>> Or is there a separate standalone source?). >>> >> >> Yes it is, you can get the package programatically doing >> >> SpaceTally package name >> >> And furthermore, get the baseline that currently is loading by doing >> >> package := SpaceTally package name. >> BaselineOf subclasses select: [ :e | >> e project version packages anySatisfy: [ :p | p name = package ]]. >> >> >>> >>> Thanks for all the support, and your email about why the contexts stack >>> up is very well received (I will comment over there). >>> >>> By the way - it looks like Martin Fowler picked up on this announcement >>> - so maybe we might get some interest from his mass of followers. >>> >>> Tim >>> >>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> >>> wrote: >>> >>> Hi Tim, >>> >>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> w >>> rote: >>> >>>> Hey guys, thanks for your enthusiasm around this - and I cannot stress >>>> enough how this was only possible because of the work that has gone into >>>> making Pharo (in particular the 64bit image, as well as having a minimal >>>> image, and some great blog posts on serialising contexts) as well as the >>>> patience from everyone in answering questions and helping me get it all >>>> working. >>>> >>>> Iâm still quite keen to get my execution time back down under 800ms and >>>> Iâd like to actually get back to writing a few skills to automate a few >>>> things around my house. >>>> >>>> To Answer Denisâ question - >>>> >>>> My final footprint is 30.4mb - thats composed of a 22mb image (with a >>>> simple example that pulls in Fuel, ZTimestamp and the S3 Library which >>>> depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>>> >>>> In my original experiments with a 6.0 minimal image - I did manage to >>>> get to a 13.4mb image (which started out as 12mb original size, and then >>>> loaded in STON and had only a simple clock example). I think the sweet spot >>>> is around 20mb total footprint as that seems to get me into the 450ms-900ms >>>> range. >>>> >>>> The 7.0 min image now starts out at 15mb and then Iâm not sure why >>>> loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not >>>> dug into that). >>>> >>> >>> You can do further space analysis using the following expression >>> >>> SpaceTally new printSpaceAnalysis >>> >>> You can do that in an eval and check what's taking space. With measures >>> we can iterate and improve :). >>> >>> >>>> Iâve also found (and this on the back of unserialising the context in >>>> my example) that the way we build images has 15+ saved stack sessions that >>>> have saved on top of each other from the way we build up the images. I >>>> donât yet know the implications of size/speed of these - but we need a >>>> better way of folding executions when we snapshot headless images. Iâm also >>>> not clear if there are any other startup tasks that take precious time >>>> (this also has implications for our fat development images as they take >>>> much longer to appear than they really should). >>>> >>> >>> I'm working on this as I'm writing this mail ;) >>> >>> https://pharo.fogbugz.com/f/cases/20309 >>> https://github.com/pharo-project/pharo/pull/196 >>> >>> I'll write down the implications further in a different thread. >>> >>> >>>> Iâll be exploring some of these size/speed tradeoffâs in follow on >>>> messages. >>>> >>>> But once again, a big thanks - Iâve not enjoyed programming like this >>>> for ages. >>>> >>>> Tim >>>> >>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>>> >>>> hi Tim, >>>> >>>> That is..... AWESOME! >>>> >>>> Very nice delivery - it flowed well with great narration. >>>> >>>> I loved @2:17 "this is the interesting piece, because PharoLambda has >>>> serialized the execution context of its application and saved it into [my >>>> S3 bucket] ... [then on the local machine] rematerializes a debugger [on >>>> that context]." >>>> >>>> There is a clarity in your video presentation that really may intrigue >>>> outsiders. As a community we should push this on the usual hacker forums - >>>> ycombinator could be a good starting point (but I'm locked out of my >>>> account there). >>>> An enticing title could be... >>>> "Debugging Lambdas by re-materializing saved execution contexts on your >>>> local machine." >>>> >>>> cheers -ben >>>> >>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com >>>> > wrote: >>>> >>>>> This is cool Tim. >>>>> >>>>> So what image size you deployed at the end? >>>>> >>>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>>> >>>>>> I just wanted to thank everyone for their help in getting my pet >>>>>> project further along, so that now I can announce that PharoLambda is now >>>>>> working with the V7 minimal image and also supports post mortem debugging >>>>>> by saving a zipped fuel context onto S3. >>>>>> >>>>>> This latter item is particularly satisfying as at a recent serverless >>>>>> conference (JeffConf) there was a panel where poor development tools on >>>>>> serverless platforms was highlighted as a real problem. >>>>>> >>>>>> In our community weâve had these kinds of tools at our fingertips for >>>>>> ages - but I donât think the wider development community has really >>>>>> noticed. Debugging something short lived like a Lambda execution is quite >>>>>> startling, as the current answer is âadd more loggingâ, and we all know >>>>>> that sucks. To this end, Iâve created a little screencast showing this in >>>>>> action - and it was pretty cool because it was a real example I encountered >>>>>> when I got everything working and was trying my test application out. >>>>>> >>>>>> Iâve also put a bit of work into tuning the excellent GitLab CI >>>>>> tools, so that I can cache many of the artefacts used between different >>>>>> build runs (this might also be of interest to others using CI systems). >>>>>> >>>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>>> >>>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS >>>>>> Lambda. Itâs early days, but I though it might be interesting to share what >>>>>> Iâve learned so far. >>>>>> >>>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>>> >>>>>> With help from many of the folks here, Iâve been able to get a simple >>>>>> example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily >>>>>> try it out yourself. This seems slightly better than what the GoLang folks >>>>>> have been able to do. >>>>>> >>>>>> Tim >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >>> -- >>> >>> Guille Polito >>> >>> Research Engineer >>> French National Center for Scientific Research - *http://www.cnrs.fr* >>> <http://www.cnrs.fr/> >>> >>> >>> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >>> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >>> >>> >>> >> >> >> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - *http://www.cnrs.fr* >> <http://www.cnrs.fr/> >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> >> >> > > > -- > > > > Guille Polito > > > Research Engineer > > French National Center for Scientific Research - *http://www.cnrs.fr* > <http://www.cnrs.fr> > > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Aug. 16, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Guillermo Polito

On Tue, Aug 15, 2017 at 11:26 PM, Tim Mackinnon <tim(a)testit.works> wrote: > Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it > seems very convoluted to load a single package into the image, I was trying > to avoid having to create a baselineOf for something so simple - I ended up > with: > I know, I also believe we have to simplify this. In any case, baselines are healthy as they allow to also express dependencies. Otherwise you'll end up loading dependencies by hand. We'll fix this soon I hope. > > repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. > > version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. > version load. > > > Anyway - in my minimal image, like in the fat image there seems to be a > surprising amount of bytestrings (4mb worth?). I think that might need some > digging into? It seems like a lot somehow. Although Benâs neat experiment > of zipping strings shows thatâs not a real route. > > In a deployed minimal image - maybe I can get rid of some other things > like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins > - but noticeable) > > Class code space # instances > inst space percent inst average size > ByteString 2640 37365 > 4823848 21.50 129.10 > Array 3742 53002 > 3961944 17.60 74.75 > CompiledMethod 19159 30481 > 2912968 13.00 95.57 > Association 1148 58348 > 1867136 8.30 32.00 > MethodChangeRecord 431 34312 > 1097984 4.90 32.00 > ByteArray 4605 290 > 908728 4.00 3133.54 > ByteSymbol 1698 22689 > 840168 3.70 37.03 > IdentitySet 408 19076 > 610432 2.70 32.00 > MethodDictionary 3310 3520 > 608688 2.70 172.92 > WeakArray 1758 3024 > 597824 2.70 197.69 > MCMethodDefinition 4318 6659 > 426176 1.90 64.00 > Protocol 1679 8382 > 268224 1.20 32.00 > OrderedCollection 6555 5509 > 220360 1.00 40.00 > > As an aside - my Gitlab project is public, the scripts that load things up > are in ./scripts (build.sh, and minimal.st and loadlocal.st) > > Tim > > On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> > wrote: > > > > On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: > >> Hi Guille - just running SpaceTally on my dev image to get a feel for it. >> It turns out that in the minimal images youâve been creating, its not >> loaded (makes sense). >> > > Yup, it's loaded afterwards. > > All packages are loaded through metacello baselines. We should start > refactoring and making standalone projects, each one with a baseline for > himself, and his own dependencies described. > > I was checking on your gitlab and I have probably no access: how are you > finally loading packages in the bootstrap image? Can you share that with us > in text? I'd like to improve that situation. > > >> Iâm wondering if there is an easy way to import it in (I guess that >> package should be in the Pharo git tree I cloned to get Fuel loaded right? >> Or is there a separate standalone source?). >> > > Yes it is, you can get the package programatically doing > > SpaceTally package name > > And furthermore, get the baseline that currently is loading by doing > > package := SpaceTally package name. > BaselineOf subclasses select: [ :e | > e project version packages anySatisfy: [ :p | p name = package ]]. > > >> >> Thanks for all the support, and your email about why the contexts stack >> up is very well received (I will comment over there). >> >> By the way - it looks like Martin Fowler picked up on this announcement - >> so maybe we might get some interest from his mass of followers. >> >> Tim >> >> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> >> Hi Tim, >> >> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> wrote: >> >>> Hey guys, thanks for your enthusiasm around this - and I cannot stress >>> enough how this was only possible because of the work that has gone into >>> making Pharo (in particular the 64bit image, as well as having a minimal >>> image, and some great blog posts on serialising contexts) as well as the >>> patience from everyone in answering questions and helping me get it all >>> working. >>> >>> Iâm still quite keen to get my execution time back down under 800ms and >>> Iâd like to actually get back to writing a few skills to automate a few >>> things around my house. >>> >>> To Answer Denisâ question - >>> >>> My final footprint is 30.4mb - thats composed of a 22mb image (with a >>> simple example that pulls in Fuel, ZTimestamp and the S3 Library which >>> depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>> >>> In my original experiments with a 6.0 minimal image - I did manage to >>> get to a 13.4mb image (which started out as 12mb original size, and then >>> loaded in STON and had only a simple clock example). I think the sweet spot >>> is around 20mb total footprint as that seems to get me into the 450ms-900ms >>> range. >>> >>> The 7.0 min image now starts out at 15mb and then Iâm not sure why >>> loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not >>> dug into that). >>> >> >> You can do further space analysis using the following expression >> >> SpaceTally new printSpaceAnalysis >> >> You can do that in an eval and check what's taking space. With measures >> we can iterate and improve :). >> >> >>> Iâve also found (and this on the back of unserialising the context in my >>> example) that the way we build images has 15+ saved stack sessions that >>> have saved on top of each other from the way we build up the images. I >>> donât yet know the implications of size/speed of these - but we need a >>> better way of folding executions when we snapshot headless images. Iâm also >>> not clear if there are any other startup tasks that take precious time >>> (this also has implications for our fat development images as they take >>> much longer to appear than they really should). >>> >> >> I'm working on this as I'm writing this mail ;) >> >> https://pharo.fogbugz.com/f/cases/20309 >> https://github.com/pharo-project/pharo/pull/196 >> >> I'll write down the implications further in a different thread. >> >> >>> Iâll be exploring some of these size/speed tradeoffâs in follow on >>> messages. >>> >>> But once again, a big thanks - Iâve not enjoyed programming like this >>> for ages. >>> >>> Tim >>> >>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>> >>> hi Tim, >>> >>> That is..... AWESOME! >>> >>> Very nice delivery - it flowed well with great narration. >>> >>> I loved @2:17 "this is the interesting piece, because PharoLambda has >>> serialized the execution context of its application and saved it into [my >>> S3 bucket] ... [then on the local machine] rematerializes a debugger [on >>> that context]." >>> >>> There is a clarity in your video presentation that really may intrigue >>> outsiders. As a community we should push this on the usual hacker forums - >>> ycombinator could be a good starting point (but I'm locked out of my >>> account there). >>> An enticing title could be... >>> "Debugging Lambdas by re-materializing saved execution contexts on your >>> local machine." >>> >>> cheers -ben >>> >>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com> >>> wrote: >>> >>>> This is cool Tim. >>>> >>>> So what image size you deployed at the end? >>>> >>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>> >>>>> I just wanted to thank everyone for their help in getting my pet >>>>> project further along, so that now I can announce that PharoLambda is now >>>>> working with the V7 minimal image and also supports post mortem debugging >>>>> by saving a zipped fuel context onto S3. >>>>> >>>>> This latter item is particularly satisfying as at a recent serverless >>>>> conference (JeffConf) there was a panel where poor development tools on >>>>> serverless platforms was highlighted as a real problem. >>>>> >>>>> In our community weâve had these kinds of tools at our fingertips for >>>>> ages - but I donât think the wider development community has really >>>>> noticed. Debugging something short lived like a Lambda execution is quite >>>>> startling, as the current answer is âadd more loggingâ, and we all know >>>>> that sucks. To this end, Iâve created a little screencast showing this in >>>>> action - and it was pretty cool because it was a real example I encountered >>>>> when I got everything working and was trying my test application out. >>>>> >>>>> Iâve also put a bit of work into tuning the excellent GitLab CI tools, >>>>> so that I can cache many of the artefacts used between different build runs >>>>> (this might also be of interest to others using CI systems). >>>>> >>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>> >>>>> Tim >>>>> >>>>> >>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>> >>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS >>>>> Lambda. Itâs early days, but I though it might be interesting to share what >>>>> Iâve learned so far. >>>>> >>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>> >>>>> With help from many of the folks here, Iâve been able to get a simple >>>>> example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily >>>>> try it out yourself. This seems slightly better than what the GoLang folks >>>>> have been able to do. >>>>> >>>>> Tim >>>>> >>>>> >>>>> >>>> >>> >>> >> >> >> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - *http://www.cnrs.fr* >> <http://www.cnrs.fr/> >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> >> >> > > > -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - *http://www.cnrs.fr* > <http://www.cnrs.fr/> > > > *Web:* *http://guillep.github.io* <http://guillep.github.io/> > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > > > -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Aug. 15, 2017

Re: [Pharo-users] What is proper fix for this? (was: Re: Big Glorp problem w/ type coercion, pls help)

by herby＠mailbox.sk

BTW I took the latter way (as method tries to be as optimized as possible), it is in http://smalltalkhub.com/#!/~herby/Glorp/versions/Glorp-HerbyVojcik.127, consider merging in. Thanks. Herby VojÄÃk wrote: > Hello! > > I think I found the culprit. Few methods posted here: > > > Mapping >> expressionFor: anObject basedOn: anExpression relation: > aSymbol > "Return our expression using the object's values. e.g. if this was a > direct mapping from id->ID and the object had id: 3, then return > TABLE.ID=3. Used when rewriting object=object into field=field" > > | myValue result | > myValue := self expressionFor: anObject. > result := nil. > myValue with: self join allTargetFields do: [:eachValue :eachField | > | source | > source := anExpression get: self attribute name. > source hasDescriptor ifTrue: [source := source getField: eachField]. > result := (source get: aSymbol withArguments: (Array with: eachValue)) > AND: result]. > ^result > > > > DirectMapping >> expressionFor: anObject basedOn: anExpression relation: > aSymbol > "Return our expression using the object's values. e.g. if this was a > direct mapping from id->ID and the object had id: 3, then return > TABLE.ID=3" > > | value | > value := anObject isNil > ifTrue: [nil] > ifFalse: > [anObject isGlorpExpression > ifTrue: [anObject getMapping: self named: self attributeName] > ifFalse: [anObject glorpIsCollection > ifTrue: [anObject collect: [:each | attribute getValueFrom: each]] > ifFalse: [attribute getValueFrom: anObject]]]. > ^(anExpression get: self attribute name) get: aSymbol withArguments: > (Array with: value) > > > > Mapping >> expressionFor: anObject > "Return an expression representing the value of the object. This can be > nil, an object value or values, an expression, or a collection of > expressions (for a composite key, if we're passed an expression)" > > anObject isNil ifTrue: [^#(nil)]. > anObject isGlorpExpression ifFalse: [ > ^self mappedFields collect: [:each | > self valueOfField: each fromObject: anObject]]. > ^self mappedFields > collect: [:each | (anObject getField: each)] > > > > Mapping >> getValueFrom: anObject > > ^self attribute getValueFrom: anObject > > > > DirectMapping >> valueOfField: aField fromObject: anObject > field = aField ifFalse: [self error: 'Mapping doesn''t describe field']. > ^self convertedDbValueOf: (self getValueFrom: anObject) > > > > DirectMapping >> mappedFields > "Return a collection of fields that this mapping will write into any of > the containing object's rows" > > ^Array with: self field > > > The thing is, both Mapping >> expressionFor:basedOn:relation: and the > overridden DirectMapping's version eventually send > > someSource get: aSymbol withArguments: (Array with: eachValue) > > but in Mapping's code, the value is taken from `myValue := self > expressionFor: anObject`. which, as seen in #expressionFor: code, gets > the value via > > self valueOfField: aMappedField fromObject: anObject > > and indeed, if tried aDirectMapping expressionFor: anObject in debugger, > it gets the value of the primary key converted in the below case (that > is, as a ByteArray). This is clear from the DirectMapping >> > valueOfField:fromObject: code above, which does `self getValueFrom: > anObject` (which passes it to `attribute getValueFrom: anObject`) > _and_converts_it_. > > But in the overridden DirectMapping >> expressionFor:basedOn:relation:, > the value to be passed in the > > someSource get: aSymbol withArguments: (Array with: value) > > is obtained by direct > > attribute getValueFrom: anObject > > but _is_not_converted_. IOW, it seems this method was heavily optimized > (`attribute getValueFrom:` instead of `self getValueFrom:`, for > example), but the conversion, normally present via expressionFor: and > ultimately valueOfField:fromObject: was optimized away as well. > > > > > Now, what is the correct way to fix the method (I hope you agree it is a > bug)? > > > This? > > DirectMapping >> expressionFor: anObject basedOn: anExpression relation: > aSymbol > "Return our expression using the object's values. e.g. if this was a > direct mapping from id->ID and the object had id: 3, then return > TABLE.ID=3" > > | value | > value := anObject isNil > ifTrue: [nil] > ifFalse: > [anObject isGlorpExpression > ifTrue: [anObject getMapping: self named: self attributeName] > ifFalse: [anObject glorpIsCollection > ifTrue: [anObject collect: [:each | self valueOfField: aField > fromObject: each]] > ifFalse: [self valueOfField: aField fromObject: anObject]]]. > ^(anExpression get: self attribute name) get: aSymbol withArguments: > (Array with: value) > > > > or this? > > DirectMapping >> expressionFor: anObject basedOn: anExpression relation: > aSymbol > "Return our expression using the object's values. e.g. if this was a > direct mapping from id->ID and the object had id: 3, then return > TABLE.ID=3" > > | value | > value := anObject isNil > ifTrue: [nil] > ifFalse: > [anObject isGlorpExpression > ifTrue: [anObject getMapping: self named: self attributeName] > ifFalse: [anObject glorpIsCollection > ifTrue: [anObject collect: [:each | self convertedDbValueOf: (attribute > getValueFrom: each)]] > ifFalse: [self convertedDbValueOf: (attribute getValueFrom: anObject)]]]. > ^(anExpression get: self attribute name) get: aSymbol withArguments: > (Array with: value) > > > > Or something completely different? > > > Thanks, Herby > > Herby VojÄÃk wrote: >> Hello! >> >> I encountered a problem with OneToOneMapping and type coercion. When >> writing data, thing work; when reading data, the right child of relation >> fails to convert. >> >> I tried everything possible to inject converters (even subclassing >> GlorpBlobType), but to no avail. RelationExpression passes conversion to >> its left child: >> >> convertedDbValueOf: anObject >> "Assume that our types match, so we can ask either child to do the >> conversion. That isn't guaranteed, but should at least work for the >> common cases." >> ^leftChild convertedDbValueOf: anObject. >> >> but the left child is FieldExpression in case of OneToOneMapping, which: >> >> convertedDbValueOf: anObject >> "We don't do any conversion" >> ^anObject >> >> What is strange, writing works (even the OneToOneMapping, I opened the >> sqlite file with an explorer), but second SELECT, one using the relation >> (`state := self dao findStateByAgent: agent` in clientSync), fails with >> "GlorpDatabaseReadError: Could not coerce arguments". FWIW, the first >> one _does_ convert when creating bindings, as it uses MappingExpression >> as left child (stepped over it in debugger). >> >> >> >> Is it meant to be a strange case that primary key is something >> non-primitive needing coercion (in this case, it is a UUID which needs >> coercion to ByteArray, even if it is its subclass)? >> >> >> >> Here's the stack of running the test which fails: >> >> PharoDatabaseAccessor(DatabaseAccessor)>>handleError:for: >> [ :ex | self handleError: ex for: command ] in [ | result | >> self checkPermissionFor: command. >> result := [ (self useBinding and: [ command useBinding ]) >> ifTrue: [ command executeBoundIn: self ] >> ifFalse: [ command executeUnboundIn: self ] ] >> on: Dialect error >> do: [ :ex | self handleError: ex for: command ]. >> aBoolean >> ifTrue: [ result ] >> ifFalse: [ result upToEnd ] ] in >> PharoDatabaseAccessor(DatabaseAccessor)>>executeCommand:returnCursor: >> BlockClosure>>cull: >> Context>>evaluateSignal: >> Context>>handleSignal: >> Error(Exception)>>signal >> Error(Exception)>>signal: >> ExternalLibraryFunction(Object)>>error: >> ExternalLibraryFunction(Object)>>externalCallFailed >> ExternalLibraryFunction(ExternalFunction)>>invokeWithArguments: >> UDBCSQLite3Library>>apiBindBlob:atColumn:with:with:with: >> UDBCSQLite3Library>>with:at:putBlob: >> UDBCSQLite3Statement>>at:putByteArray: >> UDBCSQLite3ResultSet>>execute:withIndex:withValue: >> [ :v | i := self execute: statement withIndex: i withValue: v ] in >> UDBCSQLite3ResultSet>>execute:withCollection: >> OrderedCollection>>do: >> UDBCSQLite3ResultSet>>execute:withCollection: >> UDBCSQLite3ResultSet>>execute:with:on: >> UDBCSQLite3Connection>>execute:with: >> GlorpSQLite3Driver>>basicExecuteSQLString:binding: >> PharoDatabaseAccessor>>executeCommandBound: >> QuerySelectCommand(DatabaseCommand)>>executeBoundIn: >> [ (self useBinding and: [ command useBinding ]) >> ifTrue: [ command executeBoundIn: self ] >> ifFalse: [ command executeUnboundIn: self ] ] in [ | result | >> self checkPermissionFor: command. >> result := [ (self useBinding and: [ command useBinding ]) >> ifTrue: [ command executeBoundIn: self ] >> ifFalse: [ command executeUnboundIn: self ] ] >> on: Dialect error >> do: [ :ex | self handleError: ex for: command ]. >> aBoolean >> ifTrue: [ result ] >> ifFalse: [ result upToEnd ] ] in >> PharoDatabaseAccessor(DatabaseAccessor)>>executeCommand:returnCursor: >> BlockClosure>>on:do: >> [ | result | >> self checkPermissionFor: command. >> result := [ (self useBinding and: [ command useBinding ]) >> ifTrue: [ command executeBoundIn: self ] >> ifFalse: [ command executeUnboundIn: self ] ] >> on: Dialect error >> do: [ :ex | self handleError: ex for: command ]. >> aBoolean >> ifTrue: [ result ] >> ifFalse: [ result upToEnd ] ] in >> PharoDatabaseAccessor(DatabaseAccessor)>>executeCommand:returnCursor: >> [ caught := true. >> self wait. >> blockValue := mutuallyExcludedBlock value ] in Semaphore>>critical: >> BlockClosure>>ensure: >> Semaphore>>critical: >> PharoDatabaseAccessor(DatabaseAccessor)>>executeCommand:returnCursor: >> [ session accessor executeCommand: command returnCursor: true ] in >> SimpleQuery>>rowsFromDatabaseWithParameters: >> BlockClosure>>on:do: >> SimpleQuery>>rowsFromDatabaseWithParameters: >> SimpleQuery(AbstractReadQuery)>>readFromDatabaseWithParameters: >> SimpleQuery(AbstractReadQuery)>>executeWithParameters:in: >> GlorpSession>>execute: >> GlorpSession>>readOneOf:where: >> TowergameDao>>findStateByAgent: >> [ | agent state | >> agent := self dao findAgentById: anObject agentId. >> state := self dao findStateByAgent: agent. >> ^ NeoJSONObject new >> agentId: agent id; >> stateVersion: state version; >> totalAnsweredQuestions: >> (NeoJSONObject new >> good: 0; >> bad: 0; >> yourself); >> yourself ] in Towergame>>clientSync: >> [ myUnitOfWork := self hasUnitOfWork not. >> myUnitOfWork >> ifTrue: [ self beginUnitOfWork ]. >> result := aBlock numArgs = 1 >> ifTrue: [ aBlock value: self ] >> ifFalse: [ aBlock value ]. >> myUnitOfWork >> ifTrue: [ self commitUnitOfWork ] ] in GlorpSession>>inUnitOfWorkDo: >> BlockClosure>>ifCurtailed: >> GlorpSession>>inUnitOfWorkDo: >> TowergameDao>>inUnitOfWorkDo: >> Towergame>>clientSync: >> TowergameSyncTests>>testPlayerChecksStateVersion >> TowergameSyncTests(TestCase)>>performTest >> [ self setUp. >> self performTest ] in TowergameSyncTests(TestCase)>>runCase >> BlockClosure>>ensure: >> TowergameSyncTests(TestCase)>>runCase >> [ aTestCase runCase ] in [ [ aTestCase runCase ] >> on: Halt >> do: [ :halt | >> "if test was halted we should resume all background failures >> to debug all of them together with test process" >> failedProcesses keysDo: #resume. >> halt pass ] ] in TestExecutionEnvironment>>runTestCaseSafelly: >> BlockClosure>>on:do: >> [ [ aTestCase runCase ] >> on: Halt >> do: [ :halt | >> "if test was halted we should resume all background failures >> to debug all of them together with test process" >> failedProcesses keysDo: #resume. >> halt pass ] ] in TestExecutionEnvironment>>runTestCaseSafelly: >> BlockClosure>>on:do: >> TestExecutionEnvironment>>runTestCaseSafelly: >> [ self runTestCaseSafelly: aTestCase ] in [ [ self runTestCaseSafelly: >> aTestCase ] >> ensure: [ testCompleted := true. >> watchDogSemaphore signal ]. "signal that test case completes" >> self checkForkedProcesses ] in TestExecutionEnvironment>>runTestCase: >> BlockClosure>>ensure: >> [ [ self runTestCaseSafelly: aTestCase ] >> ensure: [ testCompleted := true. >> watchDogSemaphore signal ]. "signal that test case completes" >> self checkForkedProcesses ] in TestExecutionEnvironment>>runTestCase: >> BlockClosure>>ifCurtailed: >> TestExecutionEnvironment>>runTestCase: >> [ testEnv runTestCase: aTestCase ] in >> DefaultExecutionEnvironment>>runTestCase: >> [ self value: anExecutionEnvironment. >> anExecutionEnvironment activated. >> aBlock value ] in CurrentExecutionEnvironment class>>activate:for: >> BlockClosure>>ensure: >> CurrentExecutionEnvironment class>>activate:for: >> TestExecutionEnvironment(ExecutionEnvironment)>>beActiveDuring: >> DefaultExecutionEnvironment>>runTestCase: >> CurrentExecutionEnvironment class>>runTestCase: >> TowergameSyncTests(TestCase)>>runCaseManaged >> [ aTestCase announce: TestCaseStarted withResult: self. >> aTestCase runCaseManaged. >> aTestCase announce: TestCaseEnded withResult: self. >> self addPass: aTestCase ] in TestResult>>runCaseForDebug: >> BlockClosure>>on:do: >> TestResult>>runCaseForDebug: >> [ result runCaseForDebug: self ] in TowergameSyncTests(TestCase)>>debug >> BlockClosure>>ensure: >> TowergameSyncTests(TestCase)>>debug >> [ :each | >> each debug. >> self announceTest: each. >> self changed: each ] in [ self tests >> do: [ :each | >> each debug. >> self announceTest: each. >> self changed: each ] ] in TestSuite>>debug >> OrderedCollection>>do: >> [ self tests >> do: [ :each | >> each debug. >> self announceTest: each. >> self changed: each ] ] in TestSuite>>debug >> BlockClosure>>ensure: >> TestSuite>>debug >> [ :aSuite | aSuite debug ] in TestRunner>>debugSuite: >> BlockClosure>>cull: >> BlockClosure>>cull:cull: >> [ aBlock cull: aTestSuite cull: result ] in TestRunner>>executeSuite:as: >> BlockClosure>>ensure: >> TestRunner>>executeSuite:as: >> TestRunner>>debugSuite: >> TestRunner>>debug: >> TestRunner>>errorSelected: >> PluggableListMorph>>changeModelSelection: >> PluggableListMorph>>mouseUpOnSingle: >> PluggableListMorph>>mouseUp: >> PluggableListMorph(Morph)>>handleMouseUp: >> MouseButtonEvent>>sentTo: >> PluggableListMorph(Morph)>>handleEvent: >> MorphicEventDispatcher>>dispatchDefault:with: >> MorphicEventDispatcher>>handleMouseUp: >> MouseButtonEvent>>sentTo: >> [ ^ anEvent sentTo: self ] in >> MorphicEventDispatcher>>dispatchEvent:with: >> BlockClosure>>ensure: >> MorphicEventDispatcher>>dispatchEvent:with: >> PluggableListMorph(Morph)>>processEvent:using: >> PluggableListMorph(Morph)>>processEvent: >> PluggableListMorph>>handleFocusEvent: >> [ ActiveHand := self. >> ActiveEvent := anEvent. >> result := focusHolder >> handleFocusEvent: (anEvent transformedBy: (focusHolder transformedFrom: >> self)) ] in HandMorph>>sendFocusEvent:to:clear: >> BlockClosure>>on:do: >> WorldMorph(PasteUpMorph)>>becomeActiveDuring: >> HandMorph>>sendFocusEvent:to:clear: >> HandMorph>>sendEvent:focus:clear: >> HandMorph>>sendMouseEvent: >> HandMorph>>handleEvent: >> HandMorph>>processEventsFromQueue: >> HandMorph>>processEvents >> [ :h | >> self activeHand: h. >> h processEvents. >> self activeHand: nil ] in WorldState>>doOneCycleNowFor: >> Array(SequenceableCollection)>>do: >> WorldState>>handsDo: >> WorldState>>doOneCycleNowFor: >> WorldState>>doOneCycleFor: >> WorldMorph>>doOneCycle >> WorldMorph class>>doOneCycle >> [ [ WorldMorph doOneCycle. >> Processor yield. >> false ] whileFalse: [ ] ] in MorphicUIManager>>spawnNewProcess >> [ self value. >> Processor terminateActive ] in BlockClosure>>newProcess >> >> >> >> And here's the code: >> >> >> Towergame.st: >> >> GlorpBlobType subclass: #GlorpBlob2Type >> instanceVariableNames: '' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> >> !GlorpBlob2Type methodsFor: 'types' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> converterForStType: aClass >> aClass = UUID ifTrue: [ ^ UuidConverter new ]. >> ^ super converterForStType: aClass! ! >> >> >> Object subclass: #TgAct >> instanceVariableNames: 'agent tool timestamp' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgAct commentStamp: 'HerbyVojcik 8/5/2017 19:23' prior: 0! >> I represent a relationship between a player (TgAgent) >> and a device (TgTool). >> >> In particular, I am created whenever a player logs in to the game from >> different device >> than it was last time (or first time, ever).! >> >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> agent >> ^ agent! ! >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> agent: anObject >> agent := anObject! ! >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> timestamp: anObject >> timestamp := anObject! ! >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> tool >> ^ tool! ! >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> tool: anObject >> tool := anObject! ! >> >> !TgAct methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> timestamp >> ^ timestamp! ! >> >> >> !TgAct methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> initialize >> >> super initialize. >> >> agent := nil. >> timestamp := DateAndTime now asUTC. >> tool := nil.! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgAct class >> instanceVariableNames: ''! >> >> !TgAct class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> agent: aTgAgent tool: aTgTool >> ^ self new >> agent: aTgAgent; >> tool: aTgTool; >> yourself! ! >> >> >> Object subclass: #TgAgent >> instanceVariableNames: 'id' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgAgent commentStamp: 'HerbyVojcik 8/5/2017 19:22' prior: 0! >> I represent a towergame player. >> >> I only contain player-related information; >> the game state itself is in TgState.! >> >> >> !TgAgent methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> initialize >> >> super initialize. >> >> id := nil.! ! >> >> >> !TgAgent methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> id: anObject >> id := anObject! ! >> >> !TgAgent methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> id >> ^ id! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgAgent class >> instanceVariableNames: ''! >> >> !TgAgent class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> id: aString >> ^ self new >> id: aString; >> yourself! ! >> >> >> Object subclass: #TgAnswers >> instanceVariableNames: 'good bad' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgAnswers commentStamp: 'HerbyVojcik 8/5/2017 20:23' prior: 0! >> I represent the answered question stats. >> >> I know how many good / bad answered questions there is.! >> >> >> !TgAnswers methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> good >> ^ good! ! >> >> !TgAnswers methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> bad: anObject >> bad := anObject! ! >> >> !TgAnswers methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> bad >> ^ bad! ! >> >> !TgAnswers methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> good: anObject >> good := anObject! ! >> >> >> !TgAnswers methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> initialize >> >> super initialize. >> >> bad := 0. >> good := 0.! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgAnswers class >> instanceVariableNames: ''! >> >> !TgAnswers class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> good: anInteger bad: anInteger2 >> ^ self new >> good: anInteger; >> bad: anInteger2; >> yourself! ! >> >> >> Object subclass: #TgFloors >> instanceVariableNames: 'total reinforced' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgFloors commentStamp: 'HerbyVojcik 8/5/2017 20:22' prior: 0! >> I represent the floor building status. >> >> I know how many floors are build and how many of them is reinforced.! >> >> >> !TgFloors methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> total >> ^ total! ! >> >> !TgFloors methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> total: anObject >> total := anObject! ! >> >> !TgFloors methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> reinforced >> ^ reinforced! ! >> >> !TgFloors methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> reinforced: anObject >> reinforced := anObject! ! >> >> >> !TgFloors methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> initialize >> >> super initialize. >> >> reinforced := 0. >> total := 0.! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgFloors class >> instanceVariableNames: ''! >> >> !TgFloors class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> total: anInteger >> ^ self total: anInteger reinforced: 0! ! >> >> !TgFloors class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> total: anInteger reinforced: anInteger2 >> ^ self new >> total: anInteger; >> reinforced: anInteger2; >> yourself! ! >> >> >> Object subclass: #TgState >> instanceVariableNames: 'agent version packs valuables score bestScore >> answers' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgState commentStamp: 'HerbyVojcik 8/5/2017 20:20' prior: 0! >> I represent the game state. >> >> I have relation to a player (TgAgent) and have a version. >> Then, I contain (directly or indirectly) other parts that >> make up the player's game state. >> >> Whenever I am changed by game progress, my version is changed as well.! >> >> >> !TgState methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> initialize >> >> super initialize. >> >> agent := nil. >> answers := nil. >> bestScore := nil. >> packs := Set new. >> score := nil. >> valuables := nil. >> version := nil.! ! >> >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> score: anObject >> score := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> bestScore: anObject >> bestScore := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> agent: anObject >> agent := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> score >> ^ score! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> packs >> ^ packs! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> version >> ^ version! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> bestScore >> ^ bestScore! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> agent >> ^ agent! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> answers: anObject >> answers := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> valuables: anObject >> valuables := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> valuables >> ^ valuables! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> version: anObject >> version := anObject! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> answers >> ^ answers! ! >> >> !TgState methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> packs: anObject >> packs := anObject! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgState class >> instanceVariableNames: ''! >> >> !TgState class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> agent: aTgAgent version: aString >> ^ self new >> agent: aTgAgent; >> version: aString; >> yourself! ! >> >> >> Object subclass: #TgTool >> instanceVariableNames: 'id' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgTool commentStamp: 'HerbyVojcik 8/5/2017 19:26' prior: 0! >> I represent the device (mobile phone, web browser, ..) >> that player uses to connect to game.! >> >> >> !TgTool methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> initialize >> >> super initialize. >> >> id := nil.! ! >> >> >> !TgTool methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> id: anObject >> id := anObject! ! >> >> !TgTool methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> id >> ^ id! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgTool class >> instanceVariableNames: ''! >> >> !TgTool class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> id: aString >> ^ self new >> id: aString; >> yourself! ! >> >> >> Object subclass: #TgValuables >> instanceVariableNames: 'coins gems' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !TgValuables commentStamp: 'HerbyVojcik 8/5/2017 20:22' prior: 0! >> I represent a purse. >> >> I know how many coins and gems there is.! >> >> >> !TgValuables methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> initialize >> >> super initialize. >> >> coins := 0. >> gems := 0.! ! >> >> >> !TgValuables methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> gems: anObject >> gems := anObject! ! >> >> !TgValuables methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> coins: anObject >> coins := anObject! ! >> >> !TgValuables methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> gems >> ^ gems! ! >> >> !TgValuables methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> coins >> ^ coins! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TgValuables class >> instanceVariableNames: ''! >> >> !TgValuables class methodsFor: 'instance creation' stamp: >> 'HerbertVojÃÂÃÂk 8/14/2017 18:09:53'! >> coins: anInteger gems: anInteger2 >> ^ self new >> coins: anInteger; >> gems: anInteger2; >> yourself! ! >> >> >> Object subclass: #Towergame >> instanceVariableNames: 'dao' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> !Towergame commentStamp: 'HerbyVojcik 5/17/2017 17:19' prior: 0! >> I am the Towergame app class. >> >> I configure and start towergame server processing.! >> >> >> !Towergame methodsFor: 'actions' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> clientSync: anObject >> self dao inUnitOfWorkDo: [ >> | agent state | >> agent := self dao findAgentById: anObject agentId. >> state := self dao findStateByAgent: agent. >> ^ NeoJSONObject new >> agentId: agent id; >> stateVersion: state version; >> totalAnsweredQuestions: (NeoJSONObject new good: 0; bad: 0; yourself); >> yourself ]! ! >> >> >> !Towergame methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> initialize >> >> super initialize. >> >> dao := nil. >> ! ! >> >> >> !Towergame methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> dao: anObject >> dao := anObject! ! >> >> !Towergame methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> dao >> ^ dao! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> Towergame class >> instanceVariableNames: 'default'! >> >> !Towergame class methodsFor: 'instance creation' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> dao: aDao >> ^ self new >> dao: aDao; >> yourself! ! >> >> >> !Towergame class methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> defaultDbLogin >> | databaseFile | >> databaseFile := Smalltalk imageDirectory asFileReference / >> 'towergame.db'. >> ^ Login new >> database: UDBCSQLite3Platform new; >> host: ''; >> port: ''; >> username: ''; >> password: ''; >> databaseName: databaseFile fullPath asZnUrl asString; >> yourself ! ! >> >> !Towergame class methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> default >> ^ default ifNil: [ default := self >> dao: (self daoForLogin: self defaultDbLogin) >> ]! ! >> >> !Towergame class methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> defaultPort >> ^ 4998! ! >> >> >> !Towergame class methodsFor: 'configuration' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> configureServer >> (self serverFor: self default on: self defaultPort) start; register >> ! ! >> >> !Towergame class methodsFor: 'configuration' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> configureSqlite3 >> >> PharoDatabaseAccessor DefaultDriver: GlorpSQLite3Driver! ! >> >> !Towergame class methodsFor: 'configuration' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> configure >> self configureSqlite3. >> self configureServer.! ! >> >> >> !Towergame class methodsFor: 'factory' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> daoForLogin: aLogin >> ^ TowergameDao forLogin: aLogin! ! >> >> !Towergame class methodsFor: 'factory' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> serverFor: aTowergame on: port >> ^ (ZnServer on: port) >> delegate: (TowergameDelegate on: aTowergame); >> yourself! ! >> >> >> Object subclass: #TowergameDao >> instanceVariableNames: 'glorpSession glorpLogin' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> >> !TowergameDao methodsFor: 'transactions' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> inUnitOfWorkDo: aBlock >> ^ self glorpSession inUnitOfWorkDo: aBlock! ! >> >> >> !TowergameDao methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> initialize >> >> super initialize. >> >> glorpLogin := nil. >> glorpSession := nil.! ! >> >> >> !TowergameDao methodsFor: 'initialize-release' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> reset >> glorpSession := nil.! ! >> >> >> !TowergameDao methodsFor: 'query' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> findStateByAgent: anAgent >> ^ self glorpSession readOneOf: TgState where: [ :one | one agent = >> anAgent ]! ! >> >> !TowergameDao methodsFor: 'query' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> findAgentById: anUUID >> ^ self glorpSession readOneOf: TgAgent where: [ :one | one id = anUUID >> ]! ! >> >> >> !TowergameDao methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> glorpLogin: anObject >> glorpLogin := anObject! ! >> >> !TowergameDao methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> glorpLogin >> ^ glorpLogin! ! >> >> !TowergameDao methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> glorpSession >> glorpSession ifNil: [ >> glorpSession := TowergameDescriptorSystem sessionForLogin: self >> glorpLogin ]. >> glorpSession accessor isLoggedIn ifFalse: [ >> glorpSession accessor login ]. >> ^ glorpSession! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TowergameDao class >> instanceVariableNames: ''! >> >> !TowergameDao class methodsFor: 'instance creation' stamp: >> 'HerbertVojÃÂÃÂk 8/14/2017 18:09:53'! >> forLogin: aLogin >> ^ self new >> glorpLogin: aLogin; >> yourself! ! >> >> >> ZnDispatcherDelegate subclass: #TowergameDelegate >> instanceVariableNames: 'towergame' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> >> !TowergameDelegate methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> towergame >> ^ towergame! ! >> >> !TowergameDelegate methodsFor: 'accessing' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> towergame: anObject >> towergame := anObject! ! >> >> >> !TowergameDelegate methodsFor: 'initialization' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> initialize >> super initialize. >> towergame := nil. >> self >> map: '/api/v1/sync' >> to: [ :request :response | self syncRequest: request toResponse: >> response ]! ! >> >> >> !TowergameDelegate methodsFor: 'action' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> syncRequest: request toResponse: response >> | requestPayload responsePayload uuidKeys | >> uuidKeys := #(agentId stateVersion deviceId). >> request method == #POST ifFalse: [ ^ ZnResponse methodNotAllowed: >> request ]. >> requestPayload := NeoJSONObject fromString: request contents. >> requestPayload ifNotNil: [ >> uuidKeys do: [ :each | requestPayload at: each ifPresentPut: [ :s | UUID >> fromString: s ] ] ]. >> responsePayload := self towergame clientSync: requestPayload. >> responsePayload ifNotNil: [ >> uuidKeys do: [ :each | responsePayload at: each ifPresentPut: #asString >> ] ]. >> ^ response >> entity: (ZnEntity >> with: (NeoJSONWriter toString: responsePayload) >> type: ZnMimeType applicationJson); >> yourself! ! >> >> "-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- "! >> >> TowergameDelegate class >> instanceVariableNames: ''! >> >> !TowergameDelegate class methodsFor: 'instance creation' stamp: >> 'HerbertVojÃÂÃÂk 8/14/2017 18:09:53'! >> on: aTowergame >> ^ self new towergame: aTowergame; yourself! ! >> >> >> DescriptorSystem subclass: #TowergameDescriptorSystem >> instanceVariableNames: 'uuidConverter' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> tableForAGENT: aTable >> >> (aTable createFieldNamed: 'id' type: platform blob2) bePrimaryKey. >> ! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> classModelForTgAgent: aClassModel >> aClassModel >> newAttributeNamed: #id type: UUID! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbyVojcik >> 8/14/2017 18:24'! >> tableForSTATE: aTable >> >> (aTable createFieldNamed: 'agent' type: platform blob2) in: [ >> :agentField | >> agentField bePrimaryKey. >> aTable addForeignKeyFrom: agentField to: ((self tableNamed: 'AGENT') >> fieldNamed: 'id') ]. >> (aTable createFieldNamed: 'version' type: platform blob) beIndexed. >> ! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> descriptorForTgAgent: aDescriptor >> | table | >> table := self tableNamed: 'AGENT'. >> aDescriptor table: table. >> (aDescriptor newMapping: DirectMapping) >> from: #id to: (table fieldNamed: 'id').! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> tableForACT: aTable >> >> (aTable createFieldNamed: 'agent' type: platform blob2) beIndexed. >> (aTable createFieldNamed: 'tool' type: platform blob2) beIndexed. >> (aTable createFieldNamed: 'timestamp' type: platform timestamp) >> beIndexed. >> ! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> descriptorForTgState: aDescriptor >> | table | >> table := self tableNamed: 'STATE'. >> aDescriptor table: table. >> (aDescriptor newMapping: OneToOneMapping) attributeName: #agent. >> (aDescriptor newMapping: DirectMapping) >> from: #version to: (table fieldNamed: 'version').! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> converterBetweenStType: aClass andField: aField >> (aClass = UUID and: [ aField impliedSmalltalkType = ByteArray]) >> ifTrue: [ ^ self uuidConverter ]. >> ^ super converterBetweenStType: aClass andField: aField! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> classModelForTgState: aClassModel >> "agent version packs valuables score bestScore answers" >> aClassModel >> newAttributeNamed: #agent type: TgAgent; >> newAttributeNamed: #version type: UUID! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> classModelForTgTool: aClassModel >> aClassModel >> newAttributeNamed: #id >> ! ! >> >> !TowergameDescriptorSystem methodsFor: 'glorp' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> classModelForTgAct: aClassModel >> aClassModel >> newAttributeNamed: #timestamp; >> newAttributeNamed: #agent type: TgAgent; >> newAttributeNamed: #tool type: TgTool! ! >> >> >> !TowergameDescriptorSystem methodsFor: 'accessing' stamp: >> 'HerbertVojÃÂÃÂk 8/14/2017 18:09:53'! >> uuidConverter >> ^ uuidConverter ifNil: [ uuidConverter := UuidConverter new name: >> 'uuid'; yourself ]! ! >> >> >> DatabaseConverter subclass: #UuidConverter >> instanceVariableNames: '' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame'! >> >> !UuidConverter methodsFor: 'converting' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> convert: anObject toDatabaseRepresentationAs: aDatabaseType >> ^ anObject ifNotNil: [ ByteArray withAll: anObject ]! ! >> >> !UuidConverter methodsFor: 'converting' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> convert: anObject fromDatabaseRepresentationAs: aDatabaseType >> ^ anObject ifNotNil: [ UUID withAll: anObject ]! ! >> 'From Pharo6.0 of 13 May 2016 [Latest update: #60510] on 14 August 2017 >> at 6:26:30.67905 pm'! >> >> !DatabasePlatform methodsFor: '*Towergame' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> blob2 >> ^self typeNamed: #blob ifAbsentPut: [GlorpBlob2Type new].! ! >> 'From Pharo6.0 of 13 May 2016 [Latest update: #60510] on 14 August 2017 >> at 6:26:30.68005 pm'! >> >> !Dictionary methodsFor: '*Towergame' stamp: 'HerbertVojÃÂÃÂk 8/14/2017 >> 18:09:53'! >> at: key ifPresentPut: aBlock >> "Lookup the given key in the receiver. If it is present, update it >> with the value of evaluating the given block with the value associated >> with the key. Otherwise, answer nil." >> >> ^ self at: key ifPresent: [ :value | self at: key put: (aBlock cull: >> value) ]! ! >> >> >> >> >> TowergameTests.st: >> >> TestCase subclass: #TowergameServerTests >> instanceVariableNames: 'randomPort towergame server' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame-Tests'! >> >> !TowergameServerTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> uidy: aString >> ^ UUID fromString36: aString ! ! >> >> !TowergameServerTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> setUp >> randomPort := 1700 + 32 atRandom. >> towergame := Mock new. >> server := Towergame serverFor: towergame on: randomPort. >> server start. >> self >> assert: server isRunning & server isListening >> description: ('Failed to start server on port {1}. Is there one >> already?' format: { server port }) >> ! ! >> >> !TowergameServerTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> tearDown >> server stop! ! >> >> >> !TowergameServerTests methodsFor: 'tests' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> testEmptySyncRequest >> | znClient response | >> (towergame stub clientSync: Arg payload) willReturn: nil. >> znClient := self znClientForSync: 'null'. >> response := znClient timeout: 1; post; response. >> response should satisfy: #isSuccess. >> response contentType should equal: ZnMimeType applicationJson. >> (STON fromString: response entity contents) should equal: nil. >> Arg payload should equal: nil! ! >> >> !TowergameServerTests methodsFor: 'tests' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> testRejectEmptyGetSyncRequest >> | znClient response | >> (towergame stub clientSync: Arg payload) willReturn: nil. >> znClient := self znClientForSync: 'null'. >> response := znClient timeout: 1; get; response. >> response code should equal: ZnStatusLine methodNotAllowed code. >> towergame should not receive clientSync: Any! ! >> >> !TowergameServerTests methodsFor: 'tests' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> testNonEmptySyncRequest >> | znClient response | >> (towergame stub clientSync: Arg payload) willReturn: { #agentId -> (self >> uidy: '007') } asDictionary. >> znClient := self znClientForSync: >> ('\{"deviceId":"{1}","agentAnsweredQuestions":\{"good":1,"bad":2\}\}' >> format: { self uidy: 'Q' }). >> response := znClient timeout: 1; post; response. >> response should satisfy: #isSuccess. >> response contentType should equal: ZnMimeType applicationJson. >> (STON fromString: response entity contents) should equal: { 'agentId' -> >> (self uidy: '007') asString } asDictionary. >> Arg payload in: [ :arg | >> arg deviceId should equal: (self uidy: 'Q'). >> arg agentAnsweredQuestions should satisfy: #notNil. >> arg agentAnsweredQuestions good should equal: 1. >> arg agentAnsweredQuestions bad should equal: 2 ] >> ! ! >> >> >> !TowergameServerTests methodsFor: 'private' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> znClientForSync: jsonString >> ^ ZnClient new >> url: server localUrl; >> path: '/api/v1/sync'; >> entity: >> (ZnEntity >> with: jsonString >> type: ZnMimeType applicationJson) >> ! ! >> >> >> TestCase subclass: #TowergameSyncTests >> instanceVariableNames: 'towergame session dao' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'Towergame-Tests'! >> >> !TowergameSyncTests methodsFor: 'tests' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> testPlayerChecksStateVersionAndIsBehind >> | result payload | >> session createTables. >> session inUnitOfWorkDo: [ >> | agent state | >> agent := TgAgent id: (self uidy: '007'). >> state := (TgState agent: agent version: (self uidy: '18-eff')) >> packs: #('foopack' 'barpack') asSet; >> valuables: (TgValuables coins: 20 gems: 3); >> score: (TgFloors total: 4 reinforced: 1); >> bestScore: (TgFloors total: 18); >> answers: (TgAnswers good: 2 bad: 3); >> yourself. >> session registerAll: {state. TgAct agent: agent tool: (TgTool id: (self >> uidy: 'Q7') ) } ]. >> towergame := Towergame dao: dao. >> payload := NeoJSONObject new >> agentId: (self uidy: '007'); stateVersion: (self uidy: '23-fefe'); >> deviceId: (self uidy: 'Q7'). >> result := towergame clientSync: payload. >> result where agentId should equal: (self uidy: '007'). >> result where stateVersion should equal: (self uidy: '18-eff'). >> result where purchasedPacks should satisfy: [ :x | x asSet should equal: >> #('foopack' 'barpack') asSet ]. >> result where valuables coins should equal: 20. >> result where valuables gems should equal: 3. >> result where floorsNumber current should equal: 4. >> result where floorsNumber best should equal: 18. >> result where floorsNumber reinforced should equal: 1. >> result where agentAnsweredQuestions good should equal: 2. >> result where agentAnsweredQuestions bad should equal: 3. >> result where totalAnsweredQuestions good should equal: 2. >> result where totalAnsweredQuestions bad should equal: 3! ! >> >> !TowergameSyncTests methodsFor: 'tests' stamp: 'HerbyVojcik 8/14/2017 >> 18:18'! >> testPlayerChecksStateVersion >> | result payload | >> session createTables. >> session inUnitOfWorkDo: [ >> | agent state | >> agent := TgAgent id: (self uidy: '007'). >> state := TgState agent: agent version: (self uidy: '23-fefe'). >> session registerAll: {state. TgAct agent: agent tool: (TgTool id: (self >> uidy: 'Q7') ) } ]. >> towergame := Towergame dao: dao. >> payload := NeoJSONObject new >> agentId: (self uidy: '007'); stateVersion: (self uidy: '23-fefe'); >> deviceId: (self uidy: 'Q7'). >> result := towergame clientSync: payload. >> result where agentId should equal: (self uidy: '007'). >> result where stateVersion should equal: (self uidy: '23-fefe'). >> result where totalAnsweredQuestions good should equal: 0. >> result where totalAnsweredQuestions bad should equal: 0! ! >> >> >> !TowergameSyncTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> uidy: aString >> ^ UUID fromString36: aString ! ! >> >> !TowergameSyncTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> setUp >> dao := Towergame daoForLogin: self loginToTemporaryDatabase. >> session := dao glorpSession. >> ! ! >> >> !TowergameSyncTests methodsFor: 'running' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> tearDown >> session logout! ! >> >> !TowergameSyncTests methodsFor: 'running' stamp: 'HerbyVojcik 8/14/2017 >> 18:16'! >> loginToTemporaryDatabase >> ^ Login new >> database: UDBCSQLite3Platform new; >> host: ''; >> port: ''; >> username: ''; >> password: ''; >> databaseName: ''; >> yourself! ! >> >> >> >> >> >> BaselineOfTowergame.st: >> BaselineOf subclass: #BaselineOfTowergame >> instanceVariableNames: '' >> classVariableNames: '' >> poolDictionaries: '' >> category: 'BaselineOfTowergame'! >> >> !BaselineOfTowergame methodsFor: 'baseline' stamp: 'HerbertVojÃÂÃÂk >> 8/14/2017 18:09:53'! >> baseline: spec >> <baseline> >> spec for: #common do: [ spec >> >> package: 'Towergame' with: [ spec >> requires: #('GlorpSQLite' 'NeoJSON') ]; >> package: 'Towergame-Tests' with: [ spec >> requires: #('Towergame' 'Mocketry') ]; >> >> configuration: 'GlorpSQLite' with: [ spec >> version: #stable; >> repository: >> 'http://smalltalkhub.com/mc/Pharo/MetaRepoForPharo60/main' ]; >> configuration: 'NeoJSON' with: [ spec >> version: #stable; >> repository: >> 'http://smalltalkhub.com/mc/Pharo/MetaRepoForPharo60/main' ]; >> baseline: 'Mocketry' with: [ spec >> repository: 'github://dionisiydk/Mocketry:v4.0.x' ]; >> >> group: 'default' with: #('Core'); >> group: 'development' with: #('Core' 'Tests'); >> group: 'Core' with: #('Towergame'); >> group: 'Tests' with: #('Towergame-Tests') ] >> ! ! >> >> >> > > >

Aug. 15, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Guillermo Polito

Just a hunch: could you inspect ur MethodChangeRecord instances ? Le mar. 15 aoÃ»t 2017 Ã 23:55, Tim Mackinnon <tim(a)testit.works> a Ã©crit : > A weird observation - is it possible that source code is being stored in > the image as strings somehow? When I do > > ./pharo PharoLambda.image eval "ByteString allInstances inject: > (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" > > I see to see reams of what looks like method source - but I thought source > code was stored in the .sources file and the .changes file (and I havenât > been bundling those in my deployed image). > > Iâm trying to figure out how you find references to a string object, to > chase down what is pointing to these strings as maybe there is a quick 4mb > win by simply nilâing out some obvious things. (This doesnât of course help > with a default minimal image - but maybe a few tricks for packaging and > deploying something). > > Tim > > On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: > > Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it > seems very convoluted to load a single package into the image, I was trying > to avoid having to create a baselineOf for something so simple - I ended up > with: > > repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. > > version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. > version load. > > > Anyway - in my minimal image, like in the fat image there seems to be a > surprising amount of bytestrings (4mb worth?). I think that might need some > digging into? It seems like a lot somehow. Although Benâs neat experiment > of zipping strings shows thatâs not a real route. > > In a deployed minimal image - maybe I can get rid of some other things > like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins > - but noticeable) > > Class code space # instances > inst space percent inst average size > ByteString 2640 37365 > 4823848 21.50 129.10 > Array 3742 53002 > 3961944 17.60 74.75 > CompiledMethod 19159 30481 > 2912968 13.00 95.57 > Association 1148 58348 > 1867136 8.30 32.00 > MethodChangeRecord 431 34312 > 1097984 4.90 32.00 > ByteArray 4605 290 > 908728 4.00 3133.54 > ByteSymbol 1698 22689 > 840168 3.70 37.03 > IdentitySet 408 19076 > 610432 2.70 32.00 > MethodDictionary 3310 3520 > 608688 2.70 172.92 > WeakArray 1758 3024 > 597824 2.70 197.69 > MCMethodDefinition 4318 6659 > 426176 1.90 64.00 > Protocol 1679 8382 > 268224 1.20 32.00 > OrderedCollection 6555 5509 > 220360 1.00 40.00 > > As an aside - my Gitlab project is public, the scripts that load things up > are in ./scripts (build.sh, and minimal.st and loadlocal.st) > > Tim > > On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com> > wrote: > > > > On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works> wrote: > >> Hi Guille - just running SpaceTally on my dev image to get a feel for it. >> It turns out that in the minimal images youâve been creating, its not >> loaded (makes sense). >> > > Yup, it's loaded afterwards. > > All packages are loaded through metacello baselines. We should start > refactoring and making standalone projects, each one with a baseline for > himself, and his own dependencies described. > > I was checking on your gitlab and I have probably no access: how are you > finally loading packages in the bootstrap image? Can you share that with us > in text? I'd like to improve that situation. > > >> Iâm wondering if there is an easy way to import it in (I guess that >> package should be in the Pharo git tree I cloned to get Fuel loaded right? >> Or is there a separate standalone source?). >> > > Yes it is, you can get the package programatically doing > > SpaceTally package name > > And furthermore, get the baseline that currently is loading by doing > > package := SpaceTally package name. > BaselineOf subclasses select: [ :e | > e project version packages anySatisfy: [ :p | p name = package ]]. > > >> >> Thanks for all the support, and your email about why the contexts stack >> up is very well received (I will comment over there). >> >> By the way - it looks like Martin Fowler picked up on this announcement - >> so maybe we might get some interest from his mass of followers. >> >> Tim >> >> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com> >> wrote: >> >> Hi Tim, >> >> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works> wrote: >> >>> Hey guys, thanks for your enthusiasm around this - and I cannot stress >>> enough how this was only possible because of the work that has gone into >>> making Pharo (in particular the 64bit image, as well as having a minimal >>> image, and some great blog posts on serialising contexts) as well as the >>> patience from everyone in answering questions and helping me get it all >>> working. >>> >>> Iâm still quite keen to get my execution time back down under 800ms and >>> Iâd like to actually get back to writing a few skills to automate a few >>> things around my house. >>> >>> To Answer Denisâ question - >>> >>> My final footprint is 30.4mb - thats composed of a 22mb image (with a >>> simple example that pulls in Fuel, ZTimestamp and the S3 Library which >>> depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>> >>> In my original experiments with a 6.0 minimal image - I did manage to >>> get to a 13.4mb image (which started out as 12mb original size, and then >>> loaded in STON and had only a simple clock example). I think the sweet spot >>> is around 20mb total footprint as that seems to get me into the 450ms-900ms >>> range. >>> >>> The 7.0 min image now starts out at 15mb and then Iâm not sure why >>> loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not >>> dug into that). >>> >> >> You can do further space analysis using the following expression >> >> SpaceTally new printSpaceAnalysis >> >> You can do that in an eval and check what's taking space. With measures >> we can iterate and improve :). >> >> >>> Iâve also found (and this on the back of unserialising the context in my >>> example) that the way we build images has 15+ saved stack sessions that >>> have saved on top of each other from the way we build up the images. I >>> donât yet know the implications of size/speed of these - but we need a >>> better way of folding executions when we snapshot headless images. Iâm also >>> not clear if there are any other startup tasks that take precious time >>> (this also has implications for our fat development images as they take >>> much longer to appear than they really should). >>> >> >> I'm working on this as I'm writing this mail ;) >> >> https://pharo.fogbugz.com/f/cases/20309 >> https://github.com/pharo-project/pharo/pull/196 >> >> I'll write down the implications further in a different thread. >> >> >>> Iâll be exploring some of these size/speed tradeoffâs in follow on >>> messages. >>> >>> But once again, a big thanks - Iâve not enjoyed programming like this >>> for ages. >>> >>> Tim >>> >>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com> wrote: >>> >>> hi Tim, >>> >>> That is..... AWESOME! >>> >>> Very nice delivery - it flowed well with great narration. >>> >>> I loved @2:17 "this is the interesting piece, because PharoLambda has >>> serialized the execution context of its application and saved it into [my >>> S3 bucket] ... [then on the local machine] rematerializes a debugger [on >>> that context]." >>> >>> There is a clarity in your video presentation that really may intrigue >>> outsiders. As a community we should push this on the usual hacker forums - >>> ycombinator could be a good starting point (but I'm locked out of my >>> account there). >>> An enticing title could be... >>> "Debugging Lambdas by re-materializing saved execution contexts on your >>> local machine." >>> >>> cheers -ben >>> >>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com> >>> wrote: >>> >>>> This is cool Tim. >>>> >>>> So what image size you deployed at the end? >>>> >>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works>: >>>> >>>>> I just wanted to thank everyone for their help in getting my pet >>>>> project further along, so that now I can announce that PharoLambda is now >>>>> working with the V7 minimal image and also supports post mortem debugging >>>>> by saving a zipped fuel context onto S3. >>>>> >>>>> This latter item is particularly satisfying as at a recent serverless >>>>> conference (JeffConf) there was a panel where poor development tools on >>>>> serverless platforms was highlighted as a real problem. >>>>> >>>>> In our community weâve had these kinds of tools at our fingertips for >>>>> ages - but I donât think the wider development community has really >>>>> noticed. Debugging something short lived like a Lambda execution is quite >>>>> startling, as the current answer is âadd more loggingâ, and we all know >>>>> that sucks. To this end, Iâve created a little screencast showing this in >>>>> action - and it was pretty cool because it was a real example I encountered >>>>> when I got everything working and was trying my test application out. >>>>> >>>>> Iâve also put a bit of work into tuning the excellent GitLab CI tools, >>>>> so that I can cache many of the artefacts used between different build runs >>>>> (this might also be of interest to others using CI systems). >>>>> >>>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda >>>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E >>>>> >>>>> Tim >>>>> >>>>> >>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works> wrote: >>>>> >>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS >>>>> Lambda. Itâs early days, but I though it might be interesting to share what >>>>> Iâve learned so far. >>>>> >>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda >>>>> >>>>> With help from many of the folks here, Iâve been able to get a simple >>>>> example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily >>>>> try it out yourself. This seems slightly better than what the GoLang folks >>>>> have been able to do. >>>>> >>>>> Tim >>>>> >>>>> >>>>> >>>> >>> >>> >> >> >> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - *http://www.cnrs.fr* >> <http://www.cnrs.fr/> >> >> >> *Web:* *http://guillep.github.io* <http://guillep.github.io/> >> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> >> >> >> > > > -- > > Guille Polito > > Research Engineer > French National Center for Scientific Research - *http://www.cnrs.fr* > <http://www.cnrs.fr/> > > > *Web:* *http://guillep.github.io* <http://guillep.github.io/> > *Phone: *+33 06 52 70 66 13 > > > > -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Aug. 15, 2017

Re: [Pharo-users] [ANN] PharoLambda 1.5 - Pharo running on AWS Lambda now with saved Debug sessions via S3

by Tim Mackinnon

A weird observation - is it possible that source code is being stored in the image as strings somehow? When I do ./pharo PharoLambda.image eval "ByteString allInstances inject: (OrderedCollection new) into: [:r :i | i size > 500 ifTrue: [r add: i]. r]" I see to see reams of what looks like method source - but I thought source code was stored in the .sources file and the .changes file (and I havenât been bundling those in my deployed image). Iâm trying to figure out how you find references to a string object, to chase down what is pointing to these strings as maybe there is a quick 4mb win by simply nilâing out some obvious things. (This doesnât of course help with a default minimal image - but maybe a few tricks for packaging and deploying something). Tim > On 15 Aug 2017, at 22:26, Tim Mackinnon <tim(a)testit.works> wrote: > > Hi Guille/Ben - I got a quick moment to try the SpaceTally (aside: it seems very convoluted to load a single package into the image, I was trying to avoid having to create a baselineOf for something so simple - I ended up with: > > repo := MCFileTreeRepository new directory: './bootstrap' asFileReference. > version := repo loadVersionFromFileNamed: 'Tool-Profilers.package'. > version load. > > Anyway - in my minimal image, like in the fat image there seems to be a surprising amount of bytestrings (4mb worth?). I think that might need some digging into? It seems like a lot somehow. Although Benâs neat experiment of zipping strings shows thatâs not a real route. > > In a deployed minimal image - maybe I can get rid of some other things like MethodChangeRecords or MCMethodDefiniionâs (but they are smaller wins - but noticeable) > > Class code space # instances inst space percent inst average size > ByteString 2640 37365 4823848 21.50 129.10 > Array 3742 53002 3961944 17.60 74.75 > CompiledMethod 19159 30481 2912968 13.00 95.57 > Association 1148 58348 1867136 8.30 32.00 > MethodChangeRecord 431 34312 1097984 4.90 32.00 > ByteArray 4605 290 908728 4.00 3133.54 > ByteSymbol 1698 22689 840168 3.70 37.03 > IdentitySet 408 19076 610432 2.70 32.00 > MethodDictionary 3310 3520 608688 2.70 172.92 > WeakArray 1758 3024 597824 2.70 197.69 > MCMethodDefinition 4318 6659 426176 1.90 64.00 > Protocol 1679 8382 268224 1.20 32.00 > OrderedCollection 6555 5509 220360 1.00 40.00 > > As an aside - my Gitlab project is public, the scripts that load things up are in ./scripts (build.sh, and minimal.st <http://minimal.st/> and loadlocal.st <http://loadlocal.st/>) > > Tim > >> On 15 Aug 2017, at 08:02, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: >> >> >> >> On Mon, Aug 14, 2017 at 4:42 PM, Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>> wrote: >> Hi Guille - just running SpaceTally on my dev image to get a feel for it. It turns out that in the minimal images youâve been creating, its not loaded (makes sense). >> >> Yup, it's loaded afterwards. >> >> All packages are loaded through metacello baselines. We should start refactoring and making standalone projects, each one with a baseline for himself, and his own dependencies described. >> >> I was checking on your gitlab and I have probably no access: how are you finally loading packages in the bootstrap image? Can you share that with us in text? I'd like to improve that situation. >> >> Iâm wondering if there is an easy way to import it in (I guess that package should be in the Pharo git tree I cloned to get Fuel loaded right? Or is there a separate standalone source?). >> >> Yes it is, you can get the package programatically doing >> >> SpaceTally package name >> >> And furthermore, get the baseline that currently is loading by doing >> >> package := SpaceTally package name. >> BaselineOf subclasses select: [ :e | >> e project version packages anySatisfy: [ :p | p name = package ]]. >> >> >> Thanks for all the support, and your email about why the contexts stack up is very well received (I will comment over there). >> >> By the way - it looks like Martin Fowler picked up on this announcement - so maybe we might get some interest from his mass of followers. >> >> Tim >> >>> On 14 Aug 2017, at 10:49, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: >>> >>> Hi Tim, >>> >>> On Mon, Aug 14, 2017 at 11:41 AM, Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>> wrote: >>> Hey guys, thanks for your enthusiasm around this - and I cannot stress enough how this was only possible because of the work that has gone into making Pharo (in particular the 64bit image, as well as having a minimal image, and some great blog posts on serialising contexts) as well as the patience from everyone in answering questions and helping me get it all working. >>> >>> Iâm still quite keen to get my execution time back down under 800ms and Iâd like to actually get back to writing a few skills to automate a few things around my house. >>> >>> To Answer Denisâ question - >>> >>> My final footprint is 30.4mb - thats composed of a 22mb image (with a simple example that pulls in Fuel, ZTimestamp and the S3 Library which depends on XMLParser) and then the VM (from which I removed obvious dllâs). >>> >>> In my original experiments with a 6.0 minimal image - I did manage to get to a 13.4mb image (which started out as 12mb original size, and then loaded in STON and had only a simple clock example). I think the sweet spot is around 20mb total footprint as that seems to get me into the 450ms-900ms range. >>> >>> The 7.0 min image now starts out at 15mb and then Iâm not sure why loading Fuel, S3 and XMLParser takes 7mb (it seems big to me - but Iâve not dug into that). >>> >>> You can do further space analysis using the following expression >>> >>> SpaceTally new printSpaceAnalysis >>> >>> You can do that in an eval and check what's taking space. With measures we can iterate and improve :). >>> >>> Iâve also found (and this on the back of unserialising the context in my example) that the way we build images has 15+ saved stack sessions that have saved on top of each other from the way we build up the images. I donât yet know the implications of size/speed of these - but we need a better way of folding executions when we snapshot headless images. Iâm also not clear if there are any other startup tasks that take precious time (this also has implications for our fat development images as they take much longer to appear than they really should). >>> >>> I'm working on this as I'm writing this mail ;) >>> >>> https://pharo.fogbugz.com/f/cases/20309 <https://pharo.fogbugz.com/f/cases/20309> >>> https://github.com/pharo-project/pharo/pull/196 <https://github.com/pharo-project/pharo/pull/196> >>> >>> I'll write down the implications further in a different thread. >>> >>> >>> Iâll be exploring some of these size/speed tradeoffâs in follow on messages. >>> >>> But once again, a big thanks - Iâve not enjoyed programming like this for ages. >>> >>> Tim >>> >>>> On 12 Aug 2017, at 16:26, Ben Coman <btc(a)openInWorld.com <mailto:btc@openInWorld.com>> wrote: >>>> >>>> hi Tim, >>>> >>>> That is..... AWESOME! >>>> >>>> Very nice delivery - it flowed well with great narration. >>>> >>>> I loved @2:17 "this is the interesting piece, because PharoLambda has serialized the execution context of its application and saved it into [my S3 bucket] ... [then on the local machine] rematerializes a debugger [on that context]." >>>> >>>> There is a clarity in your video presentation that really may intrigue outsiders. As a community we should push this on the usual hacker forums - ycombinator could be a good starting point (but I'm locked out of my account there). >>>> An enticing title could be... >>>> "Debugging Lambdas by re-materializing saved execution contexts on your local machine." >>>> >>>> cheers -ben >>>> >>>> On Fri, Aug 11, 2017 at 3:37 PM, Denis Kudriashov <dionisiydk(a)gmail.com <mailto:dionisiydk@gmail.com>> wrote: >>>> This is cool Tim. >>>> >>>> So what image size you deployed at the end? >>>> >>>> 2017-08-10 15:47 GMT+02:00 Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>>: >>>> I just wanted to thank everyone for their help in getting my pet project further along, so that now I can announce that PharoLambda is now working with the V7 minimal image and also supports post mortem debugging by saving a zipped fuel context onto S3. >>>> >>>> This latter item is particularly satisfying as at a recent serverless conference (JeffConf) there was a panel where poor development tools on serverless platforms was highlighted as a real problem. >>>> >>>> In our community weâve had these kinds of tools at our fingertips for ages - but I donât think the wider development community has really noticed. Debugging something short lived like a Lambda execution is quite startling, as the current answer is âadd more loggingâ, and we all know that sucks. To this end, Iâve created a little screencast showing this in action - and it was pretty cool because it was a real example I encountered when I got everything working and was trying my test application out. >>>> >>>> Iâve also put a bit of work into tuning the excellent GitLab CI tools, so that I can cache many of the artefacts used between different build runs (this might also be of interest to others using CI systems). >>>> >>>> The Gitlab project is on: https://gitlab.com/macta/PharoLambda <https://gitlab.com/macta/PharoLambda> >>>> And the screencast: https://www.youtube.com/watch?v=bNNCT1hLA3E <https://www.youtube.com/watch?v=bNNCT1hLA3E> >>>> >>>> Tim >>>> >>>> >>>>> On 15 Jul 2017, at 00:39, Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>> wrote: >>>>> >>>>> Hi - Iâve been playing around with getting Pharo to run well on AWS Lambda. Itâs early days, but I though it might be interesting to share what Iâve learned so far. >>>>> >>>>> Usage examples and code at https://gitlab.com/macta/PharoLambda <https://gitlab.com/macta/PharoLambda> >>>>> >>>>> With help from many of the folks here, Iâve been able to get a simple example to run in 500ms-1200ms with a minimal Pharo 6 image. You can easily try it out yourself. This seems slightly better than what the GoLang folks have been able to do. >>>>> >>>>> Tim >>>> >>>> >>>> >>> >>> >>> >>> >>> -- >>> >>> Guille Polito >>> >>> Research Engineer >>> French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> >>> >>> >>> Web: http://guillep.github.io <http://guillep.github.io/> >>> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013> >> >> >> >> -- >> >> Guille Polito >> >> Research Engineer >> French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/> >> >> >> Web: http://guillep.github.io <http://guillep.github.io/> >> Phone: +33 06 52 70 66 13 >

Aug. 15, 2017