- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [Vm-dev] ImageSegments and some OT (was: Re: Squeak 5 Image Segment Work [was [squeak-dev] [ANN] Squeak 5])

by Mariano Martinez Peck

On Thu, Aug 20, 2015 at 11:37 AM, Tobias Pape <Das.Linux(a)gmx.de> wrote: > > Hi all, > > On 19.08.2015, at 23:09, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > Hi All, but especially Tobias, > > just for reference, I only had time to skim this mail. > I'll work through it more in-depth in September (vacation upcoming, sorry). > > Also, I like Fuel as a way forward. I'd also like to deprecate DataStreams > in > favor of Fuel[1], I also had a prototype of Fuel-based Monticello packages > which loaded much faster for large packagesâ¦ > > Is this similar to Tanker in the way that classes and methods are binary serialized with Fuel? (no compiling) Or you just used Fuel instead of the current usage of MCDataStream? If this is the case, then you would still be compiling at load time etc. We already did this experiment long ago and we didn't find a significant speedup. What were your findings? > Best regards > -Tobias > > > [1]: PS: Hi Pharo folks, MCDataStream will break with Spur, > Have a look at System-topa.741 unil System-topa.473 in the > Squeak trunk or the attached change set, YMMV tho. > > Thanks, > > > > > > > > as we know, ImageSegments are broken in Spur. They work > occasionally, but when stressed the VM ends up crashing. The fundamental > issue is that the ImageSegment code makes assumptions about object ordering > that Spur violates. For example, here's ImageSegment's install: > > > > install > > "This operation retrieves the segment if necessary from file > storage, installs it in memory, and replaces (using become:) all the root > stubs with the reconstructed roots of the segment." > > > > | newRoots | > > state = #onFile ifTrue: [self readFromFile]. > > state = #onFileWithSymbols ifTrue: [self readFromFileWithSymbols. > > endMarker := segment nextObject. "for enumeration > of objects" > > endMarker == 0 ifTrue: [endMarker := 'End' clone]]. > > (state = #active) | (state = #imported) ifFalse: [self > errorWrongState]. > > newRoots := self loadSegmentFrom: segment outPointers: outPointers. > > state = #imported > > ifTrue: ["just came in from exported file" > > arrayOfRoots := newRoots] > > ifFalse: [ > > arrayOfRoots elementsForwardIdentityTo: newRoots]. > > state := #inactive. > > Beeper beepPrimitive > > > > So before the image segment bytes (the segment inst var) is loaded, the > object after it is assigned to endMarker, and if there isn't an object > after segment, a new object ('End' clone) is assigned to endMarker. > > > > This makes the assumption that objects are allocated in a strict order, > and therefore endMarker will always be the object after segment. > > > > Loading the segment via "newRoots := self loadSegmentFrom: segment > outPointers: outPointers" then turns segment into a zero-length WordArray > and its contents into the objects loaded by the segment. Therefore, in the > V3 system, the objects loaded from segment can be enumerated starting at > segment nextObject and repeating until endMarker is found: > > > > allObjectsDo: aBlock > > "Enumerate all objects that came from this segment. NOTE this > assumes that the segment was created (and extracted). After the segment > has been installed (install), this method allows you to enumerate its > objects." > > | obj | > > > > endMarker == nil ifTrue: [ > > ^ self error: 'Just extract and install, don''t > writeToFile:']. > > segment size ~= 1 ifTrue: [ > > ^ self error: 'Vestigial segment size must be 1 (version > word)']. > > > > obj := segment nextObject. "Start with the next object after the > vestigial header" > > [obj == endMarker] whileFalse: "Stop at the next object after the > full segment" > > [aBlock value: obj. > > obj := obj nextObject]. "Step through the objects > installed from the segment." > > > > Now, as written, this just won't work in Spur. > > > > a) the only place where there is any kind of stable order to objects is > in oldSpace, so segment /has/ to be forced to old space to have any chance > of its objects being in order when it gets converted from bytes to objects. > > > > b) 'End' clone will be in newSpace and so endMarker isn't reliable > unless it was obtained by segment nextObject when segment was already in > oldSpace. > > > > So it is perhaps possible to fix ImageSegments in Spur by forcing > segment to oldSpace and being more careful with endMarker. But I think > there is a better way. > > > > If the set of objects the segment contains can be obtained some how then > this set can be simply enumerated, not depending on nextObject. The > primitive has to answer the array of roots, so its result can't be changed > to be the entire array. But segment could be becomed into an Array of all > the objects in segment prior to it being loaded, in which case the above > would become > > > > > > install > > "This operation retrieves the segment if necessary from file > storage, installs it in memory, and replaces (using become:) all the root > stubs with the reconstructed roots of the segment." > > > > | newRoots | > > state = #onFile ifTrue: [self readFromFile]. > > state = #onFileWithSymbols ifTrue: > > [self readFromFileWithSymbols]. > > (state = #active) | (state = #imported) ifFalse: [self > errorWrongState]. > > newRoots := self loadSegmentFrom: segment outPointers: outPointers. > > state = #imported > > ifTrue: "just came in from exported file" > > [arrayOfRoots := newRoots] > > ifFalse: > > [arrayOfRoots elementsForwardIdentityTo: newRoots]. > > state := #inactive. > > Beeper beepPrimitive > > > > allObjectsDo: aBlock > > "Enumerate all objects that came from this segment. NOTE this > assumes that the segment was created (and extracted). After the segment > has been installed (install), this method allows you to enumerate its > objects." > > | obj | > > > > segment isArray ifFalse: > > [^ self error: 'Segment hasn''t been loaded?']. > > > > segment do: aBlock > > > > and the endMarker instVar would be deleted. > > > > I am willing and ready to modify the primitive to convert the segment > correctly. Who will volunteer to rewrite the image-level ImageSegment code > to use the new primitive? > > > > > > > > On Wed, Aug 12, 2015 at 10:40 PM, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > Hi Tobias, > > > > On Wed, Aug 12, 2015 at 10:18 PM, Tobias Pape <Das.Linux(a)gmx.de> wrote: > > Hi all > > On 13.08.2015, at 02:15, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > > Hi Tobias, > > > > > > forget that. I found it. THis right? > > > > > > Trunk test suite for Spur > > > Using existing cogspur r.3410 > > > cp -r /var/lib/jenkins/workspace/Trunk/default/target/cogspur.r3410 > /tmp/d20150812-28620-etbikj > > > image test suite > > > VM: /tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak > > > /tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak > -version > > > /var/lib/jenkins/workspace/Trunk/default/tests.st > > > spawning command 0 with timeout 1800 seconds: > "/tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak" > "-vm-sound-null" "-vm-display-null" > "/var/lib/jenkins/workspace/Trunk/default/target/SpurPostTestTrunkImage.image" > "../tests.st" > > > (Command started with PID 28643) > > > 2015-08-12T23:31:48.17+01:00: Loading Hudson build tools... from > /var/lib/jenkins/workspace/Trunk/default/target/HudsonBuildTools.st > > > 2015-08-12T23:31:48.388+01:00: Running tests... > > > setsockopt: Protocol not available > > > setsockopt: Protocol not available > > > 28646:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown > protocol:s23_clnt.c:612: > > > > > > Recursive not understood error encountered > > > > > > > > > I bet this is ImageSegment related. > > > > It sure is. > > > > BitmapStreamTests>testMatrixTransform2x3WithImageSegment > > sends > > BitmapStreamTests>validateImageSegment > > > > Basically the ImageSegment code has been working in Spur on a hope and > a prayer. The code assumes objects are allocated in chronological order > and that this order is preserved, not so with Spur. So post image segment > loading someObject/nextObject is used to enumerate the objects loaded. > This can't work reliably in Spur. I *think* (ok I hope) that I've > implemented the segment load primitive in Spur to answer an Array of the > objects loaded, so that these can be explicitly enumerated. > > > > So the job is a) to check that I have indeed implemented the primitive > to do this and b) to rewrite the image segment loading code in the light of > this. > > > > David, this is an example of something that isn't back portable and > should not be back-ported. > > > > The most strange thing is that I cannot reproduce this on my Macâ¦ > > There the test does not crash the imageâ¦ > > > > Well then it may be a signed/unsigned bug in image loading instead. On > linux the image typically gets loaded quite high in the address space and > so a good portion of the heap ends up above 0x7FFFFFFF, or negative > territory if misinterpreted as signed integers. On Mac and Windows the > image tends to get loaded quite low and so has to be pretty big to fall > foul of signedness issues. > > > > > > Busy right now but will check tomorrow. > > > > > > Best regards > > -Tobias > > > > > > > > > > On Wed, Aug 12, 2015 at 5:11 PM, Eliot Miranda < > eliot.miranda(a)gmail.com> wrote: > > > Hi Tobias, > > > > > > On Wed, Aug 12, 2015 at 12:49 PM, Tobias Pape <Das.Linux(a)gmx.de> > wrote: > > > > > > On 12.08.2015, at 20:55, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > > > > > > Hi All, > > > > > > > > > > > > Fabio's kindly done most of the changes. But some questions > remain for more general discussion. See below. > > > > > > > > Fabio, thanks so much for doing this, and so quickly! > > > > > > > > On Tue, Aug 11, 2015 at 8:51 PM, Eliot Miranda < > eliot.miranda(a)gmail.com> wrote: > > > > Hi All, > > > > > > > > who will update http://squeak.org/downloads/ to include the 5.0 > release? > > > > > > > > a) I suggest that the 5.0 all-in-one have a line in the left-hand > table that includes the 4.6 release and that it precede the 4.6 release in > the list. > > > > > > > > Done. > > > > > > > > > > > > b) the Trunk link points to TrunkImage.zip which contains a non-Spur > image. > > > > > > > > So this is the biggie. What should we do? > > > > > > > > - add a Trunk 5.0 build with a different link? (Noooooo) > > > > > > > > - change http://build.squeak.org/job/SqueakTrunk to build what it > says (Yessss please, who can do this?) > > > > > > Done: build.squeak.org/job/Trunk/ > > > But as I said several times, Spur/trunk test just crash for monthâ¦ > > > > > > can you point me to the crash? I'm looking at > http://build.squeak.org/job/Trunk/default/lastBuild/#showFailuresLink and > see 8 test failures but no crash. > > > > > > > > > > > > > > - add a Squeak 4.6 build job? Is that even worth it any more > considering 4.6 is released and stable? If it is then what should it > build? David? > > > > > > > > > > > > c) Spur VMs need to be linked to by an added line in the Virtual > Machines list. > > > > > > > > Not applicable. the VM links point to the root of my site so they > effectively point to both old and Spur VMs. > > > > > > > > > > > > d) Spur imagers need to be included in the Image and Changes list > under Custom Installation > > > > > > > > Done. > > > > > > > > > > > > > > > > e) The SqueakV50.sources file also needs to be included in the > Current Sources list. > > > > > > > > Done. > > > > > > > > > > > > f) we could start a History list for the 5.0 line > > > > > > > > This is probably quite a big reorg on the page and not urgent. > > > > > > > > > > > > So there are a few things to fix before 5.0 is freely downloadable. > > > > > > > > On Tue, Aug 11, 2015 at 8:23 PM, Chris Muller <ma.chris.m(a)gmail.com> > wrote: > > > > In the 17 months since Squeak 4.5 was released, a huge development > > > > effort took place to create the next generation virtual-machine for > > > > the Squeak / Pharo / Newspeak family of programming systems. Squeak > > > > is the modern incarnation of the Smalltalk-80 programming environment > > > > originally developed at the Xerox PARC. > > > > > > > > "Squeak 5" introduces this new VM and associated new memory model, > > > > collectively referred to as "Spur". Presented [1] by Eliot Miranda > > > > and ClÃ©ment BÃ©ra at the 2015 International Symposium on Memory > > > > Management, this new VM affords Squeak applications a significant > > > > boost in performance and memory management. Among other > > > > optimizations, the #become operation no longer requires a memory > scan. > > > > Object pinning and ephemerons are also now supported. The release > > > > notes [2] provide more details. > > > > > > > > The new memory model requires a new image file format. Although this > > > > new format results in about a 15% increased memory requirement for > the > > > > same number of 4.x objects, a new segmented heap allows memory to be > > > > given back to the OS when its no longer needed, a great benefit for > > > > application servers. > > > > > > > > As forward compatibility is as important to the Squeak community as > > > > backward compatibility, Squeak 5 is delivers an image with identical > > > > content as the recent 4.6 release. Although this new Squeak 5 VM > > > > cannot open images saved under the prior 4.x Cog format, objects and > > > > code can be easily exported from the 4.x image and then imported into > > > > Squeak 5. Applications whose code runs strictly above the Smalltalk > > > > meta layer will prove remarkably compatible with the new format, most > > > > applications will require no changes whatsotever. > > > > > > > > Squeak 5 is the result of monumental effort by a tiny group of very > > > > talented people, but its also just the beginning of yet a new effort; > > > > Spur is just a stepping stone to a more ambitious goals planned over > > > > the next five years. > > > > > > > > [1] -- A Partial Read Barrier for Efficient Support of Live > > > > Object-oriented Programming > > > > > http://conf.researchr.org/event/ismm-2015/ismm-2015-papers-a-partial-read-b… > > > > > > > > [2] -- Squeak 5 Release Notes > > > > http://wiki.squeak.org/squeak/6207 > > > > > -- Mariano http://marianopeck.wordpress.com

Aug. 20, 2015

ImageSegments and some OT (was: Re: Squeak 5 Image Segment Work [was [squeak-dev] [ANN] Squeak 5])

by Tobias Pape

Hi all, On 19.08.2015, at 23:09, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi All, but especially Tobias, just for reference, I only had time to skim this mail. I'll work through it more in-depth in September (vacation upcoming, sorry). Also, I like Fuel as a way forward. I'd also like to deprecate DataStreams in favor of Fuel[1], I also had a prototype of Fuel-based Monticello packages which loaded much faster for large packagesâ¦ Best regards -Tobias [1]: PS: Hi Pharo folks, MCDataStream will break with Spur, Have a look at System-topa.741 unil System-topa.473 in the Squeak trunk or the attached change set, YMMV tho. > > as we know, ImageSegments are broken in Spur. They work occasionally, but when stressed the VM ends up crashing. The fundamental issue is that the ImageSegment code makes assumptions about object ordering that Spur violates. For example, here's ImageSegment's install: > > install > "This operation retrieves the segment if necessary from file storage, installs it in memory, and replaces (using become:) all the root stubs with the reconstructed roots of the segment." > > | newRoots | > state = #onFile ifTrue: [self readFromFile]. > state = #onFileWithSymbols ifTrue: [self readFromFileWithSymbols. > endMarker := segment nextObject. "for enumeration of objects" > endMarker == 0 ifTrue: [endMarker := 'End' clone]]. > (state = #active) | (state = #imported) ifFalse: [self errorWrongState]. > newRoots := self loadSegmentFrom: segment outPointers: outPointers. > state = #imported > ifTrue: ["just came in from exported file" > arrayOfRoots := newRoots] > ifFalse: [ > arrayOfRoots elementsForwardIdentityTo: newRoots]. > state := #inactive. > Beeper beepPrimitive > > So before the image segment bytes (the segment inst var) is loaded, the object after it is assigned to endMarker, and if there isn't an object after segment, a new object ('End' clone) is assigned to endMarker. > > This makes the assumption that objects are allocated in a strict order, and therefore endMarker will always be the object after segment. > > Loading the segment via "newRoots := self loadSegmentFrom: segment outPointers: outPointers" then turns segment into a zero-length WordArray and its contents into the objects loaded by the segment. Therefore, in the V3 system, the objects loaded from segment can be enumerated starting at segment nextObject and repeating until endMarker is found: > > allObjectsDo: aBlock > "Enumerate all objects that came from this segment. NOTE this assumes that the segment was created (and extracted). After the segment has been installed (install), this method allows you to enumerate its objects." > | obj | > > endMarker == nil ifTrue: [ > ^ self error: 'Just extract and install, don''t writeToFile:']. > segment size ~= 1 ifTrue: [ > ^ self error: 'Vestigial segment size must be 1 (version word)']. > > obj := segment nextObject. "Start with the next object after the vestigial header" > [obj == endMarker] whileFalse: "Stop at the next object after the full segment" > [aBlock value: obj. > obj := obj nextObject]. "Step through the objects installed from the segment." > > Now, as written, this just won't work in Spur. > > a) the only place where there is any kind of stable order to objects is in oldSpace, so segment /has/ to be forced to old space to have any chance of its objects being in order when it gets converted from bytes to objects. > > b) 'End' clone will be in newSpace and so endMarker isn't reliable unless it was obtained by segment nextObject when segment was already in oldSpace. > > So it is perhaps possible to fix ImageSegments in Spur by forcing segment to oldSpace and being more careful with endMarker. But I think there is a better way. > > If the set of objects the segment contains can be obtained some how then this set can be simply enumerated, not depending on nextObject. The primitive has to answer the array of roots, so its result can't be changed to be the entire array. But segment could be becomed into an Array of all the objects in segment prior to it being loaded, in which case the above would become > > > install > "This operation retrieves the segment if necessary from file storage, installs it in memory, and replaces (using become:) all the root stubs with the reconstructed roots of the segment." > > | newRoots | > state = #onFile ifTrue: [self readFromFile]. > state = #onFileWithSymbols ifTrue: > [self readFromFileWithSymbols]. > (state = #active) | (state = #imported) ifFalse: [self errorWrongState]. > newRoots := self loadSegmentFrom: segment outPointers: outPointers. > state = #imported > ifTrue: "just came in from exported file" > [arrayOfRoots := newRoots] > ifFalse: > [arrayOfRoots elementsForwardIdentityTo: newRoots]. > state := #inactive. > Beeper beepPrimitive > > allObjectsDo: aBlock > "Enumerate all objects that came from this segment. NOTE this assumes that the segment was created (and extracted). After the segment has been installed (install), this method allows you to enumerate its objects." > | obj | > > segment isArray ifFalse: > [^ self error: 'Segment hasn''t been loaded?']. > > segment do: aBlock > > and the endMarker instVar would be deleted. > > I am willing and ready to modify the primitive to convert the segment correctly. Who will volunteer to rewrite the image-level ImageSegment code to use the new primitive? > > > > On Wed, Aug 12, 2015 at 10:40 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Tobias, > > On Wed, Aug 12, 2015 at 10:18 PM, Tobias Pape <Das.Linux(a)gmx.de> wrote: > Hi all > On 13.08.2015, at 02:15, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > Hi Tobias, > > > > forget that. I found it. THis right? > > > > Trunk test suite for Spur > > Using existing cogspur r.3410 > > cp -r /var/lib/jenkins/workspace/Trunk/default/target/cogspur.r3410 /tmp/d20150812-28620-etbikj > > image test suite > > VM: /tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak > > /tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak -version > > /var/lib/jenkins/workspace/Trunk/default/tests.st > > spawning command 0 with timeout 1800 seconds: "/tmp/d20150812-28620-etbikj/cogspur.r3410/cogspurlinuxht/bin/squeak" "-vm-sound-null" "-vm-display-null" "/var/lib/jenkins/workspace/Trunk/default/target/SpurPostTestTrunkImage.image" "../tests.st" > > (Command started with PID 28643) > > 2015-08-12T23:31:48.17+01:00: Loading Hudson build tools... from /var/lib/jenkins/workspace/Trunk/default/target/HudsonBuildTools.st > > 2015-08-12T23:31:48.388+01:00: Running tests... > > setsockopt: Protocol not available > > setsockopt: Protocol not available > > 28646:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol:s23_clnt.c:612: > > > > Recursive not understood error encountered > > > > > > I bet this is ImageSegment related. > > It sure is. > > BitmapStreamTests>testMatrixTransform2x3WithImageSegment > sends > BitmapStreamTests>validateImageSegment > > Basically the ImageSegment code has been working in Spur on a hope and a prayer. The code assumes objects are allocated in chronological order and that this order is preserved, not so with Spur. So post image segment loading someObject/nextObject is used to enumerate the objects loaded. This can't work reliably in Spur. I *think* (ok I hope) that I've implemented the segment load primitive in Spur to answer an Array of the objects loaded, so that these can be explicitly enumerated. > > So the job is a) to check that I have indeed implemented the primitive to do this and b) to rewrite the image segment loading code in the light of this. > > David, this is an example of something that isn't back portable and should not be back-ported. > > The most strange thing is that I cannot reproduce this on my Macâ¦ > There the test does not crash the imageâ¦ > > Well then it may be a signed/unsigned bug in image loading instead. On linux the image typically gets loaded quite high in the address space and so a good portion of the heap ends up above 0x7FFFFFFF, or negative territory if misinterpreted as signed integers. On Mac and Windows the image tends to get loaded quite low and so has to be pretty big to fall foul of signedness issues. > > > Busy right now but will check tomorrow. > > > Best regards > -Tobias > > > > > > On Wed, Aug 12, 2015 at 5:11 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Tobias, > > > > On Wed, Aug 12, 2015 at 12:49 PM, Tobias Pape <Das.Linux(a)gmx.de> wrote: > > > > On 12.08.2015, at 20:55, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > > Hi All, > > > > > > > > > Fabio's kindly done most of the changes. But some questions remain for more general discussion. See below. > > > > > > Fabio, thanks so much for doing this, and so quickly! > > > > > > On Tue, Aug 11, 2015 at 8:51 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > Hi All, > > > > > > who will update http://squeak.org/downloads/ to include the 5.0 release? > > > > > > a) I suggest that the 5.0 all-in-one have a line in the left-hand table that includes the 4.6 release and that it precede the 4.6 release in the list. > > > > > > Done. > > > > > > > > > b) the Trunk link points to TrunkImage.zip which contains a non-Spur image. > > > > > > So this is the biggie. What should we do? > > > > > > - add a Trunk 5.0 build with a different link? (Noooooo) > > > > > > - change http://build.squeak.org/job/SqueakTrunk to build what it says (Yessss please, who can do this?) > > > > Done: build.squeak.org/job/Trunk/ > > But as I said several times, Spur/trunk test just crash for monthâ¦ > > > > can you point me to the crash? I'm looking at http://build.squeak.org/job/Trunk/default/lastBuild/#showFailuresLink and see 8 test failures but no crash. > > > > > > > > > > - add a Squeak 4.6 build job? Is that even worth it any more considering 4.6 is released and stable? If it is then what should it build? David? > > > > > > > > > c) Spur VMs need to be linked to by an added line in the Virtual Machines list. > > > > > > Not applicable. the VM links point to the root of my site so they effectively point to both old and Spur VMs. > > > > > > > > > d) Spur imagers need to be included in the Image and Changes list under Custom Installation > > > > > > Done. > > > > > > > > > > > > e) The SqueakV50.sources file also needs to be included in the Current Sources list. > > > > > > Done. > > > > > > > > > f) we could start a History list for the 5.0 line > > > > > > This is probably quite a big reorg on the page and not urgent. > > > > > > > > > So there are a few things to fix before 5.0 is freely downloadable. > > > > > > On Tue, Aug 11, 2015 at 8:23 PM, Chris Muller <ma.chris.m(a)gmail.com> wrote: > > > In the 17 months since Squeak 4.5 was released, a huge development > > > effort took place to create the next generation virtual-machine for > > > the Squeak / Pharo / Newspeak family of programming systems. Squeak > > > is the modern incarnation of the Smalltalk-80 programming environment > > > originally developed at the Xerox PARC. > > > > > > "Squeak 5" introduces this new VM and associated new memory model, > > > collectively referred to as "Spur". Presented [1] by Eliot Miranda > > > and ClÃ©ment BÃ©ra at the 2015 International Symposium on Memory > > > Management, this new VM affords Squeak applications a significant > > > boost in performance and memory management. Among other > > > optimizations, the #become operation no longer requires a memory scan. > > > Object pinning and ephemerons are also now supported. The release > > > notes [2] provide more details. > > > > > > The new memory model requires a new image file format. Although this > > > new format results in about a 15% increased memory requirement for the > > > same number of 4.x objects, a new segmented heap allows memory to be > > > given back to the OS when its no longer needed, a great benefit for > > > application servers. > > > > > > As forward compatibility is as important to the Squeak community as > > > backward compatibility, Squeak 5 is delivers an image with identical > > > content as the recent 4.6 release. Although this new Squeak 5 VM > > > cannot open images saved under the prior 4.x Cog format, objects and > > > code can be easily exported from the 4.x image and then imported into > > > Squeak 5. Applications whose code runs strictly above the Smalltalk > > > meta layer will prove remarkably compatible with the new format, most > > > applications will require no changes whatsotever. > > > > > > Squeak 5 is the result of monumental effort by a tiny group of very > > > talented people, but its also just the beginning of yet a new effort; > > > Spur is just a stepping stone to a more ambitious goals planned over > > > the next five years. > > > > > > [1] -- A Partial Read Barrier for Efficient Support of Live > > > Object-oriented Programming > > > http://conf.researchr.org/event/ismm-2015/ismm-2015-papers-a-partial-read-b… > > > > > > [2] -- Squeak 5 Release Notes > > > http://wiki.squeak.org/squeak/6207

Aug. 20, 2015

Re: [Pharo-dev] Nautilus questions

by Dimitris Chloupis

Wow that pdf is very deep. Amazing abilities. Pity there is not much documentation and much exposure about these features. Thank you , I am studying it. On Thu, Aug 20, 2015 at 2:52 PM Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > 2015-08-20 11:51 GMT+02:00 Dimitris Chloupis <kilon.alios(a)gmail.com>: > >> I took a look at the class you recommended and now I see what you mean.I >> am glad this is not deeply tied to the system browser and is something >> other tools can use as well with ease. Very flexible too. Looks like the >> tools only expose a fraction of the power of the environment . >> > > Well, behind all things RB (the Refactoring Browser[1]) you have a > complete model of Smalltalk code. It allows refactoring tools to work on > live code but also on non-live code. For example, in SmaCC, which is > implemented by the same guys, the code for a parser is generated entirely > as refactorings (i.e. non-live code), then refactored for optimisations for > both code performance and to minimize the number of refactorings (but still > non-live), then split to fit the Cog Jit limits (my work!), and then > compiled to the live system in one (large) step. > > But why am I saying that? To show that those environments are very > flexible (and the tools around as well), that they may work on non-live > code (for example, a non-loaded package) and that most of the IDE needed > functions in practice apply on a model linked to the live code, including > when using them through Nautilus. > > [1] > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.91.9246&rep=rep1&t… > > >> Makes me wonder whether environment would be a good idea to be used as >> some sort of namespaces. >> > > The difference between an environment and a namespace is at the compiler > level: can you make the compiler use an environment when compiling a > method? I think that Marcus has done things in that direction recently > (such as allowing you to add "virtual" instance variables before compiling > a method). Also an environment never "renames" stuff, it only "restricts" > what is visible. > > At the programmer level, a Namespace is the ability to have two pieces of > code where the class named Morph is an entirely different class, and that > you can't understand the code anymore because you're never sure of what a > global refers to (since it may be renamed via an import statement)... Some > will say it's a feature ;) > > Thierry > > >> >> On Thu, Aug 20, 2015 at 9:08 AM Thierry Goubier < >> thierry.goubier(a)gmail.com> wrote: >> >>> Le 19/08/2015 20:26, Dimitris Chloupis a Ã©crit : >>> > >>> > >>> > On Wed, Aug 19, 2015 at 5:51 PM Sean P. DeNigris < >>> sean(a)clipperadams.com >>> > <mailto:sean@clipperadams.com>> wrote: >>> > >>> > kilon.alios wrote >>> > >>> > >>> > For example, let's say your project has a class that implements a >>> > generically-named method, like #asMorph. If you try to rename that >>> > method >>> > via Refactoring, Pharo will try to rename all #asMorph methods, and >>> > update >>> > all senders, in the entire system, not just yours. But if you >>> scope the >>> > browser first to your package or class, you can restrict the >>> > environment to >>> > which the refactoring is applied >>> > >>> > >>> > This sounds useful indeed. Is just the browser aware of the scope or is >>> > also the pharo environment aware of the scope too ? It would be nice if >>> > the scope could expand to include multiple packages , or is this >>> > something left to groups ? >>> >>> What is happening is that many commands linked to the IDE (refactorings, >>> searches) have an environment entity. By default, this environment is >>> Smalltalk, but it can be a subset of it, and any subset of it. >>> >>> Have a look into the RBBrowserEnvironment class and subclasses and you >>> will see all the variants. For example, a RBPackageEnvironment will >>> scope to one or multiple packages. >>> >>> Thierry >>> >>> >>> >>>

Aug. 20, 2015

Re: [Pharo-dev] Confused about libFT2Plugin

by Esteban Lorenzano

older images didnât have FT2 fonts active by default :) > On 20 Aug 2015, at 16:12, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > OK, thanks for replying. > > It is still a bit weird, since I can't remember experiencing it before. > > I might try to explore a bit more when I have more time. > >> On 20 Aug 2015, at 15:11, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >> >> ah, thatâs because headless mode is a fake headless (we will arrive to a real headless soon, but not yet). >> what actually happens is that VM is creating the window but not showing itâ¦ thatâs why it needs it :S >> >> Esteban >> >>> On 20 Aug 2015, at 15:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> >>>> On 20 Aug 2015, at 15:05, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >>>> >>>> most probably you are missing: >>>> >>>> sudo apt-get -qq install libfreetype6:i386 >>> >>> Yes, of course, but isn't that an X11 library that will pull in all of X11 and friends on a server that does not need a GUI ? I run all my servers pure command line, out of principle. >>> >>> Why would a headless server even have to access a font ? >>> >>>> cheers, >>>> Esteban >>>> >>>>> On 20 Aug 2015, at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Anyone ? >>>>> >>>>>> On 12 Aug 2015, at 17:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>> >>>>>> Today I did a new pharo install (get.pharo.org/40+vm) on a server (ubuntu 12.04.3 LTS, 64-bit with 32-bit libs). All went fine, as usual, but I was confused about the following: >>>>>> >>>>>> $ ./st-exec.st run-t3-proxies start t3-proxies >>>>>> Executing ./st-exec.st run-t3-proxies start t3-proxies >>>>>> Working directory /home/t3/pharo/t3-proxies >>>>>> Starting run-t3-proxies in background >>>>>> /home/t3/pharo/t3-proxies/../bin/pharo-vm/pharo --nodisplay /home/t3/pharo/t3-proxies/t3-proxies.image /home/t3/pharo/t3-proxies/run-t3-proxies.st >>>>>> >>>>>> ioLoadModule(/home/t3/pharo/bin/pharo-vm/libFT2Plugin.so): >>>>>> libfreetype.so.6: cannot open shared object file: No such file or directory >>>>>> >>>>>> Why is freetype needed/accessed for a headless image (that otherwise runs happily) ? >>>>>> >>>>>> Other operations (building, saving, ..) do not produce the error/warning. >>>>>> >>>>>> Sven >>>>>> >>>>> >>>>> >>>> >>> >>> >> >> > >

Aug. 20, 2015

Re: [Pharo-dev] Confused about libFT2Plugin

by Sven Van Caekenberghe

OK, thanks for replying. It is still a bit weird, since I can't remember experiencing it before. I might try to explore a bit more when I have more time. > On 20 Aug 2015, at 15:11, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > ah, thatâs because headless mode is a fake headless (we will arrive to a real headless soon, but not yet). > what actually happens is that VM is creating the window but not showing itâ¦ thatâs why it needs it :S > > Esteban > >> On 20 Aug 2015, at 15:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> >>> On 20 Aug 2015, at 15:05, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >>> >>> most probably you are missing: >>> >>> sudo apt-get -qq install libfreetype6:i386 >> >> Yes, of course, but isn't that an X11 library that will pull in all of X11 and friends on a server that does not need a GUI ? I run all my servers pure command line, out of principle. >> >> Why would a headless server even have to access a font ? >> >>> cheers, >>> Esteban >>> >>>> On 20 Aug 2015, at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Anyone ? >>>> >>>>> On 12 Aug 2015, at 17:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Today I did a new pharo install (get.pharo.org/40+vm) on a server (ubuntu 12.04.3 LTS, 64-bit with 32-bit libs). All went fine, as usual, but I was confused about the following: >>>>> >>>>> $ ./st-exec.st run-t3-proxies start t3-proxies >>>>> Executing ./st-exec.st run-t3-proxies start t3-proxies >>>>> Working directory /home/t3/pharo/t3-proxies >>>>> Starting run-t3-proxies in background >>>>> /home/t3/pharo/t3-proxies/../bin/pharo-vm/pharo --nodisplay /home/t3/pharo/t3-proxies/t3-proxies.image /home/t3/pharo/t3-proxies/run-t3-proxies.st >>>>> >>>>> ioLoadModule(/home/t3/pharo/bin/pharo-vm/libFT2Plugin.so): >>>>> libfreetype.so.6: cannot open shared object file: No such file or directory >>>>> >>>>> Why is freetype needed/accessed for a headless image (that otherwise runs happily) ? >>>>> >>>>> Other operations (building, saving, ..) do not produce the error/warning. >>>>> >>>>> Sven >>>>> >>>> >>>> >>> >> >> > >

Aug. 20, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50255 Home: https://github.com/pharo-project/pharo-core

Aug. 20, 2015

[pharo-project/pharo-core] 070b88: 50255

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 070b88ce5d58a684ee125ca67f1ac599987a7bcf https://github.com/pharo-project/pharo-core/commit/070b88ce5d58a684ee125ca6… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-08-20 (Thu, 20 Aug 2015) Changed paths: M Kernel.package/BlockClosure.class/instance/accessing/argumentNames.st A Kernel.package/BlockClosure.class/instance/accessing/tempNames.st A KernelTests.package/ClassTest.class/instance/testing/testAddSlotAnonymous.st M Nautilus.package/AbstractNautilusUI.class/instance/build ui/buildButtonsBar.st M Nautilus.package/AbstractNautilusUI.class/instance/menu builder/sourceCodeMenu_shifted_.st A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/README.md A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/class/accessing/uniqueIdentifierName.st A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/definition.st A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/instance/accessing/group.st A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/instance/accessing/name.st A Refactoring-Critics.package/RBSmalltalkGlobalsRule.class/instance/initialization/initialize.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50254.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50255.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50254.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50255.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A Slot-Tests.package/TemporaryVariableTest.class/README.md A Slot-Tests.package/TemporaryVariableTest.class/definition.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testHasTemporaryVariablesBlock.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testHasTemporaryVariablesMethod.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testReadTemporaryVariablesMethod.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testTemporaryVariablesBlock.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testTemporaryVariablesMethod.st A Slot-Tests.package/TemporaryVariableTest.class/instance/tests/testWriteTemporaryVariablesMethod.st M Slot.package/SlotClassBuilder.class/instance/private validating/validateClassName.st A Slot.package/TemporaryVariable.class/README.md A Slot.package/TemporaryVariable.class/definition.st A Slot.package/TemporaryVariable.class/instance/accessing/name.st A Slot.package/TemporaryVariable.class/instance/accessing/name_.st A Slot.package/TemporaryVariable.class/instance/reflecive api/readInContext_.st A Slot.package/TemporaryVariable.class/instance/reflecive api/write_InContext_.st A Slot.package/extension/BlockClosure/instance/hasTemporaryVariableNamed_.st A Slot.package/extension/BlockClosure/instance/temporaryVariableNamed_.st A Slot.package/extension/BlockClosure/instance/temporaryVariables.st A Slot.package/extension/CompiledMethod/instance/hasTemporaryVariableNamed_.st A Slot.package/extension/CompiledMethod/instance/temporaryVariableNamed_.st A Slot.package/extension/CompiledMethod/instance/temporaryVariables.st A Slot.package/extension/Context/instance/hasTemporaryVariableNamed_.st A Slot.package/extension/Context/instance/temporaryVariableNamed_.st A Slot.package/extension/Context/instance/temporaryVariables.st A Slot.package/extension/SmalltalkImage/instance/anonymousClassInstaller.st M Spec-Core.package/TextModel.class/instance/private/getMenu.st R Spec-Debugger.package/SpecDebugger.class/instance/actions code/codeContextMenu_shifted_.st M Spec-Debugger.package/SpecDebugger.class/instance/initialization/initializeWidgets.st M Tool-Finder.package/FinderUI.class/instance/items creation/buildSourceTextArea.st R Tool-Finder.package/FinderUI.class/instance/private/msgPaneMenu_shifted_.st R Tool-Workspace.package/Workspace.class/instance/menu/shiftedYellowButtonMenu.st R Tool-Workspace.package/Workspace.class/instance/menu/yellowButtonMenu.st M Traits.package/TClass.class/instance/subclass creation/classInstaller.st Log Message: ----------- 50255 16275 You cant use ctrl + shift + f anymore on the debuggeur https://pharo.fogbugz.com/f/cases/16275 16322 Move navigation arrows close to navigation drop list in Nautilus https://pharo.fogbugz.com/f/cases/16322 16323 add the concepts of a first class TemporaryVariable https://pharo.fogbugz.com/f/cases/16323 16321 started a RBSmalltalkGlobalsRule https://pharo.fogbugz.com/f/cases/16321 16307 Fix adding Slots to anonymous classes https://pharo.fogbugz.com/f/cases/16307 http://files.pharo.org/image/50/50255.zip

Aug. 20, 2015

Re: [Pharo-dev] Confused about libFT2Plugin

by Esteban Lorenzano

ah, thatâs because headless mode is a fake headless (we will arrive to a real headless soon, but not yet). what actually happens is that VM is creating the window but not showing itâ¦ thatâs why it needs it :S Esteban > On 20 Aug 2015, at 15:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > >> On 20 Aug 2015, at 15:05, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >> >> most probably you are missing: >> >> sudo apt-get -qq install libfreetype6:i386 > > Yes, of course, but isn't that an X11 library that will pull in all of X11 and friends on a server that does not need a GUI ? I run all my servers pure command line, out of principle. > > Why would a headless server even have to access a font ? > >> cheers, >> Esteban >> >>> On 20 Aug 2015, at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Anyone ? >>> >>>> On 12 Aug 2015, at 17:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Today I did a new pharo install (get.pharo.org/40+vm) on a server (ubuntu 12.04.3 LTS, 64-bit with 32-bit libs). All went fine, as usual, but I was confused about the following: >>>> >>>> $ ./st-exec.st run-t3-proxies start t3-proxies >>>> Executing ./st-exec.st run-t3-proxies start t3-proxies >>>> Working directory /home/t3/pharo/t3-proxies >>>> Starting run-t3-proxies in background >>>> /home/t3/pharo/t3-proxies/../bin/pharo-vm/pharo --nodisplay /home/t3/pharo/t3-proxies/t3-proxies.image /home/t3/pharo/t3-proxies/run-t3-proxies.st >>>> >>>> ioLoadModule(/home/t3/pharo/bin/pharo-vm/libFT2Plugin.so): >>>> libfreetype.so.6: cannot open shared object file: No such file or directory >>>> >>>> Why is freetype needed/accessed for a headless image (that otherwise runs happily) ? >>>> >>>> Other operations (building, saving, ..) do not produce the error/warning. >>>> >>>> Sven >>>> >>> >>> >> > >

Aug. 20, 2015

Re: [Pharo-dev] Confused about libFT2Plugin

by Sven Van Caekenberghe

> On 20 Aug 2015, at 15:05, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > most probably you are missing: > > sudo apt-get -qq install libfreetype6:i386 Yes, of course, but isn't that an X11 library that will pull in all of X11 and friends on a server that does not need a GUI ? I run all my servers pure command line, out of principle. Why would a headless server even have to access a font ? > cheers, > Esteban > >> On 20 Aug 2015, at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Anyone ? >> >>> On 12 Aug 2015, at 17:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Today I did a new pharo install (get.pharo.org/40+vm) on a server (ubuntu 12.04.3 LTS, 64-bit with 32-bit libs). All went fine, as usual, but I was confused about the following: >>> >>> $ ./st-exec.st run-t3-proxies start t3-proxies >>> Executing ./st-exec.st run-t3-proxies start t3-proxies >>> Working directory /home/t3/pharo/t3-proxies >>> Starting run-t3-proxies in background >>> /home/t3/pharo/t3-proxies/../bin/pharo-vm/pharo --nodisplay /home/t3/pharo/t3-proxies/t3-proxies.image /home/t3/pharo/t3-proxies/run-t3-proxies.st >>> >>> ioLoadModule(/home/t3/pharo/bin/pharo-vm/libFT2Plugin.so): >>> libfreetype.so.6: cannot open shared object file: No such file or directory >>> >>> Why is freetype needed/accessed for a headless image (that otherwise runs happily) ? >>> >>> Other operations (building, saving, ..) do not produce the error/warning. >>> >>> Sven >>> >> >> >

Aug. 20, 2015

Re: [Pharo-dev] Confused about libFT2Plugin

by Esteban Lorenzano

most probably you are missing: sudo apt-get -qq install libfreetype6:i386 cheers, Esteban > On 20 Aug 2015, at 15:00, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Anyone ? > >> On 12 Aug 2015, at 17:36, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Today I did a new pharo install (get.pharo.org/40+vm) on a server (ubuntu 12.04.3 LTS, 64-bit with 32-bit libs). All went fine, as usual, but I was confused about the following: >> >> $ ./st-exec.st run-t3-proxies start t3-proxies >> Executing ./st-exec.st run-t3-proxies start t3-proxies >> Working directory /home/t3/pharo/t3-proxies >> Starting run-t3-proxies in background >> /home/t3/pharo/t3-proxies/../bin/pharo-vm/pharo --nodisplay /home/t3/pharo/t3-proxies/t3-proxies.image /home/t3/pharo/t3-proxies/run-t3-proxies.st >> >> ioLoadModule(/home/t3/pharo/bin/pharo-vm/libFT2Plugin.so): >> libfreetype.so.6: cannot open shared object file: No such file or directory >> >> Why is freetype needed/accessed for a headless image (that otherwise runs happily) ? >> >> Other operations (building, saving, ..) do not produce the error/warning. >> >> Sven >> > >

Aug. 20, 2015