December 2017 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Alternative bytecode sets demo request

by Ben Coman

On 4 December 2017 at 10:48, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Ben, > > On Sun, Dec 3, 2017 at 6:08 PM, Ben Coman <btc(a)openinworld.com> wrote: >> >> I'm shooting in the dark here since its a bit hard to grasp... >> a. The implications of support alternative bytecode sets >> * Do the alternative bytecode sets run in parallel to normal Pharo >> bytecodes? > > > Yes. There's a single bit in the header of a compiled method that selects > between two bytecode sets. In current Squeak and Pharoi VMs, if the bit is > unset then the normal bytecode set is used, and if the nit is set, the Sista > bytecode set is used. That clarifies things a lot. So I guess the main advantage of this bit is making it easier to: a. incrementally develop and test new bytecodes? b. transition the Image between bytecode sets? >> * Can the bytecode sets be loaded adhoc/dynamically into an Image? >> or only compiled into the VM? > > > Only compiled into the VM. > >> >> b. How to implement/work with alternative bytecodes > > > In Squeak there are class-side accessors in CompiledCode. See CompiledCode > class>>installSecondaryBytecodeSet:. Note that the support isn't quite > finished yet. I hope to have the Sista vytecde set ready for the next > release of Squeak. ClÃ©ment is providing it in Pharo. > > In Squeak the BytecodeEncoder hierarchy implements the necessary support for > multiple bytecode sets. Via CompiledCode > class>>installSecondaryBytecodeSet: the system is informed as to which > bytecode set to use. This must match the bytecode set(s) that are in the > VM. Classes beneath BytecodeEncoder, in particular EncoderForV3PlusClosures > and EncoderForSistaV1 (in package BytecodeSets at > http://source.squeak.org/VMMaker) encode the current and the Sista sets > respectively. Can more than two bytecode sets be compiled into the one VM? i.e. Is a bytecode-set hard coded into position, or is it more like a variable slot that can select between several bytecode sets compiled into the VM. For example, if someone wanted to experiment with implementing Ethereum bytecodes, would they first need to rip out either EncoderForV3PlusClosures or EncoderForSistaV1? Now I made the mistake once of thinking it was Sista introducing this bytecode-bit, (btw, how do you normally refer to it?) but IIUC Sista is just making use of the feature. So I'm curious historically when the bytecode-bit was introduced? cheers -ben > >> To help (b.), would it be feasible to blog a mini-demo showing how to >> install/use alternative bytecodes? maybe for something like this simple RPN >> calculator substitute for "arith.c"... >> * https://github.com/philipaconrad/mini-vm/tree/master/examples/arith >> * >> https://github.com/philipaconrad/mini-vm/blob/master/examples/arith/arith.c > > > That's not how this will work. Essentially it is transparent. Once the > bytecode set support is installed in the compiler one simply compiles > Smalltalk (or anything else that produces parse trees that can be output to > compiled methods via the BytecodeEncoder hierarchy) and the method will be > output in the currently selected bytecode set. So it's not something one > plays around with, unless that is, one is developing the bytecode set in the > Vm simulator.

Dec. 4, 2017

[Pharo 7.0-dev] Build #356: 20793-During-Bootstrap-if-a-baseline-installation-fails-it-should-cancel-the-process

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #356 was: FAILURE. The Pull Request #567 was integrated: "20793-During-Bootstrap-if-a-baseline-installation-fails-it-should-cancel-the-process" Pull request url: https://github.com/pharo-project/pharo/pull/567 Issue Url: https://pharo.fogbugz.com/f/cases/20793 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Dec. 4, 2017

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

by Mariano Martinez Peck

Hi Luke, Not only for the details of a particular proxy implementation like Ghost, but also because of all the discussions around it, I think this journal paper [1] we wrote may be of help to you. Cheers, [1] https://hal.inria.fr/hal-01081236 On Mon, Dec 4, 2017 at 5:51 AM, Luke Gorrie <luke(a)snabb.co> wrote: > Thanks Ben & Henrik for the ideas! > > I will scratch my head a bit and then try to do something sensible. :-) > > > -- Mariano http://marianopeck.wordpress.com

Dec. 4, 2017

Re: [Pharo-dev] Pharo Launcher endless loop opening debuggers in Pharo 6.1

by Christophe Demarey

Hi, I cannot reproduce this bug. Do you have the log or the stack trace? > Le 2 dÃ©c. 2017 Ã 14:51, Bernhard Pieber <bernhard(a)pieber.com> a Ã©crit : > > I ran into an annoying problem using the latest Pharo 6.1 (stable) from Pharo Launcher. I am on macOS Sierra Version 10.12.6. > > Steps to reproduce: > 1. Open Pharo Launcher > 2. Create new Pharo 6.1 (stable) image from the official distributions, named Test > 3. Launch the Test image > 4. Open a File Browser > 5. Right click on the selected Test directory in the left file tree pane. > 6. Debuggers start opening on KeyNotFound: key #arg4 not found in Dictionary > 7. Image hangs > > Interestingly the problem does not happen if I create the image using âcurl get.pharo.org | bashâ and start it with â./pharo-ui Pharo.imageâ although About Pharo shows the same information: > Pharo 6.0 > Latest update: #60523 > > Can someone please confirm the bug? > > Bernhard

Dec. 4, 2017

Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap)

by Benoit St-Jean

Perhaps the solution resides in settings you can tweak from within Smalltalk?Â Something like MemoryPolicy you have in VisualWorks for memory & garbage collection management. I don't think fixing the behavior of memory management & garbage collection (marking, sweeping, etc) in the VM is the way to go.Â There's an awful lot of different needs for different applications out there ! Â ----------------- BenoÃ®t St-Jean Yahoo! Messenger: bstjean Twitter: @BenLeChialeux Pinterest: benoitstjean Instagram: Chef_Benito IRC: lamneth Blogue: endormitoire.wordpress.com "A standpoint is an intellectual horizon of radius zero".Â (A. Einstein) From: ClÃ©ment Bera <bera.clement(a)gmail.com> To: Squeak Virtual Machine Development Discussion <vm-dev(a)lists.squeakfoundation.org> Cc: Pharo Development List <pharo-dev(a)lists.pharo.org> Sent: Monday, December 4, 2017 2:49 AM Subject: Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap) hum... The mail is very long so I did not read all of it. Here are some ideas/things to say on the tip of my head:- Changing an object to a forwarding object is non atomic (we need to maintain at least stack invariant)- To decrease the pauses in full GC I have 2 plans:-- incremental marking (split the mark pause in multiple small pauses): Not implemented right now.-- selective compaction (compacts only part of the heap instead of all the heap and sweeps the rest, similar to G1, but uses forwarders instead of lots of card marking metadata): I implemented SpurSweeper which only sweeps but works very well.- Currently the marking phase removes all forwarders and I would like incremental marking to maintain the same invariant (forwarders are always white). - In general, Concurrent marking and sweeping have been implemented everywhere, but no concurrent compaction. For compaction you can make it selective (compact only part of the heap and the part which needs it the most) like I suggest and like in G1, which decreases considerably compaction pause time. Work on concurrent compaction is state of the art and not in production everywhere, see for exampleÂ Shenandoah Garbage Collector A. Shipilev Pause-Less GC for Improving Java ResponsivenessÂ C. Gracie And I will watch at talk on this topic tomorrow for the Android GC.- Some runtime, especially now with small servers being rent, are running on single core machines. So we need the low-pause GC to work incrementally aside from concurrently. So step 1 incremental GC. Step 2 concurrent marking and sweeping with low-pause for scavenge/compaction. No more time right now.Â On Sun, Dec 3, 2017 at 6:33 AM, Ben Coman <btc(a)openinworld.com> wrote: Hi Eliot, Clement, On 7 July 2017 at 00:41, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > - Better support for large heaps (GC tuning API, incremental GC). > > Pharo 64 bit is now able to manage large heap. However better > > performance can be proposed by offering better settings for the > > different GC zone. > > The most important thing here is the incremental GC.Â Spur has a generation > scavenger that collects garbage in newly created objects (new space), > and a mark-compact collector that collects and compacts garbage in old space. > > Right now on my 2.3GHz MacMini doing normal development, the generation > scavenger causes pauses of 1ms or less, and the mark-compact collector > than causes pauses of around 200ms.Â Both account for about 0.75% of > entire execution time (1.5% total), so the scavenger is invoked frequently > and the pauses that it creates are not noticeable.Â But the occasional > pauses created by the mark-compact collector /are/ noticeable, > especially in games and music. > > The idea for the incremental collector is to implement a mark-sweep or > mark-sweep-compact collector for old space that works incrementally, > doing a little bit of work on each invocation, probably immediately after a scavenge. > It is intended to avoid the long pauses caused by the non-incremental > mark-compact collector and make the system more suitable for games, music, etc. Reading http://www.mirandabanda.org/ cogblog/2013/09/13/lazy- become-and-a-partial-read- barrier/ Â "An alternative implementation, oft-used in Lisp systems, is to add a Â read barrier to all object access, and mark objects as forwarders. Â This can be used to implement a >>>lazy copying garbage collection<<<< Â where objects are copied from one semi-space to another in parallel to the Â main program (the âmutatorâ).Â To become, or move an object one replaces the Â objectâs header or first field with a forwarding pointer to the desired target Â or copy in a new location, marking the âcorpseâ as forwarded.Â The program Â checks the forwarded flag on each access.Â If there is hardware support, Â as in a Lisp machine, this can work well.Â But without hardware support, Â and like the object table representation, it has costs, slowing down Â program execution due to the scattering of forwarding checks and Â forwarding pointer follows throughout program execution." I'm curious... Given we now have forwarders with Spur, are we already sufficiently paying the cost of forwarding checks that a lazy copying garbage collector might be a feasible form of incremental garbage collection? I presume "parallel to the main program" means garbage collection occuring in a separate thread to the main vm thread, potentially resulting in very low main program pause times for garbage collection. I found this a useful summary of the terminology... * https://www.dynatrace.com/ resources/ebooks/javabook/ reduce-garbage-collection- pause-time/ and I'm curious how our planned Incremental CG fits those categories. That article got me contemplating our performance constraint of the VM only operating in only in a single native thread. Even though GC is a only a few percent of performance, I wondered what a concurrent GC might look like for us. I found this video describing concurrent GC in Go (ignore first 11:20 and the second half was not so interesting) * https://pusher.com/sessions/ meetup/the-realtime-guild/ golangs-realtime-garbage- collector where they present some interesting charts of their concern with latency pauses. (btw they reference a multi-language GC latency benchmark Â https://github.com/WillSewell/ gc-latency-experiment) And for balance of that I found... https://blog.plan99.net/ modern-garbage-collection- 911ef4f8bd8e And then my mind wandered around implementation details of concurrent garbage collection. To organise and quiet my thoughts I needed to put pen to paper, so I thought sharing that might stir thoughts for others.Â Probably naive and please excuse the brain dump format... Considering two threads... * Main program thread "MP" * Garbage collection thread "GC" with object-space shared between them, consisting of objects split in object-header & object-body... 1. Only "MP" mutates the object-body, updating slots creating new edges Â Â in the object-graph, and relocating objects in memory using forwarders. Â Â This rule avoids potential race conditions without needing to add Â Â synchronisation code affecting performance of "MP". 2. Concurrently "GC" performs a Marking Phase by following the object-graph Â Â tricolour tagging objects gray & black. it needs to mutate the object-header, Â Â in a synchronised way something like... Â Â Â a. Load object-header from shared object-space into local variable Â Â Â Â Â H <== object-header Â Â Â b. Modify header into another local variable Â Â Â Â Â H' <== H + updated GC color bits Â Â Â c. Atomic compare-and-swap H' back into object-space Â Â Â Â Â object-header <== if H then H' Â Â Â "MP" gets priority. Conflicts presumed rare. Then considering object mutation by "MP" concurrent/overlapped with "GC" marking... 3. TLDR; see 4. Â Â In old-space if 'mutated' object is linked to a 'target' object Â Â Â a. if 'mutated' isBlack, "MP" marks 'target' gray (to be later processed by "GC"). Â Â Â b. if 'mutated' isGray, overlapped 'mutated' gray->black by "GC" while 'target' Â Â Â Â remains white would break tricolor invariant (is it a credible case?), options... Â Â Â Â Â i. mark 'target' gray, the state anyway in case 'mutated' gray -> black Â Â Â Â Â ii. re-mark 'mutated' gray, i.e. normally gray->gray, and rarely black->gray Â Â Â Â Â iii. optimisticly just check after mutation if color changed (cached reads Â Â Â Â Â Â Â Â cheaper than writes and conflicts presumed rare) then set color to gray. Â Â Â c. if 'mutated' isWhite, options... Â Â Â Â Â i. do nothing and let "GC" reach 'target' normally, as long as overlapped white->black cannot occur Â Â Â Â Â ii. if white->black possible, do same as (3.b) "MP" marks 'target' gray, to be later processed by "GC". 4. Overall simplification of (3.) might be... Â Â Â "MP" checks for any color change during mutation, and only then marks 'mutated' object gray. Â Â Â How expensive would such a check be?Â Presumed marking is infrequent and Â Â Â can be done safely like (2.) 5. Eden/survivor space is ignored by "GC" thread. No point in adding work to the gray set Â Â until survivors filtered. Stop the world scavenging done as normal by "MP", Â Â only marking objects gray when they are moved to old-space. Â Â Post scavenging options... Â Â Â Â a. Resume the world and leave it for "GC" to process these recently grayed objects. Â Â Â Â b. Keep world stopped for "MP" to complete marking, emptying gray set to Â Â Â Â Â Â transition to Sweep Phase, at which point "MP" resumes the world. Â Â Â Â Â Â It doesn't matter if subsequently objects are added to the gray set, Â Â Â Â Â Â since existing white objects can never again be referenced by "MP". 6. Sweeping can be safely done by "GC" since white-objects are unreachable from "MP". Â Â "GC" can also take time to determine an optimum page P to compact and Â Â then (per 1.) passes to "MP" via a relocation-queue the objects to be Â Â relocated using forwarding pointers. "GC" could even spend extra time Â Â to determine the optimum relocation-destination without impacting the Â Â performance of "MP".Â When "MP" empties the relocation-queue, "GC" starts Â Â on the next Marking Phase. 7. Now a question remains about multi-threaded flattening of forwarding pointers. Â Â If two threads simultaneously perform an identical transform from... Â Â Â Â someObject-slot --> forwarder-b --> finalObject-c Â Â to... Â Â Â Â someObject-slot --> finalObject-c Â Â does it matter that these operation may be done twice overlapping? Â Â Options... Â Â Â a. One mitigation could be for "GC" to identify forwarders to be flattened Â Â Â Â and queue them for "MP" to process (reuse the compaction-queue).Â This Â Â Â Â is work that "MP" would need to do anyway, but brings it forward to be Â Â Â Â dealt with at a convenient time. Â Â Â b. I guess "MP" and "GC" could play nice together if when encountering Â Â Â Â a slot containing with a forwarding pointer they both do something similar to (2.) like... Â Â Â Â Â i. Load object-slot from shared object-space into local variable S <== slot Â Â Â Â Â ii. Set local variable S' <== flattened/followed pointer Â Â Â Â iii. Atomic compare-and-swap S' back into object-space, (object-slot <== if S then S') Â Â Â Â Â Â Â - "MP" gets priority but presume conflicts rare anyway. Â Â Â Â Â Â Â - While this violates rule (1.) the presumed frequency of Â Â Â Â Â Â Â Â encountering forwarding pointers is low for "MP", so performance Â Â Â Â Â Â Â Â should not be affected. Indeed by "GC" pre-emptively flattening forwarders Â Â Â Â Â Â Â Â the frequency "MP" sees reduces. 8. The release of the compacted page back to the OS is held up by forwarding pointers. Â Â Forwarders are part of the graph followed in the Marking Phase they get Â Â marked gray/black just like objects if they are referenced.Â After forwarders are Â Â fully flattened they are skipped by Marking and end up marked white and released Â Â just like any other object. Once all forwarders are released, the page is released Â Â to the OS.Â Â If "GC" can effectively flatten pointers concurrently with "MP" during its Â Â normal Marking Phase, then pages would be released back to the OS in a timely manner. Now one thing I am curious about how the GC tri-color marking is implemented. At https://clementbera.wordpress. com/2014/01/16/spurs-new- object-format it describes how Spur's object header has three bits for GC. * isGray * isRemembered * isMarkedÂ (I presume this means marked "black") Do the bits imply the gray set is not stored in a separate data structure on the heap, but rather distributed in-place, which I guess would require multiple passes through memory to grow the gray set? So this is not an area I'm set up to seriously work on, but I remain curious and hopefully its a useful seed discussion for others. cheers -ben P.S. At https://clementbera.wordpress. com/2017/09/19/vm-learning- memory-management/ nice to hear that you have Sophie (I presume) continuing with the VM. -- ClÃ©ment BÃ©raPharo consortium engineerhttps://clementbera.wordpress.com/ BÃ¢timent B 40, avenue Halley 59650Â Villeneuve d'Ascq

Dec. 4, 2017

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

by Luke Gorrie

Thanks Ben & Henrik for the ideas! I will scratch my head a bit and then try to do something sensible. :-)

Dec. 4, 2017

Re: [Pharo-dev] Design pattern for wrapper/proxy objects?

by Henrik Sperre Johansen

For proxies/wrappers, you might want to look into how the Ghost framework interrupts sends. (I don't know the exact details, but I'd guess step 1 is subclassing ProtoObject, rather than Object) For your specific case, isn't there already a *Struct class in the image you can subclass, create a class method containing the fields definition, and then have accessors autogenerated, rather than using doesNotUnderstand:? Usually, that would make the code easier to browse/debug later on, doesn't confuse the syntax highlighter/code critic in uses of an otherwise undefined field, etc. Cheers, Henry -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

Dec. 4, 2017

Re: [Pharo-dev] [Vm-dev] Garbage Collection (was Re: Discussing the roadmap)

by ClÃ©ment Bera

hum... The mail is very long so I did not read all of it. Here are some ideas/things to say on the tip of my head: - Changing an object to a forwarding object is non atomic (we need to maintain at least stack invariant) - To decrease the pauses in full GC I have 2 plans: -- incremental marking (split the mark pause in multiple small pauses): Not implemented right now. -- selective compaction (compacts only part of the heap instead of all the heap and sweeps the rest, similar to G1, but uses forwarders instead of lots of card marking metadata): I implemented SpurSweeper which only sweeps but works very well. - Currently the marking phase removes all forwarders and I would like incremental marking to maintain the same invariant (forwarders are always white). - In general, Concurrent marking and sweeping have been implemented everywhere, but no concurrent compaction. For compaction you can make it selective (compact only part of the heap and the part which needs it the most) like I suggest and like in G1, which decreases considerably compaction pause time. Work on concurrent compaction is state of the art and not in production everywhere, see for example Shenandoah Garbage Collector <https://shipilev.net/talks/vmm-Sep2017-shenandoah.pdf> *A. Shipilev* Pause-Less GC for Improving Java Responsiveness <https://speakerdeck.com/vmm/pause-less-gc-for-improving-java-responsiveness> *C. Gracie* And I will watch at talk on this topic tomorrow for the Android GC. - Some runtime, especially now with small servers being rent, are running on single core machines. So we need the low-pause GC to work incrementally aside from concurrently. So step 1 incremental GC. Step 2 concurrent marking and sweeping with low-pause for scavenge/compaction. No more time right now. On Sun, Dec 3, 2017 at 6:33 AM, Ben Coman <btc(a)openinworld.com> wrote: > > Hi Eliot, Clement, > > On 7 July 2017 at 00:41, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > > - Better support for large heaps (GC tuning API, incremental GC). > > > Pharo 64 bit is now able to manage large heap. However better > > > performance can be proposed by offering better settings for the > > > different GC zone. > > > > The most important thing here is the incremental GC. Spur has a > generation > > scavenger that collects garbage in newly created objects (new space), > > and a mark-compact collector that collects and compacts garbage in old > space. > > > > Right now on my 2.3GHz MacMini doing normal development, the generation > > scavenger causes pauses of 1ms or less, and the mark-compact collector > > than causes pauses of around 200ms. Both account for about 0.75% of > > entire execution time (1.5% total), so the scavenger is invoked > frequently > > and the pauses that it creates are not noticeable. But the occasional > > pauses created by the mark-compact collector /are/ noticeable, > > especially in games and music. > > > > The idea for the incremental collector is to implement a mark-sweep or > > mark-sweep-compact collector for old space that works incrementally, > > doing a little bit of work on each invocation, probably immediately > after a scavenge. > > It is intended to avoid the long pauses caused by the non-incremental > > mark-compact collector and make the system more suitable for games, > music, etc. > > Reading http://www.mirandabanda.org/cogblog/2013/09/13/lazy- > become-and-a-partial-read-barrier/ > "An alternative implementation, oft-used in Lisp systems, is to add a > read barrier to all object access, and mark objects as forwarders. > This can be used to implement a >>>lazy copying garbage collection<<<< > where objects are copied from one semi-space to another in parallel to > the > main program (the âmutatorâ). To become, or move an object one replaces > the > objectâs header or first field with a forwarding pointer to the desired > target > or copy in a new location, marking the âcorpseâ as forwarded. The > program > checks the forwarded flag on each access. If there is hardware support, > as in a Lisp machine, this can work well. But without hardware support, > and like the object table representation, it has costs, slowing down > program execution due to the scattering of forwarding checks and > forwarding pointer follows throughout program execution." > > I'm curious... Given we now have forwarders with Spur, are we > already sufficiently paying the cost of forwarding checks that a lazy > copying > garbage collector might be a feasible form of incremental garbage > collection? > > I presume "parallel to the main program" means garbage collection occuring > in a separate thread to the main vm thread, potentially resulting in > very low main program pause times for garbage collection. > > I found this a useful summary of the terminology... > * https://www.dynatrace.com/resources/ebooks/javabook/ > reduce-garbage-collection-pause-time/ > and I'm curious how our planned Incremental CG fits those categories. > > That article got me contemplating our performance constraint of the VM only > operating in only in a single native thread. Even though GC is a only > a few percent of performance, I wondered what a concurrent GC might > look like for us. > > I found this video describing concurrent GC in Go (ignore first 11:20 > and the second half was not so interesting) > * https://pusher.com/sessions/meetup/the-realtime-guild/ > golangs-realtime-garbage-collector > where they present some interesting charts of their concern with latency > pauses. > (btw they reference a multi-language GC latency benchmark > https://github.com/WillSewell/gc-latency-experiment) > > And for balance of that I found... > https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e > > > And then my mind wandered around implementation details of concurrent > garbage collection. > To organise and quiet my thoughts I needed to put pen to paper, so I > thought sharing that > might stir thoughts for others. Probably naive and please excuse the > brain dump format... > > > Considering two threads... > * Main program thread "MP" > * Garbage collection thread "GC" > with object-space shared between them, consisting of objects split in > object-header & object-body... > > 1. Only "MP" mutates the object-body, updating slots creating new edges > in the object-graph, and relocating objects in memory using forwarders. > This rule avoids potential race conditions without needing to add > synchronisation code affecting performance of "MP". > > > 2. Concurrently "GC" performs a Marking Phase by following the object-graph > tricolour tagging objects gray & black. it needs to mutate the > object-header, > in a synchronised way something like... > a. Load object-header from shared object-space into local variable > H <== object-header > b. Modify header into another local variable > H' <== H + updated GC color bits > c. Atomic compare-and-swap H' back into object-space > object-header <== if H then H' > "MP" gets priority. Conflicts presumed rare. > > > Then considering object mutation by "MP" concurrent/overlapped with > "GC" marking... > > 3. TLDR; see 4. > In old-space if 'mutated' object is linked to a 'target' object > a. if 'mutated' isBlack, "MP" marks 'target' gray (to be later > processed by "GC"). > > b. if 'mutated' isGray, overlapped 'mutated' gray->black by "GC" > while 'target' > remains white would break tricolor invariant (is it a credible > case?), options... > i. mark 'target' gray, the state anyway in case 'mutated' > gray -> black > ii. re-mark 'mutated' gray, i.e. normally gray->gray, and > rarely black->gray > iii. optimisticly just check after mutation if color changed > (cached reads > cheaper than writes and conflicts presumed rare) then > set color to gray. > > c. if 'mutated' isWhite, options... > i. do nothing and let "GC" reach 'target' normally, as long > as overlapped white->black cannot occur > ii. if white->black possible, do same as (3.b) "MP" marks > 'target' gray, to be later processed by "GC". > > > 4. Overall simplification of (3.) might be... > "MP" checks for any color change during mutation, and only then > marks 'mutated' object gray. > How expensive would such a check be? Presumed marking is infrequent > and > can be done safely like (2.) > > > 5. Eden/survivor space is ignored by "GC" thread. No point in adding > work to the gray set > until survivors filtered. Stop the world scavenging done as normal by > "MP", > only marking objects gray when they are moved to old-space. > > Post scavenging options... > a. Resume the world and leave it for "GC" to process these > recently grayed objects. > > b. Keep world stopped for "MP" to complete marking, emptying gray > set to > transition to Sweep Phase, at which point "MP" resumes the > world. > It doesn't matter if subsequently objects are added to the gray > set, > since existing white objects can never again be referenced by > "MP". > > > 6. Sweeping can be safely done by "GC" since white-objects are > unreachable from "MP". > "GC" can also take time to determine an optimum page P to compact and > then (per 1.) passes to "MP" via a relocation-queue the objects to be > relocated using forwarding pointers. "GC" could even spend extra time > to determine the optimum relocation-destination without impacting the > performance of "MP". When "MP" empties the relocation-queue, "GC" > starts > on the next Marking Phase. > > > 7. Now a question remains about multi-threaded flattening of > forwarding pointers. > If two threads simultaneously perform an identical transform from... > someObject-slot --> forwarder-b --> finalObject-c > to... > someObject-slot --> finalObject-c > does it matter that these operation may be done twice overlapping? > > Options... > > a. One mitigation could be for "GC" to identify forwarders to be > flattened > and queue them for "MP" to process (reuse the compaction-queue). > This > is work that "MP" would need to do anyway, but brings it forward > to be > dealt with at a convenient time. > > b. I guess "MP" and "GC" could play nice together if when encountering > a slot containing with a forwarding pointer they both do > something similar to (2.) like... > i. Load object-slot from shared object-space into local > variable S <== slot > ii. Set local variable S' <== flattened/followed pointer > iii. Atomic compare-and-swap S' back into object-space, > (object-slot <== if S then S') > - "MP" gets priority but presume conflicts rare anyway. > - While this violates rule (1.) the presumed frequency of > encountering forwarding pointers is low for "MP", so > performance > should not be affected. Indeed by "GC" pre-emptively > flattening forwarders > the frequency "MP" sees reduces. > > > 8. The release of the compacted page back to the OS is held up by > forwarding pointers. > Forwarders are part of the graph followed in the Marking Phase they get > marked gray/black just like objects if they are referenced. After > forwarders are > fully flattened they are skipped by Marking and end up marked white > and released > just like any other object. Once all forwarders are released, the > page is released > to the OS. If "GC" can effectively flatten pointers concurrently > with "MP" during its > normal Marking Phase, then pages would be released back to the OS > in a timely manner. > > > > Now one thing I am curious about how the GC tri-color marking is > implemented. > At https://clementbera.wordpress.com/2014/01/16/spurs-new-object-format > it describes how Spur's object header has three bits for GC. > * isGray > * isRemembered > * isMarked (I presume this means marked "black") > Do the bits imply the gray set is not stored in a separate data > structure on the heap, but rather distributed in-place, which I guess > would require multiple passes through memory to grow the gray set? > > > So this is not an area I'm set up to seriously work on, but I remain > curious > and hopefully its a useful seed discussion for others. > cheers -ben > > > P.S. At https://clementbera.wordpress.com/2017/09/19/vm-learning- > memory-management/ > nice to hear that you have Sophie (I presume) continuing with the VM. > -- ClÃ©ment BÃ©ra Pharo consortium engineer https://clementbera.wordpress.com/ BÃ¢timent B 40, avenue Halley 59650 Villeneuve d'Ascq

Dec. 4, 2017

Re: [Pharo-dev] Alternative bytecode sets demo request

by ClÃ©ment Bera

To use the alternate bytecode set in Pharo 7 use: CompilationContext bytecodeBackend: EncoderForSistaV1. OpalCompiler recompileAll. based on: https://clementbera.wordpress.com/2017/07/19/sista-open-alpha-release/ Currently in Pharo SqueakV3PlusClosure and SistaV1 are installed in CompiledCode. You need to install another set if you want to use it using #installSecondaryBytecodeSet: or similar methods. CompiledMethod installSecondaryBytecodeSet: EncoderForMyBytecodeSet. Then you can just use the first code snippet to use it: CompilationContext bytecodeBackend: EncoderForMyBytecodeSet. OpalCompiler recompileAll. On Mon, Dec 4, 2017 at 3:48 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Ben, > > On Sun, Dec 3, 2017 at 6:08 PM, Ben Coman <btc(a)openinworld.com> wrote: > >> I'm shooting in the dark here since its a bit hard to grasp... >> a. The implications of support alternative bytecode sets >> * Do the alternative bytecode sets run in parallel to normal Pharo >> bytecodes? >> > > Yes. There's a single bit in the header of a compiled method that selects > between two bytecode sets. In current Squeak and Pharoi VMs, if the bit is > unset then the normal bytecode set is used, and if the nit is set, the > Sista bytecode set is used. > > * Can the bytecode sets be loaded adhoc/dynamically into an Image? >> or only compiled into the VM? >> > > Only compiled into the VM. > > >> b. How to implement/work with alternative bytecodes >> > > In Squeak there are class-side accessors in CompiledCode. See > CompiledCode class>>installSecondaryBytecodeSet:. Note that the support > isn't quite finished yet. I hope to have the Sista vytecde set ready for > the next release of Squeak. ClÃ©ment is providing it in Pharo. > > In Squeak the BytecodeEncoder hierarchy implements the necessary support > for multiple bytecode sets. Via CompiledCode class>>installSecondaryBytecodeSet: > the system is informed as to which bytecode set to use. This must match > the bytecode set(s) that are in the VM. Classes beneath BytecodeEncoder, > in particular EncoderForV3PlusClosures and EncoderForSistaV1 (in package > BytecodeSets at http://source.squeak.org/VMMaker) encode the current and > the Sista sets respectively. > > For (a.) I guess an exciting opportunity is facilitating an Ethereum >> virtual machine >> running as a Process inside a Pharo image, managed by tools written in >> Pharo. >> Other languages are already getting in a party which would be good for >> Pharo to join. >> In such a revolutionary domain I expect people are more willing to >> experiment >> with alternative systems like Pharo, and I bet Pharo could excel in that >> domain. >> * https://github.com/ethereum/go-ethereum/blob/master/core/vm/opcodes.go >> * https://medium.com/@jeff.ethereum/optimising-the-ethereum- >> virtual-machine-58457e61ca15 >> * https://github.com/pirapira/awesome-ethereum-virtual-machi >> ne#evm-implementations >> >> >> To help (b.), would it be feasible to blog a mini-demo showing how to >> install/use alternative bytecodes? maybe for something like this simple RPN >> calculator substitute for "arith.c"... >> * https://github.com/philipaconrad/mini-vm/tree/master/examples/arith >> * https://github.com/philipaconrad/mini-vm/blob/master/ >> examples/arith/arith.c >> > > That's not how this will work. Essentially it is transparent. Once the > bytecode set support is installed in the compiler one simply compiles > Smalltalk (or anything else that produces parse trees that can be output to > compiled methods via the BytecodeEncoder hierarchy) and the method will be > output in the currently selected bytecode set. So it's not something one > plays around with, unless that is, one is developing the bytecode set in > the Vm sim,letter. > > >> >> (a calculator demo could be an interesting adjunct to Sven's calculator >> tutorial...) >> https://medium.com/concerning-pharo/rediscovering-the-ux-of- >> the-legendary-hp-35-scientific-pocket-calculator-d1d497ece999 >> >> >> cheers -ben >> > > > > -- > _,,,^..^,,,_ > best, Eliot > -- ClÃ©ment BÃ©ra Pharo consortium engineer https://clementbera.wordpress.com/ BÃ¢timent B 40, avenue Halley 59650 Villeneuve d'Ascq

Dec. 4, 2017

Esteban's ChangeLog week of 27 November 2017

by estebanlm＠gmail.com

Hello! This is my weekly ChangeLog, from 27 November 2017 to 3 December 2017. You can see it in a better format by going here: http://log.smallworks.eu/web/search?from=27/11/2017&to=3/12/2017 ChangeLog ========= 28 November 2017: ----------------- * I just released [Iceberg 0.6.4](https://github.com/pharo-vcs/iceberg/releases/tag/v0.6.4). This is a bugfix release :) cheers! Esteban

Dec. 4, 2017