January 2015 - Pharo-dev - lists.pharo.org

Finish one thing today

by Marcus Denker

This is nice: http://finishonethingtoday.com Marcus

Jan. 14, 2015

Re: [Pharo-dev] Implementing lazy loading for expandable collections (part 2)

by Alexandre Bergel

Hi Max! You did a nice experiment. Indeed, I have never considered having an empty array or a nil value. You find out that using an empty array leads to simpler code and better performance. Will you release your code? I can have a look at it. Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > On Jan 14, 2015, at 6:53 AM, Max Leske <maxleske(a)gmail.com> wrote: > > Hi Alex > > I thought the following little experiment might interest you. > > I was trying to find out (as per your suggestion) how big the difference would be between: > - setting the array in OrderedCollection to nil and initializing it to size 10 on first access > - initializing the array in OrderedCollection to size 0. > > Expectation: #grow will take time, ergo, using an empty array should be slower than using > a preallocated array. > > Experiment (Pharo 1.3 on SqueakVM): > > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 100000. > 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 17566 > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 10000. > 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 19717 > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 0. > 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 17598 > > > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 1000. > 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 18084 > > > > Result: growing a collection is always faster (or at least nearly equally fast)! > > Explanation: What makes preinitialized collections so slow is the fact the the firstIndex variable will be set to > the first third of the array (3333 in this case). When the last position in the array is reached, all elements > are copied to the beginning of the array (2n operations). This copy operation is really slow (which can be seen > from the first example with an oversized collection that does not require copying because the last position > of the array is never reached). > > > Experiment (Pharo 4 on PharoVM): > (Iâm using bigger numbers here to compensate for the faster VM :) ) > > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 1000000. > 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:01:09.225" > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 100000. > 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:37.653" > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 0. > 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:40.152" > > > > [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 10000. > 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:42.121â > > > > Result: growing a collection is easily fast enough, although minute performance gains are possible with preinitialized collections. > > Explanation: #makeRoomAtLast has been improved. Also: firstIndex is now set to the beginning of the array, preventing the copy > operation that made the former measurements so slow. > The run with the oversized collection is only so slow because it apparently takes a lot of time to assign the large array (?). > > > Consequence of my little experiment: My initial implementation of the idea in you paper set the array in OrderedCollection to nil > under the assumption that grow operations are very costly. That choice leads to many changes in methods of OrderedCollection > where it is necessary to check if the array has been initialized yet. This introduces overhead and makes the code more error prone. > I see now that it makes much more sense to use an empty array, the performance lost by growing ist neglectable. > > It is also interesting to note that there almost never seems to be a good reason to initialize an OrderedCollection with a large array. > > > Cheers, > Max

Jan. 14, 2015

Re: [Pharo-dev] Hosting the Pharo VM on MirageOS

by Marcus Denker

>> >> >> Mirarge might be a good stepping stone, but ultimately I think we want Cog running directly on the Xen "hardware" without Mirage. Erlang and Haskell are languages that already do this. [1] We need to get on that list at the bottom. >> >> Maybe a good starting place is SqueakNOS [2] since it is reported to work with Grub and there is PVGrub [3] [4]. > > or PharoNOS :) > > http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos <http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos> But this one actually has a small Linux as the baseâ¦ so it is not âNOSâ in this sense. Marcus

Jan. 14, 2015

Re: [Pharo-dev] Hosting the Pharo VM on MirageOS

by Esteban Lorenzano

> On 14 Jan 2015, at 08:15, Ben Coman <btc(a)openInWorld.com> wrote: > > > On Wed, Jan 14, 2015 at 11:39 AM, Joerg Beekmann, DeepCove Labs <joerg(a)deepcovelabs.com <mailto:joerg@deepcovelabs.com>> wrote: > Hi > > I've been exploring the possibility of hosting the Squeak/Pharo VM on MirageOS. MirageOS is a Unikernal or "Library OS" where rather than creating an executable to be run on Linux the compiler analyses dependencies right down through the device drivers and creates a kernel that can be booted on Amazon AWS EC2. > http://www.openmirage.org/ <http://www.openmirage.org/> > http://queue.acm.org/detail.cfm?id=2566628 <http://queue.acm.org/detail.cfm?id=2566628> > http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems-for… <http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems-for…> > > It seems to me that having a Smalltalk hosed in this environment would be quite useful on a number of fronts: > * A simple build and deploy of Smalltalk to the cloud. The Mirage group is building unikernals from sources and then because they are small committing the entire kernel to GitHub and then pushing the kernel to AWS EC2. Web sites and APIs are an obvious application. http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/ <http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/>. > * Smalltalk on small devices. The Mirage group is running Xen with mirage on small Intel and ARM boards. See http://openmirage.org/blog/announcing-mirage-20-release/ <http://openmirage.org/blog/announcing-mirage-20-release/>. > * These kernels can be very small, on the order of 0.25Meg for minimal HTTP servers with boot times in the ms range. A project called Jitsu https://github.com/MagnusS/jitsu <https://github.com/MagnusS/jitsu> modifies a DNS server to boot kernels in response to socket requests. The system making the request is unaware of the boot. > * The Mirage group envisions thousands of kernels running on a single Hypervisor with his speed inter-kernel communications. See http://openmirage.org/blog/update-on-vchan <http://openmirage.org/blog/update-on-vchan>. The facilitates Smalltalk systems consisting of a swarm of communicating images where each image is single threaded and concurrency is via message passing. > > I've corresponded with the MirageOS group asking if they thought a language VM could be hosted on Mirage. The discussion is here: http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00053.h… <http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00053.h…>. Assuming we understood each other it seems the answer is "yes that should be possible". > > My question is what would be involved on the Smalltalk side? I presume the effort is going to be mostly in the VM. Based on the conversation on the Mirage list I think the VM will need to be able to run a Library with an entry point called by Mirage. That seems similar to the " Embedding/VM as a DLL" project proposed here: http://www.mirandabanda.org/cogblog/cog-projects/ <http://www.mirandabanda.org/cogblog/cog-projects/>. Is this project active? The comment indicates this is mostly refactoring & repackaging. Anyone have a perspective on this? > > Best regards > > Joerg > > > > Mirarge might be a good stepping stone, but ultimately I think we want Cog running directly on the Xen "hardware" without Mirage. Erlang and Haskell are languages that already do this. [1] We need to get on that list at the bottom. > > Maybe a good starting place is SqueakNOS [2] since it is reported to work with Grub and there is PVGrub [3] [4]. or PharoNOS :) http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos <http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos> > > btw, Did you see the discussion titled "Cog In The Cloud" > http://forum.world.st/Cog-in-the-cloud-td4796515.html#a4796520 <http://forum.world.st/Cog-in-the-cloud-td4796515.html#a4796520> > and a little bit extra at the bottom of this one... > https://www.marshut.net/kuimwz/cog-in-the-cloud.html <https://www.marshut.net/kuimwz/cog-in-the-cloud.html> > > [1] http://wiki.xenproject.org/wiki/Cloud_Operating_Systems <http://wiki.xenproject.org/wiki/Cloud_Operating_Systems> > [2] http://wiki.squeak.org/squeak/1762 <http://wiki.squeak.org/squeak/1762> > [3] http://wiki.xen.org/wiki/PvGrub <http://wiki.xen.org/wiki/PvGrub> > [4] https://tech.mendix.com/linux/2014/09/06/pvgrub2/ <https://tech.mendix.com/linux/2014/09/06/pvgrub2/> > http://ptgmedia.pearsoncmg.com/images/9780132349710/samplechapter/013234971… <http://ptgmedia.pearsoncmg.com/images/9780132349710/samplechapter/013234971…>

Jan. 14, 2015

Re: [Pharo-dev] Fixing DNU binding in GT and Nautilus

by Marcus Denker

I have opened an issue for this: https://pharo.fogbugz.com/f/cases/14731/Fixing-DNU-binding-in-GT-and-Nautil… <https://pharo.fogbugz.com/f/cases/14731/Fixing-DNU-binding-in-GT-and-Nautil…> > On 13 Jan 2015, at 20:11, stepharo <stepharo(a)free.fr> wrote: > > Hi > > in nautilus method pane and GT, when I select a global variable such as > > SystemOrganization and press command B + N > > I get DNU binding. > Apparently this DNU only happen when the name does not match any class substring > Transcript or Undeclared will not raise an error because we have classes having name matching the expression. > > Fixed included. > > PS: I'm dead after 1h30 train delay (after giving 6 hours lecture) today. So this mail will be sent whenever I get connected. > > Stef > <FixDNUBinding.cs>

Jan. 14, 2015

Re: [Pharo-dev] Access to Pharo/TxText

by Marcus Denker

> On 14 Jan 2015, at 00:30, Bernardo Ezequiel Contreras <vonbecmann(a)gmail.com> wrote: > > Hi, > Could someone take a look at this > > https://pharo.fogbugz.com/f/cases/14632/Access-to-Pharo-TxText <https://pharo.fogbugz.com/f/cases/14632/Access-to-Pharo-TxText> > In general we need to organise: Who - will review fixes for TxText that are on the issue tracker? - commits them to the repo and give commit right to people? - push and coordinate to have the latest in Pharo4? Right now, all TxText contributions are ignored. Marcus

Jan. 14, 2015

Re: [Pharo-dev] Hosting the Pharo VM on MirageOS

by Ben Coman

On Wed, Jan 14, 2015 at 11:39 AM, Joerg Beekmann, DeepCove Labs < joerg(a)deepcovelabs.com> wrote: > Hi > > I've been exploring the possibility of hosting the Squeak/Pharo VM on > MirageOS. MirageOS is a Unikernal or "Library OS" where rather than > creating an executable to be run on Linux the compiler analyses > dependencies right down through the device drivers and creates a kernel > that can be booted on Amazon AWS EC2. > http://www.openmirage.org/ > http://queue.acm.org/detail.cfm?id=2566628 > > http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems-for… > > It seems to me that having a Smalltalk hosed in this environment would be > quite useful on a number of fronts: > * A simple build and deploy of Smalltalk to the cloud. The Mirage group is > building unikernals from sources and then because they are small committing > the entire kernel to GitHub and then pushing the kernel to AWS EC2. Web > sites and APIs are an obvious application. > http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/. > * Smalltalk on small devices. The Mirage group is running Xen with mirage > on small Intel and ARM boards. See > http://openmirage.org/blog/announcing-mirage-20-release/. > * These kernels can be very small, on the order of 0.25Meg for minimal > HTTP servers with boot times in the ms range. A project called Jitsu > https://github.com/MagnusS/jitsu modifies a DNS server to boot kernels in > response to socket requests. The system making the request is unaware of > the boot. > * The Mirage group envisions thousands of kernels running on a single > Hypervisor with his speed inter-kernel communications. See > http://openmirage.org/blog/update-on-vchan. The facilitates Smalltalk > systems consisting of a swarm of communicating images where each image is > single threaded and concurrency is via message passing. > > I've corresponded with the MirageOS group asking if they thought a > language VM could be hosted on Mirage. The discussion is here: > http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00053.h…. > Assuming we understood each other it seems the answer is "yes that should > be possible". > > My question is what would be involved on the Smalltalk side? I presume the > effort is going to be mostly in the VM. Based on the conversation on the > Mirage list I think the VM will need to be able to run a Library with an > entry point called by Mirage. That seems similar to the " Embedding/VM as a > DLL" project proposed here: > http://www.mirandabanda.org/cogblog/cog-projects/. Is this project > active? The comment indicates this is mostly refactoring & repackaging. > Anyone have a perspective on this? > > Best regards > > Joerg > > > Mirarge might be a good stepping stone, but ultimately I think we want Cog running directly on the Xen "hardware" without Mirage. Erlang and Haskell are languages that already do this. [1] We need to get on that list at the bottom. Maybe a good starting place is SqueakNOS [2] since it is reported to work with Grub and there is PVGrub [3] [4]. btw, Did you see the discussion titled "Cog In The Cloud" http://forum.world.st/Cog-in-the-cloud-td4796515.html#a4796520 and a little bit extra at the bottom of this one... https://www.marshut.net/kuimwz/cog-in-the-cloud.html [1] http://wiki.xenproject.org/wiki/Cloud_Operating_Systems [2] http://wiki.squeak.org/squeak/1762 [3] http://wiki.xen.org/wiki/PvGrub [4] https://tech.mendix.com/linux/2014/09/06/pvgrub2/ http://ptgmedia.pearsoncmg.com/images/9780132349710/samplechapter/013234971…

Jan. 14, 2015

Implementing lazy loading for expandable collections (part 2)

by Max Leske

Hi Alex I thought the following little experiment might interest you. I was trying to find out (as per your suggestion) how big the difference would be between: - setting the array in OrderedCollection to nil and initializing it to size 10 on first access - initializing the array in OrderedCollection to size 0. Expectation: #grow will take time, ergo, using an empty array should be slower than using a preallocated array. Experiment (Pharo 1.3 on SqueakVM): [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 100000. 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 17566 [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 10000. 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 19717 [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 0. 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 17598 [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 1000. 1 to: 10000 do: [ :i | c add: i ] ] ] timeToRun 18084 Result: growing a collection is always faster (or at least nearly equally fast)! Explanation: What makes preinitialized collections so slow is the fact the the firstIndex variable will be set to the first third of the array (3333 in this case). When the last position in the array is reached, all elements are copied to the beginning of the array (2n operations). This copy operation is really slow (which can be seen from the first example with an oversized collection that does not require copying because the last position of the array is never reached). Experiment (Pharo 4 on PharoVM): (Iâm using bigger numbers here to compensate for the faster VM :) ) [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 1000000. 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:01:09.225" [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 100000. 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:37.653" [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 0. 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:40.152" [ 10000 timesRepeat: [ | c | c := OrderedCollection new: 10000. 1 to: 100000 do: [ :i | c add: i ] ] ] timeToRun "0:00:00:42.121â Result: growing a collection is easily fast enough, although minute performance gains are possible with preinitialized collections. Explanation: #makeRoomAtLast has been improved. Also: firstIndex is now set to the beginning of the array, preventing the copy operation that made the former measurements so slow. The run with the oversized collection is only so slow because it apparently takes a lot of time to assign the large array (?). Consequence of my little experiment: My initial implementation of the idea in you paper set the array in OrderedCollection to nil under the assumption that grow operations are very costly. That choice leads to many changes in methods of OrderedCollection where it is necessary to check if the array has been initialized yet. This introduces overhead and makes the code more error prone. I see now that it makes much more sense to use an empty array, the performance lost by growing ist neglectable. It is also interesting to note that there almost never seems to be a good reason to initialize an OrderedCollection with a large array. Cheers, Max

Jan. 14, 2015

Re: [Pharo-dev] Hosting the Pharo VM on MirageOS

by Alain Rastoul

Le 14/01/2015 04:39, Joerg Beekmann, DeepCove Labs a Ã©crit : > Hi > > I've been exploring the possibility of hosting the Squeak/Pharo VM on MirageOS. MirageOS is a Unikernal or "Library OS" where rather than creating an executable to be run on Linux the compiler analyses dependencies right down through the device drivers and creates a kernel that can be booted on Amazon AWS EC2. > http://www.openmirage.org/ > http://queue.acm.org/detail.cfm?id=2566628 > http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems-for… > > It seems to me that having a Smalltalk hosed in this environment would be quite useful on a number of fronts: > * A simple build and deploy of Smalltalk to the cloud. The Mirage group is building unikernals from sources and then because they are small committing the entire kernel to GitHub and then pushing the kernel to AWS EC2. Web sites and APIs are an obvious application. http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/. > * Smalltalk on small devices. The Mirage group is running Xen with mirage on small Intel and ARM boards. See http://openmirage.org/blog/announcing-mirage-20-release/. > * These kernels can be very small, on the order of 0.25Meg for minimal HTTP servers with boot times in the ms range. A project called Jitsu https://github.com/MagnusS/jitsu modifies a DNS server to boot kernels in response to socket requests. The system making the request is unaware of the boot. > * The Mirage group envisions thousands of kernels running on a single Hypervisor with his speed inter-kernel communications. See http://openmirage.org/blog/update-on-vchan. The facilitates Smalltalk systems consisting of a swarm of communicating images where each image is single threaded and concurrency is via message passing. > > I've corresponded with the MirageOS group asking if they thought a language VM could be hosted on Mirage. The discussion is here: http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00053.h…. Assuming we understood each other it seems the answer is "yes that should be possible". > > My question is what would be involved on the Smalltalk side? I presume the effort is going to be mostly in the VM. Based on the conversation on the Mirage list I think the VM will need to be able to run a Library with an entry point called by Mirage. That seems similar to the " Embedding/VM as a DLL" project proposed here: http://www.mirandabanda.org/cogblog/cog-projects/. Is this project active? The comment indicates this is mostly refactoring & repackaging. Anyone have a perspective on this? > > Best regards > > Joerg > > > > > Hi, I've been thinking on that kind of question from a different point of view: cutting an application in parts, and running several single core pharo vm in different xen vms with inter domain communication (or network communication if clustered on different hosts) in order to achieve multi core processing. For ipc communication, I found libvchan , whose documentation seems to be only the xen source code and a small sample in the code, and the vicent bernadoff open mirage post (no a lot of doc) and I also found xen sockect, an older paper here http://www.researchgate.net/publication/221461399_XenSocket_A_High-Throughp… that look interesting because it has the same api as sockets, but no more information, it seems to be only an old research paper. Forthe last one, it has been be done on top of grant table and could probably be written but I guess not trivial at all. About openmirage, since the unikernel has to deal with code dependencies, how to integrate the dependency analyser with smalltalk code in order to build a specialized vm ? And about your idea of running the vm in a dll, how will it manage primitives in specialized libraries ? I suppose you would also have to go into the dependency toolchain to describe pharo packages dependencies in terms of openmirage functionality dependencies ? May be you could also look at Android port of pharo, since it runs the vm as a jni dll ? But the primitive part of android port (libraries) do not have to deal with dependencies since android is based on linux. -- Regards, Alain

Jan. 14, 2015

Hosting the Pharo VM on MirageOS

by Joerg Beekmann, DeepCove Labs

Hi I've been exploring the possibility of hosting the Squeak/Pharo VM on MirageOS. MirageOS is a Unikernal or "Library OS" where rather than creating an executable to be run on Linux the compiler analyses dependencies right down through the device drivers and creates a kernel that can be booted on Amazon AWS EC2. http://www.openmirage.org/ http://queue.acm.org/detail.cfm?id=2566628 http://blog.acolyer.org/2015/01/13/unikernels-library-operating-systems-for… It seems to me that having a Smalltalk hosed in this environment would be quite useful on a number of fronts: * A simple build and deploy of Smalltalk to the cloud. The Mirage group is building unikernals from sources and then because they are small committing the entire kernel to GitHub and then pushing the kernel to AWS EC2. Web sites and APIs are an obvious application. http://amirchaudhry.com/from-jekyll-to-unikernel-in-fifty-lines/. * Smalltalk on small devices. The Mirage group is running Xen with mirage on small Intel and ARM boards. See http://openmirage.org/blog/announcing-mirage-20-release/. * These kernels can be very small, on the order of 0.25Meg for minimal HTTP servers with boot times in the ms range. A project called Jitsu https://github.com/MagnusS/jitsu modifies a DNS server to boot kernels in response to socket requests. The system making the request is unaware of the boot. * The Mirage group envisions thousands of kernels running on a single Hypervisor with his speed inter-kernel communications. See http://openmirage.org/blog/update-on-vchan. The facilitates Smalltalk systems consisting of a swarm of communicating images where each image is single threaded and concurrency is via message passing. I've corresponded with the MirageOS group asking if they thought a language VM could be hosted on Mirage. The discussion is here: http://lists.xenproject.org/archives/html/mirageos-devel/2015-01/msg00053.h…. Assuming we understood each other it seems the answer is "yes that should be possible". My question is what would be involved on the Smalltalk side? I presume the effort is going to be mostly in the VM. Based on the conversation on the Mirage list I think the VM will need to be able to run a Library with an entry point called by Mirage. That seems similar to the " Embedding/VM as a DLL" project proposed here: http://www.mirandabanda.org/cogblog/cog-projects/. Is this project active? The comment indicates this is mostly refactoring & repackaging. Anyone have a perspective on this? Best regards Joerg

Jan. 14, 2015