- Pharo-dev - lists.pharo.org

Cog Rump Xen unikernel

by Ben Coman

I got curious about following up these two threads making Cog a unikernel... Cog in the cloud http://forum.world.st/Cog-in-the-cloud-td4796514.html Hosting the Pharo VM on MirageOS http://forum.world.st/Hosting-the-Pharo-VM-on-MirageOS-td4799424.html#none One path forward is to use rumpkernel, which "provides a means to run existing POSIX applications as unikernels on Xen" https://blog.xenproject.org/2015/08/06/on-rump-kernels-and-the-rumprun-unik… Deploying real-world software today as unikernels on Xen with Rumprun http://events.linuxfoundation.org/sites/events/files/slides/xdps15-talk-fin… So I thought I would try the "Building Rumprun Unikernels" tutorial here... https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikerne… which I report here in case it was interest. The tutorial is based on QEMU (and I think I saw a comment somewhere that Cog doesn't run so well on QEMU and needs Bochs), so after that I adapted it from QEMU to Xen. 64-BIT DEBIAN 8 JESSIE & XEN So first I installed Debian 8 Jessie 64-bit on my new laptop $ uname -a >> ...x86_64 Then installed Xen and after a reboot could choose it from the boot menu. $ apt-get update $ apt get install xen-system-amd64 libxen-dev REBOOTED Tested manual selection from menu to boot into Xen dom0. Checked Xen was operating with... $ xl info 32-BIT DEBIAN 8 JESSIE & XEN However later I hit some complications determining whether the rump unikernel was being compiled 32-bit or 64-bit in order to mix in with 32-bit Pharo, so to simplify things I did a complete reinstall of my system as 32-bit. $ uname -a >> ...686-pae Then installed Xen again Xen only has a 64-bit hypervisor, but this was not immediately available under 32-bit OS. I can't exactly remember but I think I just needed to add multiarch before it would find the Xen package. $ apt-get update $ apt-get install multiarch $ apt get install xen-system-amd64 libxen-dev Otherwise it may have needed something like "dpkg --add-architecture ..." REBOOTED First test, manually selected to boot into Xen dom0, after which as root the following checked Xen was operating... $ xl info Then set it to automatically boot into Xen... $ dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen $ update-grub REBOOTED ==================================== DID THE QUICK START TUTORIAL USING GENERIC HW ON QEMU 0. Installed support tools and libs $ apt-get install build-essential git-core cmake 1. Built the rumprun platform $ mkdir -p ~/Repos/rumprun-hw $ cd ~/Repos/rumprun-hw $ git clone http://repo.rumpkernel.org/rumprun $ cd rumprun $ git submodule update --init $ CC=cc ./build-rr.sh hw Took note of this output needed later... >> toolchain tuple: i486-rumprun-netbsdelf >> cc wrapper: i486-rumprun-netbsdelf-gcc $ echo $(pwd) /Users/ben/Repos/rumpkernel-hw/rumprun $ export PATH="${PATH}:$(pwd)/rumprun/bin" 2. Built a toy application $ mkdir -p test $ cd test $ cat > helloer.c << EOF .... $ cc -o helloer helloer.c $ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c $ rumprun-bake hw_generic helloer-rumprun.bin helloer-rumprun 3. Ran the toy application $ su $ rumprun qemu -i helloer-rumprun.bin In another terminal, check its running $ xl list >> Name ID Mem ...etc >> Domain-0 0 7583 >> rumprun-hellor-rumprun.bin 1 64 The app takes over the terminal its run from, so from another terminal... $ xl destroy rumprun-hellor-rumprun.bin ==================================== ADAPTED THE QUICK START TUTORIAL FOR XEN PARAVIRTUALIZATION 0. Started a new terminal to get an environment with a clean PATH 1. Built the rumprun platform $ mkdir -p ~/Repos/rumprun-xen $ cd ~/Repos/rumprun-xen $ git clone http://repo.rumpkernel.org/rumprun $ cd rumprun $ git submodule update --init $ CC=cc ./build-rr.sh xen #<<<< Took note of this output needed later... >> toolchain tuple: i486-rumprun-netbsdelf >> cc wrapper: i486-rumprun-netbsdelf-gcc $ echo $(pwd) /Users/ben/Repos/rumpkernel-xen/rumprun $ export PATH="${PATH}:$(pwd)/rumprun/bin" 2. Built a toy application $ mkdir -p test $ cd test $ cat > helloer.c << EOF .... $ cc -o helloer helloer.c $ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c $ rumprun-bake xen_pv helloer-rumprun.bin helloer-rumprun 3. Ran toy application $ cd .. $ su $ export PATH="${PATH}:`pwd`/rumprun/bin $ cd test $ rumprun xen -i helloer-rumprun.bin $ xl list >> Name ID Mem ...etc >> Domain-0 0 7583 >> rumprun-hellor-rumprun.bin 1 64 This took over that terminal, so from another terminal... $ xl destroy rumprun-hellor-rumprun.bin ==================================== TRIED IT USING CMAKE $ cat > CMakeLists.txt << EOF see this link for text... https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikerne… $ mkdir host_bld $ cd host_bld $ cmake -G "Unix Makefiles" .. $ make $ ./helloer $ cd .. $ mkdir rumprun_bld $ cd rumprun_bld Now with the given cmake line I got an error message: Could not find toolchain file: ../../rumprun/rumprun/share/x86_64-rumprun-netbsd-toolchain.cmake $ find ../.. -name x86_64-rumprun-netbsd-toolchain.cmake found it, so that the following worked... $ cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake .. $ make $ rumprun-bake xen_pv helloer.bin helloer $ rumprun-bake hw_generic helloer.bin helloer $ rumprun xen -i helloer.bin Next I'll report on initial steps to get Cog working on Xen as a Rump unikernel... cheers -ben

Dec. 11, 2015

Re: [Pharo-dev] Why Zinc servers implemented with options dictionary instead of normal instance variables?

by Dimitris Chloupis

easy to answer that .... initialize their values , make sure you use sensible names for the dictionary entries. documented and still no image pollution. On Fri, Dec 11, 2015 at 8:02 PM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > On 11 Dec 2015, at 17:45, Dimitris Chloupis <kilon.alios(a)gmail.com> > wrote: > > > > the real question here is why pharo classes have the tendency to output > so much data through individual methods when a single method returning a > dictionary or any other collection would make much more sense and would be > far more flexible and elegant to deal with. Unnecessary pollution of the > image with methods. > > Hmm, I wouldn't call that pollution. I think it is a form of > documentation. The options are a good example of that. We could add an > accessor for #options, just returning the dictionary that is now > encapsulated, and then delete all the nice accessors, like > #useGzipCompressionAndChunking. With the accessors gone, how would you > discover or see what is available ? Where should the existing options be > documented ? Now you can browse them in the options protocol, all in one > place, one by one, with documentation and clearly visible defaults. > > > On Fri, Dec 11, 2015 at 6:11 PM Sven Van Caekenberghe <sven(a)stfx.eu> > wrote: > > > > > On 11 Dec 2015, at 16:28, Denis Kudriashov <dionisiydk(a)gmail.com> > wrote: > > > > > > > > > 2015-12-11 15:51 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > > Because there are many options, 10, 20, and at that time slots were > not yet available. > > > > > > Is not it is same case as class with 20 variables which always smell > bad? > > > > That's why it is a dictionary ;-) > > > > Can't an object have 10, 20 or more properties (independent of how they > are stored) ? > > > > You could of course delegate options to another sub object but why ? > > > > Apart from that, I do agree that both the main server object and the > main client object are too big, they are like big interfaces to a lot of > related functionality. The objects behind them, the ones in Zinc-HTTP-Core, > are much more single purpose and simpler. > > >

Dec. 11, 2015

Re: [Pharo-dev] Why Zinc servers implemented with options dictionary instead of normal instance variables?

by Sven Van Caekenberghe

> On 11 Dec 2015, at 17:45, Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > > the real question here is why pharo classes have the tendency to output so much data through individual methods when a single method returning a dictionary or any other collection would make much more sense and would be far more flexible and elegant to deal with. Unnecessary pollution of the image with methods. Hmm, I wouldn't call that pollution. I think it is a form of documentation. The options are a good example of that. We could add an accessor for #options, just returning the dictionary that is now encapsulated, and then delete all the nice accessors, like #useGzipCompressionAndChunking. With the accessors gone, how would you discover or see what is available ? Where should the existing options be documented ? Now you can browse them in the options protocol, all in one place, one by one, with documentation and clearly visible defaults. > On Fri, Dec 11, 2015 at 6:11 PM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > On 11 Dec 2015, at 16:28, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > > > > > > 2015-12-11 15:51 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > Because there are many options, 10, 20, and at that time slots were not yet available. > > > > Is not it is same case as class with 20 variables which always smell bad? > > That's why it is a dictionary ;-) > > Can't an object have 10, 20 or more properties (independent of how they are stored) ? > > You could of course delegate options to another sub object but why ? > > Apart from that, I do agree that both the main server object and the main client object are too big, they are like big interfaces to a lot of related functionality. The objects behind them, the ones in Zinc-HTTP-Core, are much more single purpose and simpler.

Dec. 11, 2015

[pharo-project/pharo-core] 950ae8: 50495

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 950ae8914e92c7b765a8e39656335c8140d8c938 https://github.com/pharo-project/pharo-core/commit/950ae8914e92c7b765a8e396… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-11 (Fri, 11 Dec 2015) Changed paths: M Morphic-Widgets-Pluggable.package/PluggableListMorph.class/instance/debug and other/userString.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50494.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50495.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50494.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50495.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 50495 17226 PluggableListMorph>>#userString should return one string https://pharo.fogbugz.com/f/cases/17226 http://files.pharo.org/image/50/50495.zip

Dec. 11, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50495 Home: https://github.com/pharo-project/pharo-core

Dec. 11, 2015

Re: [Pharo-dev] Why Zinc servers implemented with options dictionary instead of normal instance variables?

by Dimitris Chloupis

the real question here is why pharo classes have the tendency to output so much data through individual methods when a single method returning a dictionary or any other collection would make much more sense and would be far more flexible and elegant to deal with. Unnecessary pollution of the image with methods. On Fri, Dec 11, 2015 at 6:11 PM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > On 11 Dec 2015, at 16:28, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > > > > > > 2015-12-11 15:51 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > Because there are many options, 10, 20, and at that time slots were not > yet available. > > > > Is not it is same case as class with 20 variables which always smell bad? > > That's why it is a dictionary ;-) > > Can't an object have 10, 20 or more properties (independent of how they > are stored) ? > > You could of course delegate options to another sub object but why ? > > Apart from that, I do agree that both the main server object and the main > client object are too big, they are like big interfaces to a lot of related > functionality. The objects behind them, the ones in Zinc-HTTP-Core, are > much more single purpose and simpler. >

Dec. 11, 2015

Re: [Pharo-dev] Why Zinc servers implemented with options dictionary instead of normal instance variables?

by Sven Van Caekenberghe

> On 11 Dec 2015, at 16:28, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > > > 2015-12-11 15:51 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Because there are many options, 10, 20, and at that time slots were not yet available. > > Is not it is same case as class with 20 variables which always smell bad? That's why it is a dictionary ;-) Can't an object have 10, 20 or more properties (independent of how they are stored) ? You could of course delegate options to another sub object but why ? Apart from that, I do agree that both the main server object and the main client object are too big, they are like big interfaces to a lot of related functionality. The objects behind them, the ones in Zinc-HTTP-Core, are much more single purpose and simpler.

Dec. 11, 2015

Re: [Pharo-dev] Hook "WACurrentRequestContext" into debugger?

by Mariano Martinez Peck

On Fri, Dec 11, 2015 at 12:51 PM, Max Leske <maxleske(a)gmail.com> wrote: > > On 11 Dec 2015, at 16:39, Mariano Martinez Peck <marianopeck(a)gmail.com> > wrote: > > > > On Fri, Dec 11, 2015 at 12:25 PM, Max Leske <maxleske(a)gmail.com> wrote: > >> >> On 10 Dec 2015, at 21:21, Mariano Martinez Peck <marianopeck(a)gmail.com> >> wrote: >> >> >> >> On Thu, Dec 10, 2015 at 4:12 PM, Mariano Martinez Peck < >> marianopeck(a)gmail.com> wrote: >> >>> >>> >>> On Thu, Dec 10, 2015 at 7:27 AM, Max Leske <maxleske(a)gmail.com> wrote: >>> >>>> >>>> On 07 Dec 2015, at 17:17, Mariano Martinez Peck <marianopeck(a)gmail.com> >>>> wrote: >>>> >>>> OK Max. In the last version you can find the "likely" final version. >>>> It's extremely simple, easy to use, and very very little code. Only one >>>> override. All WADynamicVariables subclasses handled automatically. All you >>>> need to do is to register the handler: >>>> >>>> app filter configuration at: #exceptionHandler put: >>>> WAPharoDebuggererErrorHandler. >>>> >>>> It works for exceptions and both Halt too. The only limit is only the >>>> "last opened debugger" will be correct and all previous will be wrong since >>>> they will be using another values for dynamic variables. >>>> >>>> I will see if people want to integrate this directly into Seaside. >>>> >>>> >>>> Very nice! And since you provide a separate error handler itâs easy to >>>> integrate into seaside too by simply adding it to the handler chain. >>>> >>>> >>> Exactly. The only issue to fix is the override in WADynamicVariable >> >>> defaultAction. >>> And that's not easy. Why? Because at that level, we know nothing about >>> the current seaside app or anything. We don't know which seaside app those >>> signal came from. Of course, I can still do the check to the handler to see >>> if that has that variable stored. But... What if now you have set another >>> error handler? Or what about OTHER possible registered apps which use a >>> different error handler. >>> Thoughts? I have some HACKS in mind, but they are hacks and none solves >>> all problems. And only for Pharo. >>> Any idea to solve this would be nice. >>> >>> >> Assuming you are using GTDebugger (should be very easy to adapt for >> normal debugger), we can do this: >> >> defaultAction >> " This is an override from SeasidePharoDebugging because before raising a >> signal we >> first check if we have the value stored. That way, everywhere >> we evaluate code in a debugger that end ups doing >> 'WACurrentRequestContext value' will simply >> get up to this place. Since in WAPharoDebuggererErrorHandler >> #open: we >> stored the dynamic variables, we should have the value " >> | openedSeasideErrorDebuggers | >> *openedSeasideErrorDebuggers := ((UIManager default currentWorld >> submorphs select: [ :each | each model isKindOf: GTGenericStackDebugger ]) * >> * collect: [ :each | each model ]) * >> * select: [ :each2 | (each2 session interruptedProcess suspendedContext >> method selector = #openDebuggerOn:)* >> * and: (each2 session interruptedProcess suspendedContext sender receiver >> class = WADynamicVariablesErrorHandler) ].* >> ^ openedSeasideErrorDebuggers >> ifEmpty: [ self class defaultValue ] >> ifNotEmpty: [ >> (WADynamicVariablesErrorHandler storedDynamicVariable: self class) >> ifNil: [ self class defaultValue ] >> ] >> >> But that..the only thing it solves is that we would not be getting the >> value of dynamic variables from previous errors if there is none debugger >> opened from a seaside continuation. If you let a debugger opened debugging >> a seaside error, then there is no magic we can do I think. >> Imagine you were in a debugger and you evaluated: "WAComponent new >> session". The signal raised from there knows NOTHING about the debugger >> where that was evaluated... so at #defaultAction level you know nothing >> about what triggered that. >> >> The only thing to solve this would be to hook into the do-it, print it, >> inspect etc...too complicated. >> >> >> Thoughts? >> >> >> I think youâre solution is already pretty good. I wouldnât really care if >> multiple debuggers are a problem. Evaluating a piece of code in context of >> the dynamic variables is already a lot of help. Although, if you donât know >> that youâre seeing a different session / request, thatâs bad... >> >> I do have one idea but it might be a bit more complicated and is not >> Seaside specific: if we could attach the context of the evaluation to the >> suspended context, the handler search should automatically yield the >> correct handlers for the dynamic variables (although I donât know exactly >> how the VM resolves the handlers). Alternatively, instead of attaching the >> whole stack, it would suffice to copy the handler contexts and attach them >> to the evaluation context. Both approaches would also solve the problem of >> multiple debuggers. >> > > Yes, I have thought about doing something similar. In GemStone for > example, there is another possibility since it allows global static handler > for exceptions. So for example you can say install a block to be executed > when nobody handled exception XXX. And that closure execution is BEFORE the > #defaultAction. So for example in this case I would be able to do that > without the override in WADynamicVariable >> defaultAction. > mmmmm I am not 100% sure about what I said about GemStone hahahaha. > > > Intersting. But wouldnât you have the same problem with multiple > debuggers? Theyâd all use the same global handler, wouldnât they? > > Yes. But I would solve the override :) > > >> For this to work the debugger would need highjack the evaluation of code >> e.g. by using a custom code editor. >> > > Exactly. We would need to hook on everything: print it, do it, debug it, > inspect etc. Maybe at the first level is not that hard (say at debugger > level). But imagine you do a "self session inspect". But then, once inside > the inspector, you can inspect other objects and open other inspectors, > evaluate another piece of code etc... and so the scope would end up in the > first level. > > > Yes, right. Thatâs a problem, although what I usually do is inspect the > session and then use the session in a new handler context to run my code: > > WASession use: self during: [ â¦ ] > Indeed, that works. But again, works only inside that closure. So if you want to open a new inspector, or given the same inspector navigate other objects, then execute or or then type something in a playground etc, you are out of context. Each code evaluation will have it's own new context. So you would need to type above code every time you need that. But at least we agree in the pros and cons of each solution :) > > Thus I donât care too much about what happens outside of the debugger. Of > course, not everyone understands the session handling mechanism, so itâs > probably not the solution for everyone. Still, having access to dynamic > variables *at least* in the debugger, is already a big improvement in > itself. We donât need to rethink everything. > > Well...I think I personally prefer to have it accessible everywhere (even outside the debugger) with the known limitations (that we have discussed many times). But tastes are tastes :) If you prefer only at debugger level, allow multiple debugger, etc... and if you use GT Debugger it looked not so hard to hook into code execution. But make sure you are able to first make the "backend" support (copy exception handlers from suspended process to the new process result of evaluation). > > With current approach we simply propagate the values to everybody , > everywhere. But it's also nice to be able to inspect, explore etc from > anywhere and you know you will be dealing with the context of the last > debugger :) I mean, you can do anything in the image and you will be > dealing with the last debug error context :) > So I think that even both approaches have advantages and disadvantages, > right? > > >> The cool thing about this would be that every evaluation would run in the >> same exception handling contexts as the original process. >> The bad thing is of course that in some situations you actually donât >> care (or want to care) about those handlers (e.g. when you want to trigger >> an exception explicitly during evaluation and donât want it to be swallowed >> by the exception handlers). >> >> Iâve CCâed Andrei because he knows more about UI <-> Debugger >> interaction. Maybe he has an idea how to do this. >> >> > OK, Thanks. > >> >> >> >> >>> >>> >>>> Two tiny things: >>>> - youâre not using the #reset method you wrote but instead manually >>>> assign nil to the variable >>>> >>> >>> yes, thanks. >>> >>> >>>> - I guess the dictionary in the handler could be a SmallDictionary. >>>> There wonât be many entries. >>>> >>>> >>> Well..since there will be likely.. less than 10 entries I don't care >>> which Dictionary to use. However, I would prefer to keep Dictionary. Or if >>> not GRSmallDictionary. >>> >>> >>>> Thanks a lot for this. Iâll probably port this back to Seaside 2.8 so I >>>> can use it too :) >>>> >>> >>> No problem. Thanks for reviewing and give feedback. >>> >>> >>> >>>> >>>> On Sat, Dec 5, 2015 at 11:36 AM, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>>> >>>>> On 05 Dec 2015, at 12:58, Mariano Martinez Peck <marianopeck(a)gmail.com> >>>>> wrote: >>>>> >>>>> Can you try with the latest version that does not use LPC if it works? >>>>> >>>>> >>>>> Works perfectly! >>>>> >>>> >>>> >>>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.com >>>> >>>> >>>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > > -- Mariano http://marianopeck.wordpress.com

Dec. 11, 2015

Re: [Pharo-dev] CompilationContext class>>initialize

by Marcus Denker

they donâtâ¦ I think that is a leftover from back when we had some rule or automatic suggestion to add the âsuper initializeâ, which is wrong on the class side. > On 11 Dec 2015, at 16:55, Stephan Eggermont <stephan(a)stack.nl> wrote: > > Why do CompilationContext and CCompilationContext need super initialize in class side initialize? > > Stephan > >

Dec. 11, 2015

CompilationContext class>>initialize

by Stephan Eggermont

Why do CompilationContext and CCompilationContext need super initialize in class side initialize? Stephan

Dec. 11, 2015