- Pharo-dev - lists.pharo.org

Cog Rump Xen unikernel
by Ben Coman Dec. 13, 2015

Dec. 13, 2015

I got curious about following up these two threads making Cog a unikernel... Cog in the cloud http://forum.world.st/Cog-in-the-cloud-td4796514.html Hosting the Pharo VM on MirageOS http://forum.world.st/Hosting-the-Pharo-VM-on-MirageOS-td4799424.html#none One path forward is to use rumpkernel, which "provides a means to run existing POSIX applications as unikernels on Xen" https://blog.xenproject.org/2015/08/06/on-rump-kernels-and-the-rumprun-unik… Deploying real-world software today as unikernels on Xen with Rumprun http://events.linuxfoundation.org/sites/events/files/slides/xdps15-talk-fin… So I thought I would try the "Building Rumprun Unikernels" tutorial here... https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikerne… which I report here in case it was interest. The tutorial is based on QEMU (and I think I saw a comment somewhere that Cog doesn't run so well on QEMU and needs Bochs), so after that I adapted it from QEMU to Xen. 64-BIT DEBIAN 8 JESSIE & XEN So first I installed Debian 8 Jessie 64-bit on my new laptop $ uname -a >> ...x86_64 Then installed Xen and after a reboot could choose it from the boot menu. $ apt-get update $ apt get install xen-system-amd64 libxen-dev REBOOTED Tested manual selection from menu to boot into Xen dom0. Checked Xen was operating with... $ xl info 32-BIT DEBIAN 8 JESSIE & XEN However later I hit some complications determining whether the rump unikernel was being compiled 32-bit or 64-bit in order to mix in with 32-bit Pharo, so to simplify things I did a complete reinstall of my system as 32-bit. $ uname -a >> ...686-pae Then installed Xen again Xen only has a 64-bit hypervisor, but this was not immediately available under 32-bit OS. I can't exactly remember but I think I just needed to add multiarch before it would find the Xen package. $ apt-get update $ apt-get install multiarch $ apt get install xen-system-amd64 libxen-dev Otherwise it may have needed something like "dpkg --add-architecture ..." REBOOTED First test, manually selected to boot into Xen dom0, after which as root the following checked Xen was operating... $ xl info Then set it to automatically boot into Xen... $ dpkg-divert --divert /etc/grub.d/08_linux_xen --rename /etc/grub.d/20_linux_xen $ update-grub REBOOTED ==================================== DID THE QUICK START TUTORIAL USING GENERIC HW ON QEMU 0. Installed support tools and libs $ apt-get install build-essential git-core cmake 1. Built the rumprun platform $ mkdir -p ~/Repos/rumprun-hw $ cd ~/Repos/rumprun-hw $ git clone http://repo.rumpkernel.org/rumprun $ cd rumprun $ git submodule update --init $ CC=cc ./build-rr.sh hw Took note of this output needed later... >> toolchain tuple: i486-rumprun-netbsdelf >> cc wrapper: i486-rumprun-netbsdelf-gcc $ echo $(pwd) /Users/ben/Repos/rumpkernel-hw/rumprun $ export PATH="${PATH}:$(pwd)/rumprun/bin" 2. Built a toy application $ mkdir -p test $ cd test $ cat > helloer.c << EOF .... $ cc -o helloer helloer.c $ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c $ rumprun-bake hw_generic helloer-rumprun.bin helloer-rumprun 3. Ran the toy application $ su $ rumprun qemu -i helloer-rumprun.bin In another terminal, check its running $ xl list >> Name ID Mem ...etc >> Domain-0 0 7583 >> rumprun-hellor-rumprun.bin 1 64 The app takes over the terminal its run from, so from another terminal... $ xl destroy rumprun-hellor-rumprun.bin ==================================== ADAPTED THE QUICK START TUTORIAL FOR XEN PARAVIRTUALIZATION 0. Started a new terminal to get an environment with a clean PATH 1. Built the rumprun platform $ mkdir -p ~/Repos/rumprun-xen $ cd ~/Repos/rumprun-xen $ git clone http://repo.rumpkernel.org/rumprun $ cd rumprun $ git submodule update --init $ CC=cc ./build-rr.sh xen #<<<< Took note of this output needed later... >> toolchain tuple: i486-rumprun-netbsdelf >> cc wrapper: i486-rumprun-netbsdelf-gcc $ echo $(pwd) /Users/ben/Repos/rumpkernel-xen/rumprun $ export PATH="${PATH}:$(pwd)/rumprun/bin" 2. Built a toy application $ mkdir -p test $ cd test $ cat > helloer.c << EOF .... $ cc -o helloer helloer.c $ i486-rumprun-netbsdelf-gcc -o helloer-rumprun helloer.c $ rumprun-bake xen_pv helloer-rumprun.bin helloer-rumprun 3. Ran toy application $ cd .. $ su $ export PATH="${PATH}:`pwd`/rumprun/bin $ cd test $ rumprun xen -i helloer-rumprun.bin $ xl list >> Name ID Mem ...etc >> Domain-0 0 7583 >> rumprun-hellor-rumprun.bin 1 64 This took over that terminal, so from another terminal... $ xl destroy rumprun-hellor-rumprun.bin ==================================== TRIED IT USING CMAKE $ cat > CMakeLists.txt << EOF see this link for text... https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikerne… $ mkdir host_bld $ cd host_bld $ cmake -G "Unix Makefiles" .. $ make $ ./helloer $ cd .. $ mkdir rumprun_bld $ cd rumprun_bld Now with the given cmake line I got an error message: Could not find toolchain file: ../../rumprun/rumprun/share/x86_64-rumprun-netbsd-toolchain.cmake $ find ../.. -name x86_64-rumprun-netbsd-toolchain.cmake found it, so that the following worked... $ cmake -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake .. $ make $ rumprun-bake xen_pv helloer.bin helloer $ rumprun-bake hw_generic helloer.bin helloer $ rumprun xen -i helloer.bin Next I'll report on initial steps to get Cog working on Xen as a Rump unikernel... cheers -ben

3 6

Unicode Support
by EuanM Dec. 13, 2015

Dec. 13, 2015

I'm currently groping my way to seeing how feature-complete our Unicode support is. I am doing this to establish what still needs to be done to provide full Unicode support. This seems to me to be an area where it would be best to write it once, and then have the same codebase incorporated into the Smalltalks that most share a common ancestry. I am keen to get: equality-testing for strings; sortability for strings which have ligatures and diacritic characters; and correct round-tripping of data. Call to action: ========== If you have comments on these proposals - such as "but we already have that facility" or "the reason we do not have these facilities is because they are dog-slow" - please let me know them. If you would like to help out, please let me know. If you have Unicode experience and expertise, and would like to be, or would be willing to be, in the 'council of experts' for this project, please let me know. If you have comments or ideas on anything mentioned in this email In the first instance, the initiative's website will be: http://smalltalk.uk.to/unicode.html I have created a SqueakSource.com project called UnicodeSupport I want to avoid re-inventing any facilities which already exist. Except where they prevent us reaching the goals of: - sortable UTF8 strings - sortable UTF16 strings - equivalence testing of 2 UTF8 strings - equivalence testing of 2 UTF16 strings - round-tripping UTF8 strings through Smalltalk - roundtripping UTF16 strings through Smalltalk. As I understand it, we have limited Unicode support atm. Current state of play =============== ByteString gets converted to WideString when need is automagically detected. Is there anything else that currently exists? Definition of Terms ============== A quick definition of terms before I go any further: Standard terms from the Unicode standard =============================== a compatibility character : an additional encoding of a *normal* character, for compatibility and round-trip conversion purposes. For instance, a 1-byte encoding of a Latin character with a diacritic. Made-up terms ============ a convenience codepoint : a single codepoint which represents an item that is also encoded as a string of codepoints. (I tend to use the terms compatibility character and compatibility codepoint interchangably. The standard only refers to them as compatibility characters. However, the standard is determined to emphasise that characters are abstract and that codepoints are concrete. So I think it is often more useful and productive to think of compatibility or convenience codepoints). a composed character : a character made up of several codepoints Unicode encoding explained ===================== A convenience codepoint can therefore be thought of as a code point used for a character which also has a composed form. The way Unicode works is that sometimes you can encode a character in one byte, sometimes not. Sometimes you can encode it in two bytes, sometimes not. You can therefore have a long stream of ASCII which is single-byte Unicode. If there is an occasional Cyrillic or Greek character in the stream, it would be represented either by a compatibility character or by a multi-byte combination. Using compatibility characters can prevent proper sorting and equivalence testing. Using "pure" Unicode, ie. "normal encodings", can cause compatibility and round-tripping probelms. Although avoiding them can *also* cause compatibility issues and round-tripping problems. Currently my thinking is: a Utf8String class an Ordered collection, with 1 byte characters as the modal element, but short arrays of wider strings where necessary a Utf16String class an Ordered collection, with 2 byte characters as the modal element, but short arrays of wider strings beginning with a 2-byte endianness indicator. Utf8Strings sometimes need to be sortable, and sometimes need to be compatible. So my thinking is that Utf8String will contain convenience codepoints, for round-tripping. And where there are multiple convenience codepoints for a character, that it standardises on one. And that there is a Utf8SortableString which uses *only* normal characters. We then need methods to convert between the two. aUtf8String asUtf8SortableString and aUtf8SortableString asUtf8String Sort orders are culture and context dependent - Sweden and Germany have different sort orders for the same diacritic-ed characters. Some countries have one order in general usage, and another for specific usages, such as phone directories (e.g. UK and France) Similarly for Utf16 : Utf16String and Utf16SortableString and conversion methods A list of sorted words would be a SortedCollection, and there could be pre-prepared sortBlocks for them, e.g. frPhoneBookOrder, deOrder, seOrder, ukOrder, etc along the lines of aListOfWords := SortedCollection sortBlock: deOrder If a word is either a Utf8SortableString, or a well-formed Utf8String, then we can perform equivalence testing on them trivially. To make sure a Utf8String is well formed, we would need to have a way of cleaning up any convenience codepoints which were valid, but which were for a character which has multiple equally-valid alternative convenience codepoints, and for which the string currently had the "wrong" convenience codepoint. (i.e for any character with valid alternative convenience codepoints, we would choose one to be in the well-formed Utf8String, and we would need a method for cleaning the alternative convenience codepoints out of the string, and replacing them with the chosen approved convenience codepoint. aUtf8String cleanUtf8String With WideString, a lot of the issues disappear - except round-tripping(although I'm sure I have seen something recently about 4-byte strings that also have an additional bit. Which would make some Unicode characters 5-bytes long.) (I'm starting to zone out now - if I've overlooked anything - obvious, subtle, or somewhere in between, please let me know) Cheers, Euan

11 41

D language
by phil＠highoctane.be Dec. 13, 2015

Dec. 13, 2015

Anyone having used D to build system-level extensions for Pharo? TIA, Phil

2 5

[JOB] Pharo Engineer, IoT
by Marcus Denker Dec. 12, 2015

Dec. 12, 2015

Engineer Position 1 Year =================== INRIA RMoD and the Pharo Consortium are looking for a full-time engineer for helping with developing Pharo. The focus of this job is "Pharo on embedded and mobile devices for the internet of things". - Embedded Pharo on ARM The currently available class of ARM devices (BeagleBone, Raspberry PI) are a very interesting platform for Pharo. The engineer should strengthen the existing efforts to get Pharo working on these platforms. - Pharo on Android Improve the support of Android devices. With Pharo6, we will migrate to a complete new design and implementation of UI, Windowing and graphics. The engineer will work on making these features available on Android devices. - Pharo itself The engineer is expected to be actively involved in helping to improve the infrastructure that the Pharo community used (CI server for continues integration, interplay with issue tracking) and to actively help with day-to-day bug fixing. We are looking for a candidate with some prior experience with Pharo - Knowledge in Dynamic Languages - Experience with mobile development is a plus - English Work in a research lab, but no paper writing, no teaching, no administration. In addition: - centrally located (one hour from Paris, 1h20 from London, 35 min from Brussels) - try a lot of good beer (our office is just 10km from the border to Belgium) - Program with Pharo all day! Lille is an active city but not expensive (lot of students...), centrally located with good connections by train to everywhere (including CDG and Brussels airports). Duration: 12 month Starting date: as soon as possible (Spring 2016) Salary: 2600-3200 euros brut / month, depending on experience Location: Lille (no remote job possible) More about Pharo: http://www.pharo.org INRIA RMOD: http://rmod.lille.inria.fr INRIA Lille: http://www.inria.fr/lille/ INRIA in General: http://www.inria.fr Lille: http://en.wikipedia.org/wiki/Lille http://wikitravel.org/en/Lille Do not hesitate to contact us for more information.

1 0

[JOB] Pharo Engineer, Infrastructure
by Marcus Denker Dec. 12, 2015

Dec. 12, 2015

Engineer Position 1 year + 1 year ================================== INRIA RMoD and the Pharo Consortium are looking for a full-time engineer for helping with developing Pharo and the infrastructure around it. The focus of this job is the infrastructure that Pharo provides to external projects as well as the infrastructure we use to develop Pharo itself. - Infrastructure for external projects Pharo is used in many contexts: an ever growing number of frameworks are contributed by the community. Pharo already provides CI infrastructure, but what is missing is the automatic check of collections of frameworks and libraries. We want to provides validated "distributions" focussing on certain tasks, for example Pharo for Web development. - Pharo and Git Pharo has some support for using Git. The goal is to improve the support of git and other versioning systems. - Pharo itself The engineer is expected to be actively involved in helping to improve the infrastructure that the Pharo community uses (CI server for continues integration, interplay with issue tracking) and to actively help with day-to-day bug fixing. We are looking for a candidate with prior experience with Pharo and possibly some background in continuous integration: - Knowledge in Dynamic Languages - Continuous integration and deployment - English Work in a research lab, but no paper writing, no teaching, no administration. In addition: - Centrally located (one hour from Paris, 1h20 from London, 35 min from Brussels) - Try a lot of good beer (our office is just 10km from the border to Belgium) - Program with Pharo all day! Lille is an active city but not expensive (lot of students...), centrally located with good connections by train to everywhere (including CDG and Brussels airports). Duration: 12 + 12 months Starting date: Feb or March 2016 Salary: 2600-3200 euros brut / month, depending on experience Location: Lille (no remote job possible) More about Pharo: http://www.pharo.org INRIA RMOD: http://rmod.lille.inria.fr INRIA Lille: http://www.inria.fr/lille/ INRIA in General: http://www.inria.fr Lille: http://en.wikipedia.org/wiki/Lille http://wikitravel.org/en/Lille Do not hesitate to contact us for more information.

1 0

squeak & pharo versions of crypto and SecureSession
by Robert Withers Dec. 12, 2015

Dec. 12, 2015

Folks, I cross-tested with Pharo and here are the good versions: Cryptography - squeak: Cryptography-rww.49.mcz - pharo: Cryptography-Pharo-rww.38.mcz SecureSession: - squeak & pharo: SecureSession-RobertWithers.20.mcz I hope this helps, Robert

1 0

Why Zinc servers implemented with options dictionary instead of normal instance variables?
by Denis Kudriashov Dec. 11, 2015

Dec. 11, 2015

Hello. Just looked on ZnServer code. And I was wondering when see that everything is stored inside "options" variable (which is dictionary). For example port number is option which is really strange. What the reason for that?

3 6

[pharo-project/pharo-core]
by GitHub Dec. 11, 2015

Dec. 11, 2015

Branch: refs/tags/50495 Home: https://github.com/pharo-project/pharo-core

1 0

[pharo-project/pharo-core] 950ae8: 50495
by GitHub Dec. 11, 2015

Dec. 11, 2015

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 950ae8914e92c7b765a8e39656335c8140d8c938 https://github.com/pharo-project/pharo-core/commit/950ae8914e92c7b765a8e396… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-11 (Fri, 11 Dec 2015) Changed paths: M Morphic-Widgets-Pluggable.package/PluggableListMorph.class/instance/debug and other/userString.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50494.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50495.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50494.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50495.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 50495 17226 PluggableListMorph>>#userString should return one string https://pharo.fogbugz.com/f/cases/17226 http://files.pharo.org/image/50/50495.zip

1 0

Hook "WACurrentRequestContext" into debugger?
by Mariano Martinez Peck Dec. 11, 2015

Dec. 11, 2015

Hi guys, This thing I will ask in this email it's in my mind since YEARS. But I have always thought it was like that and that there was nothing we could do. However, I think it's time I ask again :) For those that have used Seaside, and you try to debug, you know that upon request processing seaside uses Exceptions mechanisim to always have access to the request, session, etc. They way that is done is very smart :) WACurrentRequestContext use: self during: aBlock In that case, "self" is the request instance and aBlock the closure that takes care of the request processing. So, inside that closure, everywhere you do "WACurrentRequestContext value" you get the correct request instance. So..that's great for Seaside, but debugging gets complicated. While you can restart, proceed, etc, once inside debugger, you cannot evaluate any piece of code that will use the session or request because you get a WARequestContextNotFound. Of course, because I guess the evaluation you do from cmd+d on a piece of text or via the debugger inspector, creates another closure/context which does not receive the WACurrentRequestContext instance. Now....besides WACurrentRequestContext I have my own class UserContextInformation where I basically have a bunch of stuff associated to the logged user. And I do exactly the same as the WACurrentRequestContext. And I have the same problem. I really want to be able to fix this. Anyone have an idea on how can I do it? I guess I can change the debugger, in the place where I evaluate code so that I wrap that evaluation with my request context instance??? Thoughts? -- Mariano http://marianopeck.wordpress.com

4 30