December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [IMPORTANT] Starting migration to Spur VM

by Esteban Lorenzano

> On 14 Dec 2015, at 11:24, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > We will start migration to Spur today. > To complete it, we will require some time, specially to adapt the CI and check everything is ok. > > Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: > - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). > - No more GC stops (noticeable when running large systems) > - We will be able to scale our systems up to 2G memory consumption without loosing performance. > > But, this will have some drawbacks in the first times: > > 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). > - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. > > 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: > - there are some stuff not possible to compatibilise, notably: > - Structures now need to inherit from FFIExternalStructure > - Arrays now are now shadowed > - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. > - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. > 2.1) ASMJIT will be removed from system and put in a separated packages. > NOTE: There will be a blog post explaining FFI-NB architecture during the week. > > 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) > > 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. > > 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted). > ah, I forget it! 6) Some changes to the VM accumulated during the time of transition (some months now). So, some changes introduced there will not be present in new VM. I will merge this changes in the next weeks. cheers! Esteban

Dec. 14, 2015

[IMPORTANT] Starting migration to Spur VM

by Esteban Lorenzano

We will start migration to Spur today. To complete it, we will require some time, specially to adapt the CI and check everything is ok. Spur will allow us to do a big step forward in Pharo development, in the concrete you will see it immediately for this: - We have noticed a speed increment of 100% in tiny benchmarks (and according to Eliot, it will be at least 35% in general on the system). - No more GC stops (noticeable when running large systems) - We will be able to scale our systems up to 2G memory consumption without loosing performance. But, this will have some drawbacks in the first times: 1) VM will not be compatible between versions anymore: Pharo 5.0 will have a Pharo Spur VM associated (and they are not compatible). - There WILL NOT be a "non-spur" version of Pharo 5.0. Once completed the transitions, this will be the only one. 2) NativeBoost-FFI implementation has been replaced with a new implementation who relies in ThreadedFFIPlugin and IA32Plugin. While we worked a lot to do this transition as painless as possible and we achieve a good level of backward compatibility (most uses of #nbCall: should work out of the box), there are some problems we cannot solve: - there are some stuff not possible to compatibilise, notably: - Structures now need to inherit from FFIExternalStructure - Arrays now are now shadowed - in general is a bit slower (impossible to compete with ASM) but in general is not perceptible. - THERE WILL BE BUGS AND NON-IMPLEMENTED FEATURES: Current implementation is validated with Athens and even Roassal was working, but of course that does not covers all cases. 2.1) ASMJIT will be removed from system and put in a separated packages. NOTE: There will be a blog post explaining FFI-NB architecture during the week. 3) There are more or less 70 new failing tests, some of them important than we need to fix as soon as possible. Please, please, please, help us with them :) 4) In general we foresee the system will became unstable some weeks, before it gets back to normal. Please be patient. 5) You will need to adapt your Pharo 5.0 related apps and CI processes (to take care about new VM). Some programs will stop work at all (for example, I think Pharo-launcher will need to be adapted).

Dec. 14, 2015

[pharo-project/pharo-core] 8812a3: 50496

by GitHub

Branch: refs/heads/5.0 Home: https://github.com/pharo-project/pharo-core Commit: 8812a3ea1d836dff057c7f915ffc1fed9b473e6b https://github.com/pharo-project/pharo-core/commit/8812a3ea1d836dff057c7f91… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2015-12-14 (Mon, 14 Dec 2015) Changed paths: A Collections-Sequenceable.package/Interval.class/instance/copying/shuffled.st A Collections-Tests.package/IntervalTest.class/instance/tests/testShuffled.st A OpalCompiler-Core.package/OCASTClosureAnalyzer.class/instance/visiting/visitArgumentNode_.st M OpalCompiler-Core.package/OCASTClosureAnalyzer.class/instance/visiting/visitBlockNode_.st M OpalCompiler-Core.package/OCASTClosureAnalyzer.class/instance/visiting/visitMethodNode_.st A OpalCompiler-Tests.package/OCASTClosureAnalyzerTest.class/instance/testing - variables/testBlockArgumentIsArgumentVariable.st A OpalCompiler-Tests.package/OCASTClosureAnalyzerTest.class/instance/testing - variables/testInlinedBlockArgumentIsArgumentVariable.st A OpalCompiler-Tests.package/OCASTClosureAnalyzerTest.class/instance/testing - variables/testMethodArgumentIsArgumentVariable.st A OpalCompiler-Tests.package/OCOpalExamples.class/instance/examples-variables/exampleForBlockArgument.st A OpalCompiler-Tests.package/OCOpalExamples.class/instance/examples-variables/exampleForInlinedBlockArgument.st A OpalCompiler-Tests.package/OCOpalExamples.class/instance/examples-variables/exampleWithArgument_.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50495.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - scripts/script50496.st R ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50495.st A ScriptLoader50.package/ScriptLoader.class/instance/pharo - updates/update50496.st M ScriptLoader50.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 50496 17233 Interval cannot be "shuffled" https://pharo.fogbugz.com/f/cases/17233 17179 Semantic analysis doesnt compute RBArgumentNode instances https://pharo.fogbugz.com/f/cases/17179 http://files.pharo.org/image/50/50496.zip

Dec. 14, 2015

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/50496 Home: https://github.com/pharo-project/pharo-core

Dec. 14, 2015

New web tutorial

by stepharo

Hi guys with Olivier Auverlot we did several attempts (over the year) and we are happy to release a first draft of a web tutorial around Pharo. We ask the user to build a simple blog engine. It goes over - Mongo - Seaside - Magritte https://ci.inria.fr/pharo-contribution/job/TinyBlogTutorial/5/artifact/book… Soon Rest/Deployment/Garage?/NeoJSON/XML If you want to participate (translate it to english) or improve it you are welcome. We hope that it will serve as a basis to a lot of Seaside applications. This tutorial will be used in the lecture I'm giving next weeks in Togo and will be edited as a lulu book. Stef and Olivier.

Dec. 13, 2015

Interval cannot be "shuffled"

by Alexandre Bergel

Hi! Type: (1 to: 20) shuffled and an error is raised. I have fixed this. Check: https://pharo.fogbugz.com/f/cases/17233/Interval-cannot-be-shuffled Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 13, 2015

Re: [Pharo-dev] Cog Rump Xen unikernel

by Ben Coman

There were a couple of minor log errors below, so I attach a script that does it all, tested on 32-bit Debian Jessie. cheers -ben On Sat, Dec 12, 2015 at 2:57 PM, Ben Coman <btc(a)openinworld.com> wrote: > Here is the recipe for my first successful build and run through to an error... > StackVM.unikernel: could not find any display driver > > This is done under 32-bit Debian 8 Jessie. > Attached is StackRumpXenUnikernelConfig.st > > # Built the Rumprun Platform > > $ mkdir -p cogunikernel > $ git clone http://repo.rumpkernel.org/rumprun > $ cd rumprun > $ git submodule update --init > $ CC=cc ./build-rr.sh xen >>> toolchain tuple: i486-rumprun-netbsdelf >>> cc wrapper: i486-rumprun-netbsdelf-gcc >>> installed to "/home/ben/Repos/cogunikernel/rumprun/./rumprun" > > > # Generated Cog VM sources > > $ cd ../../cogunikernel > $ git clone https://github.com/pharo-project/pharo-vm.git > $ cd pharo-vm/image > $ ./newImage.sh > $ ./pharo-ui generator.image > Tools > File Browser > Filed in StackRumpXenUnikernelConfig.st > From class comment, evaluated... > StackRumpXenUnikernelConfig new > generateSources; generate. > > > # Built the StackVM and StackVM.unikernel > > $ bash build.sh > > > # Tested them... > > $ cd ../results > $ ./StackVM > > bash: ./StackVM: cannot execute binary file: Exec format error > Good. That was the expected result. > > $ rumprun xen -i StackVM.unikernel > > > StackVM.unikernel: could not find any display driver > > > # Stop unikernel machine running, from another terminal... > $ xl list > $ xl destroy rumprun-StackVM.unikernel > > ================ > > Now I basically just poked, cut and slashed the config until it could > compile and boot up the unikernel, without regard for what that may > break. Making the VM *actually* work is bigger step that for now is > beyond me, but I'll have a bite at it. Really my motivation in doing > this after I stumbled on the rumprun tutorial, was I wanted to crack > the seal on it to interest others capable :). I'd be interested if > the above recipe is successful for others. > > Todo: > * incorporate rumprun as an auto-built third party library, rather > than outside, perhaps using git submodule like the rump kernel already > does (if these can be nested) > * determine how to deal with different rumprun toolchains (depending > on which platform the target) > * move the baking step from the build script to the generated CMake > configuration > * determine how to deal with lack of display > * determine how an image file might be passed to the VM - maybe baking > the VM and image into a ram disk (?) > > cheers -ben

Dec. 13, 2015

Re: [Pharo-dev] Unicode Support

by stepharo

> I am pretty sure that this whole discussion does more harm than good for most people's understanding of Unicode. > > It is best and (mostly) correct to think of a Unicode string as a sequence of Unicode characters, each defined/identified by a code point (out of 10.000s covering all languages). That is what we have today in Pharo (with the distinction between ByteString and WideString as mostly invisible implementation details). > > To encode Unicode for external representation as bytes, we use UTF-8 like the rest of the modern world. > > So far, so good. > > Why all the confusion ? Because the world is a complex place and the Unicode standard tries to cover all possible things. Citing all these exceptions and special cases will make people crazy and give up. I am sure that most stopped reading this thread. like me ;) I will wait for a conclusion with code :) Stef > > Why then is there confusion about the seemingly simple concept of a character ? Because Unicode allows different ways to say the same thing. The simplest example in a common language is (the French letter Ã©) is > > LATIN SMALL LETTER E WITH ACUTE [U+00E9] > > which can also be written as > > LATIN SMALL LETTER E [U+0065] followed by COMBINING ACUTE ACCENT [U+0301] > > The former being a composed normal form, the latter a decomposed normal form. (And yes, it is even much more complicated than that, it goes on for 1000s of pages). > > In the above example, the concept of character/string is indeed fuzzy. > > HTH, > > Sven > > >

Dec. 13, 2015

Re: [Pharo-dev] D language

by Dimitris Chloupis

Ah ok so this is the bridge that OpenQwaq uses to communicate with Python, interesting. I was wondering if I could use this bridge when people mentioned openQwaq to me , I decided not to try because I could not figure out how it worked but now with your link its all clear. With your links you finally answer whether PyBridge would have ben a viable solution for my case. I made a good choice to make my own implementation because reading your code it seems it calls Python as a dynamic library which would not work in my case cause in my case cpython is embeded inside the binary executable of Blender and its impossible to link dynamically to it. Shows what I was talked about in the previous reply, different people, different problems. Oh and I can use 64 bit python , because if I could not it would have been a major issue for me since I deal with big data. Of course that means keeping most of the big data at python side, since the image can easily choke. I really love all these area of cross language coding , it has some fascinating problems and challenges to have your cake and eat it too :) Now I have to figure out a good model of how to reference python data from pharo via dictionaries (python already uses dicitionaries for all its reference values/objects) and how to keep those two sides in sync. On Sun, Dec 13, 2015 at 2:47 PM phil(a)highoctane.be <phil(a)highoctane.be> wrote: > This is PyBridge > > > https://github.com/itsmeront/openqwaq/tree/master/qwaqvm/src/plugins/PyBrid… > > https://github.com/itsmeront/openqwaq/blob/master/qwaqvm/platforms/Cross/pl… > > Useful for sure, even in 32 bits. > > Phil > > On Sun, Dec 13, 2015 at 1:07 PM, Dimitris Chloupis <kilon.alios(a)gmail.com> > wrote: > >> Fair enough >> >> Whats that PyBridge you talking about ? First time I hear that. >> >> Well my own python bridge works well for what I need it for, its not that >> hard to bridge languages at least for some form of basic RPC. Deeper >> language integration would need money and multiple developers of course. >> >> As always the hard part is to make something that works not only for your >> own personal needs but for the needs of many other users. So many diffirent >> designs have to applied and planned ahead. >> >> On Sun, Dec 13, 2015 at 1:54 PM phil(a)highoctane.be <phil(a)highoctane.be> >> wrote: >> >>> A a C++ developer, I think I can better relate to D. >>> >>> Yeah, Nim is nice. >>> >>> I asked about the PyBridge as well (as there is in OpenQwaq|Terf) but no >>> response from Ron so far. >>> >>> We need more bridges, that's my long term play. >>> >>> Some decent cash is coming my way, so I hope that I'll be able to devote >>> more time to these projects. >>> That's always the same story, cash buys time... >>> >>> Phil >>> >>> >>> On Sun, Dec 13, 2015 at 11:34 AM, Dimitris Chloupis < >>> kilon.alios(a)gmail.com> wrote: >>> >>>> From a basic search it appears that D can generate dynamically linked >>>> libraries (DLL , SO, etc) which will be the easiest way to extend pharo >>>> using D. >>>> >>>> However from a very basic search I did it looks like generating >>>> dynamically linked libraries with D is not as straightforward as someone >>>> would expect. >>>> >>>> http://forum.dlang.org/thread/k3vfm9$1tq$1@digitalmars.com >>>> >>>> If you are not bound to D for creating system-level extensions for >>>> Pharo you have several options >>>> >>>> 1) nim language is used by gokr and his company and they use it to >>>> extend Squeak >>>> >>>> here is a blog post about what nim is >>>> >>>> http://hookrace.net/blog/what-is-special-about-nim/ >>>> >>>> and here is a blog post of gokr about how to get it to work with squeak >>>> and pharo >>>> >>>> http://goran.krampe.se/2014/11/03/squeak-to-nim/ >>>> >>>> 2) Maybe just maybe you could use Slang , which is basically smalltalk >>>> that compiles to C and is what the pharo VM uses a lot. But I am not >>>> certain. >>>> >>>> PS: Just realised you commented on that blog post so you may be aware >>>> of all this, but you can take this as a friendly reminder of your potential >>>> solutions :) >>>> >>>> >>>> >>>> On Sun, Dec 13, 2015 at 2:16 AM phil(a)highoctane.be <phil(a)highoctane.be> >>>> wrote: >>>> >>>>> Anyone having used D to build system-level extensions for Pharo? >>>>> >>>>> TIA, >>>>> Phil >>>>> >>>> >>> >

Dec. 13, 2015

Re: [Pharo-dev] D language

by phil＠highoctane.be

This is PyBridge https://github.com/itsmeront/openqwaq/tree/master/qwaqvm/src/plugins/PyBrid… https://github.com/itsmeront/openqwaq/blob/master/qwaqvm/platforms/Cross/pl… Useful for sure, even in 32 bits. Phil On Sun, Dec 13, 2015 at 1:07 PM, Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > Fair enough > > Whats that PyBridge you talking about ? First time I hear that. > > Well my own python bridge works well for what I need it for, its not that > hard to bridge languages at least for some form of basic RPC. Deeper > language integration would need money and multiple developers of course. > > As always the hard part is to make something that works not only for your > own personal needs but for the needs of many other users. So many diffirent > designs have to applied and planned ahead. > > On Sun, Dec 13, 2015 at 1:54 PM phil(a)highoctane.be <phil(a)highoctane.be> > wrote: > >> A a C++ developer, I think I can better relate to D. >> >> Yeah, Nim is nice. >> >> I asked about the PyBridge as well (as there is in OpenQwaq|Terf) but no >> response from Ron so far. >> >> We need more bridges, that's my long term play. >> >> Some decent cash is coming my way, so I hope that I'll be able to devote >> more time to these projects. >> That's always the same story, cash buys time... >> >> Phil >> >> >> On Sun, Dec 13, 2015 at 11:34 AM, Dimitris Chloupis < >> kilon.alios(a)gmail.com> wrote: >> >>> From a basic search it appears that D can generate dynamically linked >>> libraries (DLL , SO, etc) which will be the easiest way to extend pharo >>> using D. >>> >>> However from a very basic search I did it looks like generating >>> dynamically linked libraries with D is not as straightforward as someone >>> would expect. >>> >>> http://forum.dlang.org/thread/k3vfm9$1tq$1@digitalmars.com >>> >>> If you are not bound to D for creating system-level extensions for Pharo >>> you have several options >>> >>> 1) nim language is used by gokr and his company and they use it to >>> extend Squeak >>> >>> here is a blog post about what nim is >>> >>> http://hookrace.net/blog/what-is-special-about-nim/ >>> >>> and here is a blog post of gokr about how to get it to work with squeak >>> and pharo >>> >>> http://goran.krampe.se/2014/11/03/squeak-to-nim/ >>> >>> 2) Maybe just maybe you could use Slang , which is basically smalltalk >>> that compiles to C and is what the pharo VM uses a lot. But I am not >>> certain. >>> >>> PS: Just realised you commented on that blog post so you may be aware of >>> all this, but you can take this as a friendly reminder of your potential >>> solutions :) >>> >>> >>> >>> On Sun, Dec 13, 2015 at 2:16 AM phil(a)highoctane.be <phil(a)highoctane.be> >>> wrote: >>> >>>> Anyone having used D to build system-level extensions for Pharo? >>>> >>>> TIA, >>>> Phil >>>> >>> >>

Dec. 13, 2015