December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Preview of new FFI?

by Esteban Lorenzano

as Torsten said, is in the image, and I cannot provide support until next weekâ¦ Iâm counting the bytes here, to send mails :) but after that, of course I will provide support, documentation (Iâm already working on it), etc. Esteban > On 23 Dec 2015, at 12:16, Pierce Ng <pierce(a)samadhiweb.com> wrote: > > Hi, > > (For Esteban perhaps...) > > Is there an image/VM combo to preview the new FFI? I just made NBSQLite3 work > with SQLcipher (for transparent database encryption) but don't want to spend > any effort refactoring if I need to redo for the new FFI. > > Pierce >

Dec. 23, 2015

AgileVisualization - call for reviewers

by Alexandre Bergel

Dear All, As you may have heard, AgileVisualization is a book about the Roassal visualization engine. We hope to have the book released soon, within a couple of months. It would be fantastic to have feedback: http://agilevisualization.com The book is a fantastic result of the community. You guys made this possible, and we wish to thank you very much for this. Beers will flow at ESUG for everybody who comment on the chapters :-) We also have live discussion on Slack, channel #Roassal (see http://pharo.org/community on how to join Slack) Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Dec. 23, 2015

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo Consortium Sponsored Development Effort

by phil＠highoctane.be

On Wed, Dec 23, 2015 at 1:54 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > >>> >>> >>>> Also I read [j] that a problem with popen is that you do not get pid >>>> of child so you cannot wait on command to complete. Could posix_spawn >>>> be preferrable to popen? >>>> >>>> >>> Exactly. There is no standard way to get child pid from popen() so you >>> can't do polling in image side to wait it's completion. >>> So yeah, posix_spawn() does fix this issue as well as many many other. >>> >> >> >> Looks nice and at the same time, popen support is useful/wanted. >> > > Yes, but I wonder if it would still make sense if I finish the posix_spawn > flavor. Anyway, we can decide later since the Popen wrapper is almost > written (a very very first prototype) > > >> A generic way to deal with pids (in Linux at least, is to have the >> process write its pid in /var/run/ or a subdir of it. >> > > That's right, but that only works for a certain number of cases, and > polling with checking files does not seem good. > Lots of software already works that way. In the name of proper integration of external processes/"things", it is a must have. Well, I need it in production. > I think that you if you expect to run a a command that takes some time, > then use the posix_spawn wrapper and use popen only in "blocking" mode. > > >> >> That's how monit checks for processes for example. >> >> http://unix.stackexchange.com/questions/12815/what-are-pid-and-lock-files-f… >> >> We need as many options as we can here. >> > > I not fully convinced with that statement. > Anything making Pharo less of an island is a step in the right direction. popen is how it is done in one of the most successful platforms. I am using it too, so, being able to port knowledge from point A to B is useful in my view. https://docs.python.org/2/library/subprocess.html You'll see spawn and popen. And Popen2,3, 4. > > >> Mariano, I added you to a Trello board that captured a couple of items >> about the topic you deal with. >> https://trello.com/b/Hji4F0Zp/clipharo >> >> > Thanks! > > >> We also have a card that says the following: >> >> Pharo follows Smalltalk which uses a global object named Smalltalk which >> in fact is an instance of SmalltalkImage, confusing if you ask me. >> >> I would add a simple class named System which have class-side accessors >> for static global things like >> > > My answers below. Just as a matter of how that System class could be > written: > > >> >> - image => links to the old Smalltalk >> >> > Well..we have now "Smalltalk image" > > >> - stdin >> - stdout >> - session >> >> > OK about those. > > >> >> - environment for ENV vars >> >> > we have "Smalltalk os environment" > > >> >> - workingDirectory accessing the process working directory $PWD >> >> > We have "FileSystem workingDirectory" > Look above, one has to peek all around to find them (not to mention being aware. And I'd even say, foreget FileSystem, why not FileLocator, which is meant for that). FWIW, https://nodejs.org/api/os.html > > >> >> - 'pid' >> >> > Well, we need OSProcess right now, but it is also very very easy to do via > FFI. > > > >> That's a very easy thing to do, but IMO it will help anybody coming from >> outside smalltalk to easily find these things. Probably there are more >> things that would fit here. >> >> I'd like to have that. >> >> The discussion involved Camillo who was very into these topics. His >> Pinocchio work seemed to have contributions that can be useful of the CLI >> support for Pharo. >> >> Phil >> >> >>> >>> >>>> and just btw, mildly interesting is using sockets rather than pipes >>>> for interprocess communication [l] >>>> >>>> [j] >>>> http://stackoverflow.com/questions/3884103/can-popen-make-bidirectional-pip… >>>> [k] http://stackoverflow.com/questions/6743771/popen-alternative >>>> [l] >>>> http://stackoverflow.com/questions/25161377/open-a-cmd-program-with-full-fu… >>>> >>>> >>> Thanks. I will check that on a second step ;) >>> >>> >>>> > >>>> > 2) We believe that there is a large number of OSProcess users that use >>>> > OSProcess only for executing OS commands AND popen() approach would >>>> be more >>>> > than enough (we will do a survey about this). So, a very simply >>>> solution via >>>> > FFI for these users rather than having all OSProcess, Command Shell, >>>> the >>>> > plugins etc, may be worth. >>>> > >>>> > What do you think? >>>> > >>>> >> >>>> >> Now, given some of the difficulties seen when dealing with external >>>> >> processes, I'd be wary of the "async" side of things. Random failures >>>> >> seemingly linked to signal loss are certainly not something I'd like >>>> to >>>> >> debug. >>>> >> >>>> > >>>> > Indeed, it won't be easy I guess. But sometimes the process you >>>> execute may >>>> > take quite some time and blocking the VM for that period is not a good >>>> > choice. So... what would you prefer to solve that? Do you prefer >>>> image-based >>>> > pooling as OSProcess does when IOPlugin is not used? >>>> >>>> >>>> How much might your work overlap with the Threaded FFI project for >>>> Cog?... >>>> ..."The Cog VM is single-threaded, providing green threads to the >>>> Smalltalk level, as do most Smalltalk VMs. There is a prototype >>>> subclass of the CoInterpreter, CoInterpreterMT, which provides a >>>> Python-like multi-threaded VM where any number of threads can share >>>> the VM with only one running the VM at any one time. This uses an >>>> extremely simple low-level scheme for releasing and re-acquiring >>>> ownership of the VM designed by David Simmons. This results in any >>>> and all FFI calls being non-blocking since there is such a low >>>> overhead to making a non-blocking FFI call that all FFI calls can be >>>> non-blocking. Eliot has worked on this, implementing the prototype >>>> and the reentrant FFI plugin it requires. But this needs to be >>>> revived. It would be a really powerful enhancement." >>>> >>>> >>> Well, Eliot said I may need to give a hand to them in order to finish >>> the Threaded FFI. However, I am still not 100% certain if the threaded FFI >>> calls to read from pipes would be worth. Eliot said that yes. Anyway, once >>> I have the first version working I will digest this and goes to the next >>> level and see how I can make things better than polling, if polling proves >>> to be a problem. >>> >>> >>> >>>> cheers -ben >>>> >>>> P.S. Just another article warning against mixing fork and >>>> multiple-threads (which it seems posix_spawn may avoid)... >>>> >>>> http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-usi… >>>> >>>> >>> Thanks for the reference. >>> >>> Cheers, >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 23, 2015

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo Consortium Sponsored Development Effort

by Ben Coman

On Wed, Dec 23, 2015 at 7:46 PM, phil(a)highoctane.be <phil(a)highoctane.be> wrote: > > Pharo follows Smalltalk which uses a global object named Smalltalk which in > fact is an instance of SmalltalkImage, confusing if you ask me. I also have felt that same impedance mismatch in the naming, but I thought the accessors were deliberately moved to the instance side to help (maybe later) to minimise global state. So I am guessing that adding class-side accessors is a step backwards - but maybe System class could provide a singleton instance of itself. cheers -ben > > I would add a simple class named System which have class-side accessors for > static global things like > > image => links to the old Smalltalk > stdin > stdout > session > environment for ENV vars > workingDirectory accessing the process working directory $PWD > 'pid' > > That's a very easy thing to do, but IMO it will help anybody coming from > outside smalltalk to easily find these things. Probably there are more > things that would fit here. > > > I'd like to have that. > > The discussion involved Camillo who was very into these topics. His > Pinocchio work seemed to have contributions that can be useful of the CLI > support for Pharo. > > Phil > >> >> >>> >>> and just btw, mildly interesting is using sockets rather than pipes >>> for interprocess communication [l] >>> >>> [j] >>> http://stackoverflow.com/questions/3884103/can-popen-make-bidirectional-pip… >>> [k] http://stackoverflow.com/questions/6743771/popen-alternative >>> [l] >>> http://stackoverflow.com/questions/25161377/open-a-cmd-program-with-full-fu… >>> >> >> Thanks. I will check that on a second step ;) >> >>> >>> > >>> > 2) We believe that there is a large number of OSProcess users that use >>> > OSProcess only for executing OS commands AND popen() approach would be >>> > more >>> > than enough (we will do a survey about this). So, a very simply >>> > solution via >>> > FFI for these users rather than having all OSProcess, Command Shell, >>> > the >>> > plugins etc, may be worth. >>> > >>> > What do you think? >>> > >>> >> >>> >> Now, given some of the difficulties seen when dealing with external >>> >> processes, I'd be wary of the "async" side of things. Random failures >>> >> seemingly linked to signal loss are certainly not something I'd like >>> >> to >>> >> debug. >>> >> >>> > >>> > Indeed, it won't be easy I guess. But sometimes the process you execute >>> > may >>> > take quite some time and blocking the VM for that period is not a good >>> > choice. So... what would you prefer to solve that? Do you prefer >>> > image-based >>> > pooling as OSProcess does when IOPlugin is not used? >>> >>> >>> How much might your work overlap with the Threaded FFI project for >>> Cog?... >>> ..."The Cog VM is single-threaded, providing green threads to the >>> Smalltalk level, as do most Smalltalk VMs. There is a prototype >>> subclass of the CoInterpreter, CoInterpreterMT, which provides a >>> Python-like multi-threaded VM where any number of threads can share >>> the VM with only one running the VM at any one time. This uses an >>> extremely simple low-level scheme for releasing and re-acquiring >>> ownership of the VM designed by David Simmons. This results in any >>> and all FFI calls being non-blocking since there is such a low >>> overhead to making a non-blocking FFI call that all FFI calls can be >>> non-blocking. Eliot has worked on this, implementing the prototype >>> and the reentrant FFI plugin it requires. But this needs to be >>> revived. It would be a really powerful enhancement." >>> >> >> Well, Eliot said I may need to give a hand to them in order to finish the >> Threaded FFI. However, I am still not 100% certain if the threaded FFI calls >> to read from pipes would be worth. Eliot said that yes. Anyway, once I have >> the first version working I will digest this and goes to the next level and >> see how I can make things better than polling, if polling proves to be a >> problem. >> >> >>> >>> cheers -ben >>> >>> P.S. Just another article warning against mixing fork and >>> multiple-threads (which it seems posix_spawn may avoid)... >>> >>> http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-usi… >>> >> >> Thanks for the reference. >> >> Cheers, >> >> -- >> Mariano >> http://marianopeck.wordpress.com > >

Dec. 23, 2015

Re: [Pharo-dev] Preview of new FFI?

by Torsten Bergmann

I guess Esteban is busy as he is with Stef in Togo. I started porting to new FFI initial stuff and chatted with Esteban on Slack. When you use the latest Pharo 5 image (together with the spur VM) you have the new FFI right inside. Athens is running already on the new FFI. NB is still not unloaded - but it will be unloaded soon. The idea is that people can recover what they need afterwards. For some cases it should be straightforward to convert. A short summary of my findings: - check the "FFI-NB" package in recent Pharo 5 images like Pharo 5 #50510 as of today The name is misleading - it is not the NativeBoost but the new FFI - check the new classes in this package that are prefixed with "FFI" like FFIExternalObject (replacement for NBExternalObject), FFIExternalStructure (replacement for NBExternalStructure), ... - If you generate fields from a struct the accessors are not generated into non-package "*generated-code-non-existing-package" but are now generated into method category "accessing structure variables" and will therefore be part of your MCZ code package - #nbCall: and #nbCall:module: work for compatibility, I guess we should add a deprecation to them - it is better to use ffiCall:, ffiCall:module: now - there is no need to have a pragma with NativeBoostPlugin anymore, at least it worked for me without it just by using #ffiCall: ... - there is a new repo: http://smalltalkhub.com/#!/~Pharo/FFI-NB since last week, but I guess the code in the image is more recent Problems I've experienced so far while trying to port my OS-Windows project: - FIICallout does not understand #optionAt:, no support for options like #WinUnicode so far - FFIExternalType sizeOf: #ulong failed - the spur VM for Windows does not include all latest fixes for Windows VM, like CTRL keys (https://pharo.fogbugz.com/f/cases/17289) Hope this helps a little bit until some more bugs are sorted out/a more detailed info for conversion is available. Bye T.Â Gesendet:Â Mittwoch, 23. Dezember 2015 um 13:55 Uhr Von:Â "Mariano Martinez Peck" <marianopeck(a)gmail.com> An:Â "Pharo Development List" <pharo-dev(a)lists.pharo.org> Betreff:Â Re: [Pharo-dev] Preview of new FFI? We are in the same boat. Even worst, I suspect I am using some parts of ASMJIT which I guess won't be in new FFI, so I have on clue how I would make that parts to work....Â Â On Wed, Dec 23, 2015 at 9:16 AM, Pierce Ng <pierce(a)samadhiweb.com> wrote:Hi, (For Esteban perhaps...) Is there an image/VM combo to preview the new FFI? I just made NBSQLite3 work with SQLcipher (for transparent database encryption) but don't want to spend any effort refactoring if I need to redo for the new FFI. Pierce Â Â Â -- Mariano http://marianopeck.wordpress.com[http://marianopeck.wordpress.com]

Dec. 23, 2015

Re: [Pharo-dev] Preview of new FFI?

by Mariano Martinez Peck

We are in the same boat. Even worst, I suspect I am using some parts of ASMJIT which I guess won't be in new FFI, so I have on clue how I would make that parts to work.... On Wed, Dec 23, 2015 at 9:16 AM, Pierce Ng <pierce(a)samadhiweb.com> wrote: > Hi, > > (For Esteban perhaps...) > > Is there an image/VM combo to preview the new FFI? I just made NBSQLite3 > work > with SQLcipher (for transparent database encryption) but don't want to > spend > any effort refactoring if I need to redo for the new FFI. > > Pierce > > -- Mariano http://marianopeck.wordpress.com

Dec. 23, 2015

Re: [Pharo-dev] [Pharo-users] [ANN] Pharo Consortium Sponsored Development Effort

by Mariano Martinez Peck

> > >> >> >>> Also I read [j] that a problem with popen is that you do not get pid >>> of child so you cannot wait on command to complete. Could posix_spawn >>> be preferrable to popen? >>> >>> >> Exactly. There is no standard way to get child pid from popen() so you >> can't do polling in image side to wait it's completion. >> So yeah, posix_spawn() does fix this issue as well as many many other. >> > > > Looks nice and at the same time, popen support is useful/wanted. > Yes, but I wonder if it would still make sense if I finish the posix_spawn flavor. Anyway, we can decide later since the Popen wrapper is almost written (a very very first prototype) > A generic way to deal with pids (in Linux at least, is to have the process > write its pid in /var/run/ or a subdir of it. > That's right, but that only works for a certain number of cases, and polling with checking files does not seem good. I think that you if you expect to run a a command that takes some time, then use the posix_spawn wrapper and use popen only in "blocking" mode. > > That's how monit checks for processes for example. > > http://unix.stackexchange.com/questions/12815/what-are-pid-and-lock-files-f… > > We need as many options as we can here. > I not fully convinced with that statement. > Mariano, I added you to a Trello board that captured a couple of items > about the topic you deal with. > https://trello.com/b/Hji4F0Zp/clipharo > > Thanks! > We also have a card that says the following: > > Pharo follows Smalltalk which uses a global object named Smalltalk which > in fact is an instance of SmalltalkImage, confusing if you ask me. > > I would add a simple class named System which have class-side accessors > for static global things like > My answers below. Just as a matter of how that System class could be written: > > - image => links to the old Smalltalk > > Well..we have now "Smalltalk image" > - stdin > - stdout > - session > > OK about those. > > - environment for ENV vars > > we have "Smalltalk os environment" > > - workingDirectory accessing the process working directory $PWD > > We have "FileSystem workingDirectory" > > - 'pid' > > Well, we need OSProcess right now, but it is also very very easy to do via FFI. > That's a very easy thing to do, but IMO it will help anybody coming from > outside smalltalk to easily find these things. Probably there are more > things that would fit here. > > I'd like to have that. > > The discussion involved Camillo who was very into these topics. His > Pinocchio work seemed to have contributions that can be useful of the CLI > support for Pharo. > > Phil > > >> >> >>> and just btw, mildly interesting is using sockets rather than pipes >>> for interprocess communication [l] >>> >>> [j] >>> http://stackoverflow.com/questions/3884103/can-popen-make-bidirectional-pip… >>> [k] http://stackoverflow.com/questions/6743771/popen-alternative >>> [l] >>> http://stackoverflow.com/questions/25161377/open-a-cmd-program-with-full-fu… >>> >>> >> Thanks. I will check that on a second step ;) >> >> >>> > >>> > 2) We believe that there is a large number of OSProcess users that use >>> > OSProcess only for executing OS commands AND popen() approach would be >>> more >>> > than enough (we will do a survey about this). So, a very simply >>> solution via >>> > FFI for these users rather than having all OSProcess, Command Shell, >>> the >>> > plugins etc, may be worth. >>> > >>> > What do you think? >>> > >>> >> >>> >> Now, given some of the difficulties seen when dealing with external >>> >> processes, I'd be wary of the "async" side of things. Random failures >>> >> seemingly linked to signal loss are certainly not something I'd like >>> to >>> >> debug. >>> >> >>> > >>> > Indeed, it won't be easy I guess. But sometimes the process you >>> execute may >>> > take quite some time and blocking the VM for that period is not a good >>> > choice. So... what would you prefer to solve that? Do you prefer >>> image-based >>> > pooling as OSProcess does when IOPlugin is not used? >>> >>> >>> How much might your work overlap with the Threaded FFI project for >>> Cog?... >>> ..."The Cog VM is single-threaded, providing green threads to the >>> Smalltalk level, as do most Smalltalk VMs. There is a prototype >>> subclass of the CoInterpreter, CoInterpreterMT, which provides a >>> Python-like multi-threaded VM where any number of threads can share >>> the VM with only one running the VM at any one time. This uses an >>> extremely simple low-level scheme for releasing and re-acquiring >>> ownership of the VM designed by David Simmons. This results in any >>> and all FFI calls being non-blocking since there is such a low >>> overhead to making a non-blocking FFI call that all FFI calls can be >>> non-blocking. Eliot has worked on this, implementing the prototype >>> and the reentrant FFI plugin it requires. But this needs to be >>> revived. It would be a really powerful enhancement." >>> >>> >> Well, Eliot said I may need to give a hand to them in order to finish the >> Threaded FFI. However, I am still not 100% certain if the threaded FFI >> calls to read from pipes would be worth. Eliot said that yes. Anyway, once >> I have the first version working I will digest this and goes to the next >> level and see how I can make things better than polling, if polling proves >> to be a problem. >> >> >> >>> cheers -ben >>> >>> P.S. Just another article warning against mixing fork and >>> multiple-threads (which it seems posix_spawn may avoid)... >>> >>> http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-usi… >>> >>> >> Thanks for the reference. >> >> Cheers, >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> > > -- Mariano http://marianopeck.wordpress.com

Dec. 23, 2015

A Suggested Promotion

by Torsten Bergmann

Jecel Assumpcao Jr. wrote on squeak-dev: >>In fact, it could be an extremely stripped down Linux focused exclusively >>on running Squeak (like Dan Ingalls did for his weather station). One >>advantage of that is that the boot time would be much smaller than what >>people are used to. John Hylands wrote on squeak-dev >My brother and I have been experimenting with Tiny Core Linux, which is >available on the Pi as piCore. > >It boots in 12 seconds, from power on to login prompt, on a Pi 2. That >gives you wifi plus an openssh server. It would probably add a few seconds >to get a graphical shell running, which it also includes. Note that for Pharo there is a project from Mike Filonov (owner of pharocloud.com) which was initially called "PharoNOS". It is also based on TinyCore Linux, here x86 based directly booting into Pharo. A description and an downloadable ISO is available on [1]. Runs fine on VirtualBox. >From the name often people thought it was a project similar to "SqueakNOS" (but SqueakNOS boots into Squeak and needed no Linux or other underneath as even the drivers were written in Smalltalk). I guess therefore Mike renamed the project to "Pharocloud OS" now, he also made the build scripts available on [2]. I also did some experiments with Pharo on the Pi - described in an article on [3]. The result was similar: directly booting into the Pharo environment/ a Smalltalk application - but on the Pi. There is Raspbian underneath. A video is available on [4]. However: if there is now some interest in having a fast and direct Squeak/Pharo booting on the Pi maybe this could be a shared effort between both communities as well. Beside squeak-dev I therefore CC the pharo-dev mailinglist as well. Bye T. [1] http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos [2] http://os.pharocloud.com [3] https://medium.com/concerning-pharo/pharo-pi-9eef257b6a21#.qgqg6765p [4] https://www.youtube.com/watch?v=sfxFqQIuawg

Dec. 23, 2015

Preview of new FFI?

by Pierce Ng

Hi, (For Esteban perhaps...) Is there an image/VM combo to preview the new FFI? I just made NBSQLite3 work with SQLcipher (for transparent database encryption) but don't want to spend any effort refactoring if I need to redo for the new FFI. Pierce

Dec. 23, 2015

[JOB][OFFTOPIC] Mobile Developer: iOS.

by marcus.denker＠inria.fr

(a bit off-topic, 2Denker GmbH, a member of the Pharo Consortium, is looking to hire an iOS developer) 2Denker GmbH is looking for a full time iOS developer to strengthen our international team of mobile developers. - Permanent job - International Team - Cologne! With Carneval! 2Denker GmbH is a small company with offices in the center of Cologne (close to Neumarkt square). We work in small teams on software for mobile devices (fronted / backend). The work consists both in maintenance (and further development) of existing applications as well as developing new projects from scratch. We are looking for a developer to join the team locally in Cologne, to be hired on a permanent contract. - iOS programming - UI Design - Location based services - Speaking german is a plus but not required Company: 2denker GmbH Contact: Herr Norbert Hartl Where: Reinoldstr. 6 50676 Koeln Phone: +49 221 27225945 E-Mail: mail(a)2denker.de Website: http://2denker.de (english website is quite out of date...)

Dec. 23, 2015