On Wed, Dec 23, 2015 at 1:54 PM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

��
Also I read [j] that a problem with popen is that you do not get pid
of child so you cannot wait on command to complete.�� Could posix_spawn
be preferrable to popen?


Exactly. There is no standard way to get child pid from popen() so you can't do polling in image side to wait it's completion.��
So yeah, posix_spawn() does fix this issue as well as many many other.��


Looks nice and at the same time, popen support is useful/wanted.

Yes, but I wonder if it would still make sense if I finish the posix_spawn flavor. Anyway, we can decide later since the Popen wrapper is almost written (a very very first prototype)
��
A generic way to deal with pids (in Linux at least, is to have the process write its pid in /var/run/ or a subdir of it.

That's right, but that only works for a certain number of cases, and polling with checking files does not seem good.

Lots of software already works that way. In the name of proper integration of external processes/"things", it is a must have.
Well, I need it in production.
��
I think that you if you expect to run a a command that takes some time, then use the posix_spawn wrapper and use popen only in "blocking" mode. ��
��

That's how monit checks for processes for example.��

We need as many options as we can here.��

I not fully convinced with that statement.��

Anything making Pharo less of an island is a step in the right direction. popen is how it is done in one of the most successful platforms. I am using it too, so, being able to port knowledge from point A to B is useful in my view.��https://docs.python.org/2/library/subprocess.html You'll see spawn and popen. And Popen2,3, 4.
��
��
Mariano, I added you to a Trello board that captured a couple of items about the topic you deal with.


Thanks!
��
We also have a card that says the following:

Pharo follows Smalltalk which uses a global object named��Smalltalk��which in fact is an instance of��SmalltalkImage, confusing if you ask me.

I would add a simple class named��System��which have class-side accessors for static global things like


My answers below. Just as a matter of how that System class could be written:
��
  • image��=> links to the old Smalltalk

Well..we have now "Smalltalk image"��

  • stdin
  • stdout
  • session

OK about those.��
��
  • environment��for ENV vars

we have "Smalltalk os environment"
��
  • workingDirectory��accessing the process working directory��$PWD

We have "FileSystem workingDirectory"

Look above, one has to peek all around to find them (not to mention being aware. And I'd even say, foreget FileSystem, why not FileLocator, which is meant for that).

FWIW,��https://nodejs.org/api/os.html��
��
��
  • 'pid'


Well, we need OSProcess right now, but it is also very very easy to do via FFI.

��

That's a very easy thing to do, but IMO it will help anybody coming from outside smalltalk to easily find these things. Probably there are more things that would fit here.


I'd like to have that.

The discussion involved Camillo who was very into these topics. His Pinocchio work seemed to have contributions that can be useful of the CLI support for Pharo.

Phil
��
��
and just btw, mildly interesting is using sockets rather than pipes
for interprocess communication [l]

[j] http://stackoverflow.com/questions/3884103/can-popen-make-bidirectional-pipes-like-pipe-fork
[k] http://stackoverflow.com/questions/6743771/popen-alternative
[l] http://stackoverflow.com/questions/25161377/open-a-cmd-program-with-full-functionality-i-o/25177958#25177958


Thanks. I will check that on a second step ;)
��
>
> 2) We believe that there is a large number of OSProcess users that use
> OSProcess only for executing OS commands AND popen() approach would be more
> than enough (we will do a survey about this). So, a very simply solution via
> FFI for these users rather than having all OSProcess, Command Shell, the
> plugins etc, may be worth.
>
> What do you think?
>
>>
>> Now, given some of the difficulties seen when dealing with external
>> processes, I'd be wary of the "async" side of things. Random failures
>> seemingly linked to signal loss are certainly not something I'd like to
>> debug.
>>
>
> Indeed, it won't be easy I guess. But sometimes the process you execute may
> take quite some time and blocking the VM for that period is not a good
> choice. So... what would you prefer to solve that? Do you prefer image-based
> pooling as OSProcess does when IOPlugin is not used?


How much might your work overlap with the Threaded FFI project for Cog?...
..."The Cog VM is single-threaded, providing green threads to the
Smalltalk level, as do most Smalltalk VMs. There is a prototype
subclass of the CoInterpreter, CoInterpreterMT, which provides a
Python-like multi-threaded VM where any number of threads can share
the VM with only one running the VM at any one time.�� This uses an
extremely simple low-level scheme for releasing and re-acquiring
ownership of the VM designed by David Simmons.�� This results in any
and all FFI calls being non-blocking since there is such a low
overhead to making a non-blocking FFI call that all FFI calls can be
non-blocking.�� Eliot has worked on this, implementing the prototype
and the reentrant FFI plugin it requires.�� But this needs to be
revived.�� It would be a really powerful enhancement."


Well, Eliot said I may need to give a hand to them in order to finish the Threaded FFI. However, I am still not 100% certain if the threaded FFI calls to read from pipes would be worth. Eliot said that yes. Anyway, once I have the first version working I will digest this and goes to the next level and see how I can make things better than polling, if polling proves to be a problem. ��

��
cheers -ben

P.S. Just another article warning against mixing fork and
multiple-threads (which it seems posix_spawn may avoid)...
http://www.linuxprogrammingblog.com/threads-and-fork-think-twice-before-using-them


Thanks for the reference.��

Cheers,��

--




--