[Pharo-users] Pharo and OpenMP

Dimitris Chloupis kilon.alios at gmail.com
Tue Jan 26 12:24:34 EST 2016


First my apology for the very long post but I think this will interest a
lot of people.

I have seen that someone out there .... HELOOOOOOO.... has tried Pharo with
OpenMP

https://pharoweekly.wordpress.com/2015/07/17/pharo-and-openmp/

For those that dont know what the hell OpenMP is, it nothing more than a
collection of C pragmas, not very diffirent to our pharo pragmas , that
tell to a compiler how to make code parallel.

By parallel I don't mean pharo threads which are not really code that
executes at the same time but true hardware threads and taking advantage of
multiple cores etc. A pharo holy grail.

Parallel coding is a bitch, there a lot of logical problems to face when
you have pieces of code executing at the same time and you have to be aware
of how the hardware is doing it because if you dont it so easy to get code
as slow as running it on a single thread.

I have been watching youtube tutorials by one of the guys that made OpenMP
and I have to say I am very impressed. And it got me wondering "why the
hell not pharo". The nice thing about OpenMP is that it does not matter
that pharo VM can do only 1 thread which is the main thread, you can do
multiple threads at C side using OpenMP and then collapse them to one
thread and send the result back to pharo.

Also note OpenMP is engineered by Intel so its a very serious library and
comes included with most modern compilers and that includes GCC. Visual
Studio and Clang / LLVM

On the other hand lately I am also researching shared memory, actually its
shared memory that lead me to OpenMP.   My idea is basically extending the
live coding enviroment of pharo beyond pharo.

Its sound a bit crazy but the way I see it from my very small knowledge of
C and shared memory is that right now, pharo live coding evniroment is
isolated from outside world. You can do live coding while inside pharo but
the moment you go outside , say you call a C library , live coding goes out
of the window. This why we need to have session management to manage
resource of calling C libraries to make sure when we reload the image , the
image does not freak out that the live state of the C libraries is gone.

This is also why many people push things to the image side. And of course
its much more fun to code in Pharo than C.

Now bare with me because things are now getting really interesting....

 what if we could have an image flle for C code ?

The things is that I am interested into shared memory because I want a very
fast IPC, shared memory is the faster. I want to communicate Pharo with
Blender, that I have accomplished via sockets, I can call python code and
python libraries from pharo. But sockets are expensive in large loops, they
work great for simple communication and much faster than pipes when used
locally but try a large loop and socket will lag.

Shared memory on the other hand is super fast, it basically it gives a part
of memory that is shared among process that means also different
applications.

Now how a C image file fits into all this you may ask.

Its simple.

The most popular way of shared memory is called "memory mapped file", what
it does is that it takes a file and loads it to memory as it is... sound
familiar ? ;)

what however is also doing is that it allows you share that part of the
memory with any other application/ process as long as it also tries to load
the file mapped to memory the same way.

So what that means that we can have a place in memory that is shared
between C code, including C dlls, and Pharo code. That file can act as an
image file because we take that shared memory at any time that has been
modified either by pharo, C or whatever else has access to shared memmory
and store it back to that file. This is functionality that memory mapped
files provide called msync.

We talk here about a C image that work at least on surface very similarly
to Pharo image.

Now where OpenMP fits in all this ?

Since OpenMP is about code that eats processors like peanuts , yes we could
call it from a DLL, but with my approach even if that code crashes we will
be able to restore it back to its live state because of that image format.
All we do is reopen the memory mapped file and restore from it the shared
memory. So we end up with true parallel live C/Pharo code with live state
that and of course live code that is shared between Pharo and pretty much
anything that uses C or python or why not other languages.

So far I know how to do all of the above, well at least I am learning it
now.

Now the real challenge, which I have no clue about,  is to make a Pharo
parser to C using OpenMP. My thinking is as following.

Why code in C and OpenMP when you could take something like Slang, which is
basically pharo code that compiles to C, use pharo code with some pharo
pragmas corresponding to OpenMP pragmas that will allow you to compile
Slang OpenMP to C OpenMP and then compile it to an executable or dll that
uses the shared memory memory mapped file solution.

Crazy ? realistic ? far fetched ? maybe ? good in theory back in practice?

Open to your thoughts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-users_lists.pharo.org/attachments/20160126/a8cfa4d9/attachment.html>


More information about the Pharo-users mailing list