[Pharo-users] Pharo and OpenMP

Ben Coman btc at openinworld.com
Sun Jan 31 01:27:21 EST 2016


On Wed, Jan 27, 2016 at 1:24 AM, Dimitris Chloupis
<kilon.alios at gmail.com> wrote:
> First my apology for the very long post but I think this will interest a lot
> of people.
>
> I have seen that someone out there .... HELOOOOOOO.... has tried Pharo with
> OpenMP
>
> https://pharoweekly.wordpress.com/2015/07/17/pharo-and-openmp/
>
> For those that dont know what the hell OpenMP is, it nothing more than a
> collection of C pragmas, not very diffirent to our pharo pragmas , that tell
> to a compiler how to make code parallel.
>
> By parallel I don't mean pharo threads which are not really code that
> executes at the same time but true hardware threads and taking advantage of
> multiple cores etc. A pharo holy grail.
>
> Parallel coding is a bitch, there a lot of logical problems to face when you
> have pieces of code executing at the same time and you have to be aware of
> how the hardware is doing it because if you dont it so easy to get code as
> slow as running it on a single thread.
>
> I have been watching youtube tutorials by one of the guys that made OpenMP
> and I have to say I am very impressed. And it got me wondering "why the hell
> not pharo". The nice thing about OpenMP is that it does not matter that
> pharo VM can do only 1 thread which is the main thread, you can do multiple
> threads at C side using OpenMP and then collapse them to one thread and send
> the result back to pharo.
>
> Also note OpenMP is engineered by Intel so its a very serious library and
> comes included with most modern compilers and that includes GCC. Visual
> Studio and Clang / LLVM
>
> On the other hand lately I am also researching shared memory, actually its
> shared memory that lead me to OpenMP.   My idea is basically extending the
> live coding enviroment of pharo beyond pharo.
>
> Its sound a bit crazy but the way I see it from my very small knowledge of C
> and shared memory is that right now, pharo live coding evniroment is
> isolated from outside world. You can do live coding while inside pharo but
> the moment you go outside , say you call a C library , live coding goes out
> of the window. This why we need to have session management to manage
> resource of calling C libraries to make sure when we reload the image , the
> image does not freak out that the live state of the C libraries is gone.


I guess you won't avoid session management with this idea of memory
mapped C-image, since the external resources provide from the OS like
files, display buffers, network buffers & GPU threads would still be
destroyed by the OS when Pharo exits. Consider the case of moving your
Pharo-image + C-image to another machine.


>
> This is also why many people push things to the image side. And of course
> its much more fun to code in Pharo than C.
>
> Now bear with me because things are now getting really interesting....
>
>  what if we could have an image flle for C code ?
>
> The things is that I am interested into shared memory because I want a very
> fast IPC, shared memory is the faster. I want to communicate Pharo with
> Blender, that I have accomplished via sockets, I can call python code and
> python libraries from pharo. But sockets are expensive in large loops, they
> work great for simple communication and much faster than pipes when used
> locally but try a large loop and socket will lag.
>
> Shared memory on the other hand is super fast, it basically it gives a part
> of memory that is shared among process that means also different
> applications.
>
> Now how a C image file fits into all this you may ask.
>
> Its simple.
>
> The most popular way of shared memory is called "memory mapped file", what
> it does is that it takes a file and loads it to memory as it is... sound
> familiar ? ;)
>
> what however is also doing is that it allows you share that part of the
> memory with any other application/ process as long as it also tries to load
> the file mapped to memory the same way.
>
> So what that means that we can have a place in memory that is shared between
> C code, including C dlls, and Pharo code. That file can act as an image file
> because we take that shared memory at any time that has been modified either
> by pharo, C or whatever else has access to shared memmory and store it back
> to that file. This is functionality that memory mapped files provide called
> msync.
>
> We talk here about a C image that work at least on surface very similarly to
> Pharo image.
>
> Now where OpenMP fits in all this ?
>
> Since OpenMP is about code that eats processors like peanuts , yes we could
> call it from a DLL, but with my approach even if that code crashes we will
> be able to restore it back to its live state because of that image format.
> All we do is reopen the memory mapped file and restore from it the shared
> memory. So we end up with true parallel live C/Pharo code with live state
> that and of course live code that is shared between Pharo and pretty much
> anything that uses C or python or why not other languages.
>
> So far I know how to do all of the above, well at least I am learning it
> now.
>
> Now the real challenge, which I have no clue about,  is to make a Pharo
> parser to C using OpenMP. My thinking is as following.
>
> Why code in C and OpenMP when you could take something like Slang, which is
> basically pharo code that compiles to C,


Just to clarify (for myself also if someone likes to correct me) I
have gathered the idea that Slang should be thought of less as
Smalltalk compiling to C, and more as C with Smalltalk syntax which
works well with our usual tools.  So compiling Slang to C is "trivial"
compared to compiling general Pharo code to C. The trick I guess would
be dynamically generating and compiling the C code and then linking it
back in.

> use pharo code with some pharo pragmas corresponding to
> OpenMP pragmas that will allow you to compile Slang
> OpenMP to C OpenMP and then compile it to an executable or dll that uses the
> shared memory memory mapped file solution.
>
> Crazy ? realistic ? far fetched ? maybe ? good in theory back in practice?
>
> Open to your thoughts.

I think applying some kind of OpenMP pragma to random parts of the
image's code would be quite difficult because it of the interaction
with the VM - but if it was restricted to code that ran detached from
the image (e.g. in a separate native or on a GPU) then that could be
interesting -- but I can't think of how you would marshall parameters
and results between them.

cheers -ben




More information about the Pharo-users mailing list