There are some good C libraries out there, which are suitable to connect programs with each other: a good example is 0MQ. Marten -- Marten Feldtmann

Thanks you for your answers. 0mq looks great, but released with LGPL licence, and I don't like that (I don't understand why there is also a GPL licence file in the distribution ? - clearly a *noway*), I prefer a full smalltalk implementation I can debug when things go wrong. I think latency and timing problems would be the same, no big difference in performance. BTW knowing about this library is cool and may be of some help (who knows), thank you Marten for the reference. I agree totally with you Sven about performance and standards. And I also agree with you about reusing someone else's work ... I don't like to reinvent the wheel, especially a good wheel when mine would be worst, I would if I were a wheel specialist - clearly not the case here. Much (if not all) of what I found on the web about WebSocket is about a browser/server connection and does not talk about use in a server/server connection, hence my question. But you answered it, and your answer is ok with my thoughts (plus using Zn components is a breeze) Thanks again regards, Alain Le 07/10/2014 18:26, Paul DeBruicker a écrit :

marten wrote

...

There are some good C libraries out there, which are suitable to connect programs with each other: a good example is 0MQ.

Marten

-- Marten Feldtmann

Hi Marten,

Is your 0mq library posted somewhere and MIT licensed?

Thanks

Paul

-- View this message in context: http://forum.world.st/About-Zinc-http-components-tp4783071p4783223.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Marten, I keep a close eye to MQ based architectures, because somehow they feel "the best way" to implement interprocess/service messaging. And also because I always wanted to implement something based on message queues, just for the sake of it. :) However I don't find them suitable for interactive bidirectional communication, but instead as their name suggest, queues. Ideally for asynchronous communication, a "fire and forget" way of messaging. If you go by the MQ path Sven implemented STAMP [1], which can work with RabbitMQ's adapter [2] Unless you have an evented/callback based subscription to a queue, the polling and message consumption from the queue itself will consume about similar CPU cycles. What you gain using MQ is that you can offload computation to other "worker images". Regards! [1] https://github.com/svenvc/docs/blob/master/neo/stamp.md [2] http://www.rabbitmq.com/stomp.html Esteban A. Maringolo 2014-10-07 15:18 GMT-03:00 itlists@schrievkrom.de <itlists@schrievkrom.de>:

I have a little different view. lpgl is no real problem for me - it may be used for non-open-soure stuff without problems. Therefore I see no real problems with that.

The other thing is: speed. And here it might differ a lot. I had written a heavy communication server in Smalltalk and with about 30 clients the Smalltalk process was CPU bound. No way to go.

Introducing 0MQ and put all communication to an external thread and Smalltalk can fly again. That's the difference.

Marten

Am 07.10.2014 um 20:12 schrieb Alain Rastoul:

...

Thanks you for your answers.

0mq looks great, but released with LGPL licence, and I don't like that (I don't understand why there is also a GPL licence file in the distribution ? - clearly a *noway*), I prefer a full smalltalk implementation I can debug when things go wrong. I think latency and timing problems would be the same, no big difference in performance. BTW knowing about this library is cool and may be of some help (who knows), thank you Marten for the reference.

I agree totally with you Sven about performance and standards. And I also agree with you about reusing someone else's work ... I don't like to reinvent the wheel, especially a good wheel when mine would be worst, I would if I were a wheel specialist - clearly not the case here.

-- Marten Feldtmann

2014-10-07 16:19 GMT-03:00 Norbert Hartl <norbert@hartl.name>:

Esteban,

die mq in 0mq is misleading here because it is not a message queue. 0mq is called sockets on stereoids where you can plug communication channels with less overhead. It is more of an orchestration toolkit for distributed computing.

I can see. Thank you. I overlooked it, It seems really cool! But certainly it is more low level than a "regular" MQ :) Regards!

About 0mq, to be honest, I missed the external thread part, and that's *very* interesting , for sure. I find the license part quite complicated, but rereading it, you are right, it should not be a problem (though still weird). My goal is not to build a server with a lot of connections, and no message queuing neither, but (for this part) a server with some basic and raw "streaming" functionality and quite few connections for a start - this functionality is not the core of the system, just a part of it, and I'm trying to find building blocks. I think I have to make some compatible api on top of whatever I choose (ZnWebSockets/0mq or nanomsg) to be able to change easily. Do you know some existing good bindings for this library that I could reuse? The one on gemstone/squeaksource3 (from you?) seems to be empty, one on smalltalkhub (panuw) that I was able to load correctly and was maintained until february 2014. Thanks Alain Le 07/10/2014 21:23, itlists@schrievkrom.de a écrit :

0MQ is not message queueing in the normal sense as known from all other MQ tools/libraries.

0MQ is about networking in general ... and their introduction text with all their examples is one of the best examples documentation I've ever read in the area of practical networking. Even though one might not use 0MQ one should read their views about putting network patterns together to form a distributed network of software nodes (in different languages).

For Smalltalk 0MQ is a wonderful library:

-> because of the interactive character of Smalltalk you might run a 0MQ node and still can inspect the whole traffic. That's a thing other languages can dream of.

-> due to the fact, that the whole communcation is done in an external thread you simply win Smalltalk CPU time (which is always needed).

Marten

Nanosg looks interesting too, thank you Kilon, I will look at it closer too. Do you known some working bindings with pharo ? Alain Le 07/10/2014 21:12, kilon alios a écrit :

there is nanomsg which is a rethinking on 0mq and from same developers, its MIT licensed --> http://nanomsg.org/index.html

On Tue, Oct 7, 2014 at 9:12 PM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

Thanks you for your answers.

0mq looks great, but released with LGPL licence, and I don't like that (I don't understand why there is also a GPL licence file in the distribution ? - clearly a *noway*), I prefer a full smalltalk implementation I can debug when things go wrong. I think latency and timing problems would be the same, no big difference in performance. BTW knowing about this library is cool and may be of some help (who knows), thank you Marten for the reference.

I agree totally with you Sven about performance and standards. And I also agree with you about reusing someone else's work ... I don't like to reinvent the wheel, especially a good wheel when mine would be worst, I would if I were a wheel specialist - clearly not the case here.

Much (if not all) of what I found on the web about WebSocket is about a browser/server connection and does not talk about use in a server/server connection, hence my question. But you answered it, and your answer is ok with my thoughts (plus using Zn components is a breeze)

Thanks again

regards,

Alain Le 07/10/2014 18:26, Paul DeBruicker a écrit :

marten wrote

There are some good C libraries out there, which are suitable to connect programs with each other: a good example is 0MQ.

Marten

-- Marten Feldtmann

Hi Marten,

Is your 0mq library posted somewhere and MIT licensed?

Thanks

Paul

-- View this message in context: http://forum.world.st/About-__Zinc-http-components-__tp4783071p4783223.html <http://forum.world.st/About-Zinc-http-components-tp4783071p4783223.html> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

0MQ is defined by its exported C interface ... Am 07.10.2014 um 23:51 schrieb kilon alios:

nope but it is made (unlike 0mq which is made in C++) in C so its should be relative simple to wrap with NB or even TalkFFI. At least the parts that interest you.

-- Marten Feldtmann

I am almost always right. maybe ask the original author/s via github ? Why do things the hard way ;) On Wed, Oct 8, 2014 at 8:54 AM, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

You are both right. Question about nanomsg is the thread model (a big bonus of 0mq), which is not clear to me. they state: "...In nanomsg the objects are not tightly bound to particular threads and thus these problems don't exist...", about some thread related issues in 0mq. I'll have to check (with sources or more docs readings) if they are bound to an external thread. If they account for an external thread support, this is not possible with the current vm without writing some external C code (I could but I would not like). Le 08/10/2014 07:07, itlists@schrievkrom.de a écrit :

0MQ is defined by its exported C interface ...

...

Am 07.10.2014 um 23:51 schrieb kilon alios:

...

nope but it is made (unlike 0mq which is made in C++) in C so its should be relative simple to wrap with NB or even TalkFFI. At least the parts that interest you.

I am interested too, my project depends on streamsocket to send python commands from pharo to blender . So far I have not stressed it too much and it looks like it works fast enough. Maybe something like websockets or nanomsg is better for me in the long run so I too keep an eye on these solutions. Time will tell. On Fri, Oct 10, 2014 at 12:38 AM, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

Le 09/10/2014 14:14, kilon alios a écrit :

...

I am almost always right.

Good medicine :)

maybe ask the original author/s via github ? Why do things the hard way ;)

...

Yes may be I will after having looked closely at the source,

no hard way here, it's not a problem to read c or c++ code, just a little pain (for me) to write some (I did a lot, and even liked it long time ago, but for now, I prefer simpler and cleaner way - like smalltalk - to do just what I want - it's more a matter of taste). If I use this library they won't debug my code. In smalltalk with ZnWebSocket, I could easily, in c or c++ I could too but a bit harder - I'll have to rebuild libraries, use an external debugguer and so on - not a big deal but I don't like it much.

On Wed, Oct 8, 2014 at 8:54 AM, Alain Rastoul

...

<alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

You are both right. Question about nanomsg is the thread model (a big bonus of 0mq), which is not clear to me. they state: "...In nanomsg the objects are not tightly bound to particular threads and thus these problems don't exist...", about some thread related issues in 0mq. I'll have to check (with sources or more docs readings) if they are bound to an external thread. If they account for an external thread support, this is not possible with the current vm without writing some external C code (I could but I would not like). Le 08/10/2014 07:07, itlists@schrievkrom.de <mailto:itlists@schrievkrom.de> a écrit :

0MQ is defined by its exported C interface ...

Am 07.10.2014 um 23:51 schrieb kilon alios:

nope but it is made (unlike 0mq which is made in C++) in C so its should be relative simple to wrap with NB or even TalkFFI. At least the parts that interest you.

You are lucky that you hate windoom sometimes, I hate it all the time :D Yes as I said I am very happy so far with StreamSockets and I have found ways to minimise the socket communications so that the user don't experience any noticeable lag so I don't currently need either 0mq or nanomsg. So its unlikely that I will be wrapping nanomsg any time soon since my hands are already full with bringing Blender features to Pharo. I have done some very basic benchmarks with StreamSocket and it gives me around 1ms from the messages I am sending which is usually just small bytestring of one line of python code. It will be rare to want to send more than 100 messages in under a second since it would be better in that case if the code is that big to write a python module from pharo and load it to python. So speed for now does not seem an issue for me. Even if I notice delays my second option will be websockets. And if that is not good enough I could venture in the real of shared memory but I rather not. There are many things one wishes and wants to do, but in the end time is very limited. I also love to keep things easy and simple because I know how easy it is to underestimate the difficulty of things to do. Right now I ask myself what is the very easy things I can do with Pharo that can be practically useful for me. So far python and pharo sockets have been very easy and I am very happy with them. Its a lot of fun to manipulate external applications from pharo. On Sat, Oct 11, 2014 at 12:40 PM, Alain Rastoul <alf.mmm.cat@gmail.com> wrote:

Hi Kilon,

I wanted to make some experiments with Nanomsg this morning, but had some problems to build the libraries (they is no prebuilt binaries for windows): I discovered that the system path of my os has been scratched by some recent mercurial installation, the user path scratched by delphi and bonus, my mingw/cygwin install seems to be sick too. sometimes I hate windows :( - or is this a sign ?

btw, my feeling about nanomsg is that it is still in beta, and 0mq much more advanced (they say some users includes AT&T, Cisco, EA, Los Alamos Labs, NASA, Weta Digital, Zynga, Spotify, Samsung Electronics, Microsoft, and CERN - not too bad). The material about it (docs, presentations) sounds good too. Not saying that nanomsg is not good, may be it will be the best in few years, but at that time I have a small preference in favour of 0mq.

This part is not essential for me right now (the same as you if I understand) and I have lot of other things to finish, so I will delay it for some time, but I am still interested in your experiments too : bindings, benchs, findings.

Regards,

Alain

Le 10/10/2014 20:12, kilon alios a écrit :

I am interested too, my project depends on streamsocket to send python

...

commands from pharo to blender . So far I have not stressed it too much and it looks like it works fast enough. Maybe something like websockets or nanomsg is better for me in the long run so I too keep an eye on these solutions. Time will tell.

Hi Phil, Yes, multi core support and communication is important, having that in pharo could be great. Unfortunately, no I actually have no usable project for that, mine is not a communication project and still really in pre-alpha stage. For now, I'm starting with Zinc, 0mq is just from a prospective eye. I have no experience with 0mq, I discovered it just few days ago (see Marten's messages), but it seemsto be easy to use. What I did is just read the echo server sample, and threw few lines in a workspace to see how it works (here modified to send/receive 8k pages instead of echoing the message). I am also interested in something more elaborate , as I will probably use it later, I will think about a simple test project (the workspace here is just a very first experiment) and see what I can do. The bindings are FFI, may be it should be rewritten/modified to use native boost (I know only the name here). If you want to look here it is, it's very basic and only the REQ/REP pattern (not sure the other works, I tried but the vm hanged, may be my fault, may be it also requires more work). You have to load Panu 0mq bindings before and put a zmq.dll in your path. The server (kill the server with Process browser to stop it) "====================================== classic sockets - REQ/REP server" [ | context serverSocket request count | [ count := 0 . context := ZmqContext new . serverSocket := ZmqResponseSocket new. serverSocket bind: 'tcp://*:5555' . [ true] whileTrue: [ request := serverSocket receive . "fake request" serverSocket send: (ByteArray new: 8*1024 ). "block response" count := count+1 ]. ] ensure: [ serverSocket close. ('server exited -', count asString, ' pages sent ') explore ] ] forkAt: Processor "systemBackgroundPriority" userBackgroundPriority "lowIOPriority hangs" . The client (5 sec bench of request/reply) : "======================================== classic sockets - REQ/REP client" [ | context socket response count bench | [ context := ZmqContext new . socket := ZmqRequestSocket new. count := 0. socket connect: 'tcp://127.0.0.1:5555' . bench := [ socket send: 'get xxx' asByteArray . "fake request" response := socket receive . count := count + 1 ] bench . ] ensure: [ socket close. bench explore. ('client asked for ',count asString) explore ] ] forkAt: Processor userBackgroundPriority "lowIOPriority" . Le 11/10/2014 12:31, phil@highoctane.be a écrit :

Alain,​

Yes, 0mq seems to be a good way to go as there is just more support across the board.

Have you a project on Smalltalkhub (or elsewhere) with your experiments so that I could have a look? I've updated the 0MQ bindings page to list the smalltalkhub pharo client and also had a look at what was in the GST client.

It would be nice to port the GST code to Pharo so that we would have a nice package.

0mq could help with the multicore support very nicely. We are going to use SDL for low level, why not go 0mq for interimage communication? It makes sense to me.

Phil

On Thu, Oct 09, 2014 at 11:38:47PM +0200, Alain Rastoul wrote:

but for now, I prefer simpler and cleaner way - like smalltalk - to do just what I want - it's more a matter of taste).

Whether using Zinc's websockets or some C library wrapper, you are still slinging serialized Smalltalk objects ie plain old bytes over the wire, no? I am guessing that RST provides synchronous remote method invocation. I recall there was a remote messaging prototype/concept based on promises developed in Squeak... ah, Squeak-Elib, here: http://wiki.squeak.org/squeak/6011. Is your current use case mostly synchronous or asynchronous? Pierce

It would be really interesting to get a binding for Pharo. Stef On 7/10/14 23:51, kilon alios wrote:

nope but it is made (unlike 0mq which is made in C++) in C so its should be relative simple to wrap with NB or even TalkFFI. At least the parts that interest you.

On Tue, Oct 7, 2014 at 11:56 PM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

Nanosg looks interesting too, thank you Kilon, I will look at it closer too. Do you known some working bindings with pharo ?

Alain Le 07/10/2014 21:12, kilon alios a écrit :

there is nanomsg which is a rethinking on 0mq and from same developers, its MIT licensed --> http://nanomsg.org/index.html

On Tue, Oct 7, 2014 at 9:12 PM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com> <mailto:alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>>> wrote:

Thanks you for your answers.

0mq looks great, but released with LGPL licence, and I don't like that (I don't understand why there is also a GPL licence file in the distribution ? - clearly a *noway*), I prefer a full smalltalk implementation I can debug when things go wrong. I think latency and timing problems would be the same, no big difference in performance. BTW knowing about this library is cool and may be of some help (who knows), thank you Marten for the reference.

I agree totally with you Sven about performance and standards. And I also agree with you about reusing someone else's work ... I don't like to reinvent the wheel, especially a good wheel when mine would be worst, I would if I were a wheel specialist - clearly not the case here.

Much (if not all) of what I found on the web about WebSocket is about a browser/server connection and does not talk about use in a server/server connection, hence my question. But you answered it, and your answer is ok with my thoughts (plus using Zn components is a breeze)

Thanks again

regards,

Alain Le 07/10/2014 18:26, Paul DeBruicker a écrit :

marten wrote

There are some good C libraries out there, which are suitable to connect programs with each other: a good example is 0MQ.

Marten

-- Marten Feldtmann

Hi Marten,

Is your 0mq library posted somewhere and MIT licensed?

Thanks

Paul

-- View this message in context: http://forum.world.st/About-__Zinc-http-components-__tp4783071p4783223.html

<http://forum.world.st/About-Zinc-http-components-tp4783071p4783223.html> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

This is another subject and another functionality I will need too. The Rst link is of interest (thank you Denis) , I was thinking of sending command objects with Fuel as all images will have the same classes and performance is not very important here. I was able to load RST in Pharo3, but did not made any test yet. Does it works with Pharo ? @SKrish you are right, but I want to keep as simple as possible at the beginning and tcpip should fit for a start. Cheers, Alain Le 07/10/2014 21:40, S Krish a écrit :

Typical issue in the EJB world : Remote Beans / Local Beans , though on inter process it was remote beans with its full stack RMI marshalling / unmarshalling.

But can we not exploit shared memory and efficient Event mechanism to make the two process coordinate, that should be lot more efficient than TCP - IP / but if that performance suffices it would be fine..

On Tue, Oct 7, 2014 at 4:40 AM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

Hi, I played a bit with ZnServer and other zinc components and have a question I can't answer myself (googling a bit didn't help neither), and I'm seeking for advice: does it makes sense to use a ZnServer/ZnWebSocket as a mechanism to transfer data between two pharo processes - in my case 8k ByteArray blocks ? Or is it a total non sense ? is it reliable ? First tests looks good : 18k blocks / second (145 Mb/s) on a laptop core i5 2.6Ghz.

All comments and suggestions welcome

Thanks in advance,

Alain

Sorry, I think I have screwed my tests, I wasn't able to reload it right now (I think I was browsing the changeset while loading 0mq package in another image). There are missing references and lot of errors I will investigate later, I have other works in progress, and this one is in my list later, If I get it working I will contribute with my changes ... :) BTW this link is worth knowing about, thank you for the link. Regards, Alain Le 08/10/2014 06:03, Denis Kudriashov a écrit :

I was try it many years ago with squeak. It was work great. I am sure it should be not difficult to adobt it for latest pharo

08 окт. 2014 г. 1:09 пользователь "Alain Rastoul" <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> написал:

This is another subject and another functionality I will need too.

The Rst link is of interest (thank you Denis) , I was thinking of sending command objects with Fuel as all images will have the same classes and performance is not very important here. I was able to load RST in Pharo3, but did not made any test yet. Does it works with Pharo ?

@SKrish you are right, but I want to keep as simple as possible at the beginning and tcpip should fit for a start.

Cheers,

Alain

Le 07/10/2014 21:40, S Krish a écrit :

Typical issue in the EJB world : Remote Beans / Local Beans , though on inter process it was remote beans with its full stack RMI marshalling / unmarshalling.

But can we not exploit shared memory and efficient Event mechanism to make the two process coordinate, that should be lot more efficient than TCP - IP / but if that performance suffices it would be fine..

On Tue, Oct 7, 2014 at 4:40 AM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com> <mailto:alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>>__> wrote:

Hi, I played a bit with ZnServer and other zinc components and have a question I can't answer myself (googling a bit didn't help neither), and I'm seeking for advice: does it makes sense to use a ZnServer/ZnWebSocket as a mechanism to transfer data between two pharo processes - in my case 8k ByteArray blocks ? Or is it a total non sense ? is it reliable ? First tests looks good : 18k blocks / second (145 Mb/s) on a laptop core i5 2.6Ghz.

All comments and suggestions welcome

Thanks in advance,

Alain

(I thought I already responded , and rereading all just saw I did not, I think I did enregistrer and closed no send, sorry) Thank you for the update, I 'll have a look at that too. It does not goes exactly the same way I will, and the slideshare presentation looks a bit scary ... Die socket Die ! ? :) but interesting, I often go to look esug presentations and never saw this one (lot of things to investigate too). Since yesterday, thinking about your advice (small steps) and Phil's question about my experiments I went to the conclusion that I have to publish a small project abut my ramblings on smalltalkhub, a good way to start, but kind some work to clean the bazard :) Le 11/10/2014 08:41, stepharo a écrit :

Nick Papoulias developed Seamless because he could not fix rST (but he oversold Seamless a bit) and I asked him to do a pass on Seamless and write a documentation.

Stef

On 7/10/14 23:08, Alain Rastoul wrote:

...

This is another subject and another functionality I will need too.

The Rst link is of interest (thank you Denis) , I was thinking of sending command objects with Fuel as all images will have the same classes and performance is not very important here. I was able to load RST in Pharo3, but did not made any test yet. Does it works with Pharo ?

@SKrish you are right, but I want to keep as simple as possible at the beginning and tcpip should fit for a start.

Cheers,

Alain

Le 07/10/2014 21:40, S Krish a écrit :

...

Typical issue in the EJB world : Remote Beans / Local Beans , though on inter process it was remote beans with its full stack RMI marshalling / unmarshalling.

But can we not exploit shared memory and efficient Event mechanism to make the two process coordinate, that should be lot more efficient than TCP - IP / but if that performance suffices it would be fine..

On Tue, Oct 7, 2014 at 4:40 AM, Alain Rastoul <alf.mmm.cat@gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote:

Hi, I played a bit with ZnServer and other zinc components and have a question I can't answer myself (googling a bit didn't help neither), and I'm seeking for advice: does it makes sense to use a ZnServer/ZnWebSocket as a mechanism to transfer data between two pharo processes - in my case 8k ByteArray blocks ? Or is it a total non sense ? is it reliable ? First tests looks good : 18k blocks / second (145 Mb/s) on a laptop core i5 2.6Ghz.

All comments and suggestions welcome

Thanks in advance,

Alain