On 29 Jun 2021, at 04:55, Esteban Maringolo emaringolo@gmail.com wrote:

Hi,

I'm rearchitecting a web app to perform updates only when necessary
(instead of computing them all the time) on each request, I can have a
global announcer and subscribers to know when to update within an
image, but is there a way to have something like that but for
inter-image coordination?

I'd only need to communicate the id and the class name (or a similar
identifier), so on other images they'll update accordingly, and if
there is an update in one image, it will notify the other images. The
common data is on the database, so this is just to avoid re-reading a
lot of things.

Is a message queue a good fit for this? Pub/Sub?
What is available in Pharo that works without having to set up a lot of things?

Thanks!

Esteban A. Maringolo

Hi Esteban,

On 29 Jun 2021, at 04:55, Esteban Maringolo emaringolo@gmail.com wrote:

Hi,

I'm rearchitecting a web app to perform updates only when necessary
(instead of computing them all the time) on each request, I can have a
global announcer and subscribers to know when to update within an
image, but is there a way to have something like that but for
inter-image coordination?

I'd only need to communicate the id and the class name (or a similar
identifier), so on other images they'll update accordingly, and if
there is an update in one image, it will notify the other images. The
common data is on the database, so this is just to avoid re-reading a
lot of things.

Is a message queue a good fit for this? Pub/Sub?
What is available in Pharo that works without having to set up a lot of things?

Thanks!

Esteban A. Maringolo

RabbitMQ (for which there is the STOMP client 'STAMP' https://github.com/svenvc/stamp) is one option, but it is a bit more complex.

I guess Redis would work too (https://medium.com/concerning-pharo/quick-write-me-a-redis-client-5fbe4ddfb13d).

More recently I have been using MQTT (with the client https://github.com/svenvc/mqtt) which is much simpler.

You post a message on a topic and have one or more listeners see it. You can even arrange for the broker to keep the messages you miss (within a reasonable window) - which is very nice for all kinds of reasons (especially operational). I have a system in production that uses this mechanism to coordinate different images (and handle ingress of data) for more than a year now.

Sven

On 29 Jun 2021, at 15:50, Esteban Maringolo emaringolo@gmail.com wrote:

Hi Sven,

I thought about both RabittMQ and MQTT too.

For RabittMQ I noticed you provide a docker config to get a container
running quickly.

What is the easy-go solution for MQTT?

Hi,

I'm rearchitecting a web app to perform updates only when necessary
(instead of computing them all the time) on each request, I can have a
global announcer and subscribers to know when to update within an
image, but is there a way to have something like that but for
inter-image coordination?

I'd only need to communicate the id and the class name (or a similar
identifier), so on other images they'll update accordingly, and if
there is an update in one image, it will notify the other images. The
common data is on the database, so this is just to avoid re-reading a
lot of things.

Is a message queue a good fit for this? Pub/Sub?
What is available in Pharo that works without having to set up a lot of
things?

Thanks!

Esteban A. Maringolo

Maybe ZeroMQ fits you, but unlucky is not documented and there are only a few examples. I use ir in my jupyter kernel. You can install my port to pharo64 uFFI doing:

Metacello new
baseline: 'JupyterTalk';
repository: 'github://jmari/JupyterTalk:master/repository';
load:'zmq'

But it needs ZeroMQ previously installed on your system.
It was ported from http://smalltalkhub.com/#!/~panuw/zeromq now it is at: https://github.com/dellani/zeroMQ but I'm not sure if the original repo works on newer Pharos....I had no time to contact the original author to join both repos...

El mar, 29 jun 2021 a las 4:56, Esteban Maringolo (emaringolo@gmail.com) escribió:

Hi,

I'm rearchitecting a web app to perform updates only when necessary
(instead of computing them all the time) on each request, I can have a
global announcer and subscribers to know when to update within an
image, but is there a way to have something like that but for
inter-image coordination?

I'd only need to communicate the id and the class name (or a similar
identifier), so on other images they'll update accordingly, and if
there is an update in one image, it will notify the other images. The
common data is on the database, so this is just to avoid re-reading a
lot of things.

Is a message queue a good fit for this? Pub/Sub?
What is available in Pharo that works without having to set up a lot of things?

Thanks!

Esteban A. Maringolo

If you want to use Rabbit you can also take a look at Ansible for connecting using the protocol instead of STOMP.

On Tue, Jun 29, 2021 at 3:11 PM Esteban Maringolo emaringolo@gmail.com wrote:

As far as I understand, ZeroMQ does not require a broker to perform
the communication between publishers and subscribers, I don't know how
that is implemented (what do they connect to? how each client
discovers each other?).

I already got MQTT running, It's not completely clear to me how the
QoS setting works (atLeastOnce, exactlyOnce, atMostOnce), I don't
remember using that setting in Java (it probably had a sensible
default).

Regards!

Esteban A. Maringolo

On Tue, Jun 29, 2021 at 2:45 PM Jesus Mari Aguirre
jmariaguirre@gmail.com wrote:

Maybe ZeroMQ fits you, but unlucky is not documented and there are only a few examples. I use ir in my jupyter kernel. You can install my port to pharo64 uFFI doing:

Metacello new
baseline: 'JupyterTalk';
repository: 'github://jmari/JupyterTalk:master/repository';
load:'zmq'

But it needs ZeroMQ previously installed on your system.
It was ported from http://smalltalkhub.com/#!/~panuw/zeromq now it is at: https://github.com/dellani/zeroMQ but I'm not sure if the original repo works on newer Pharos....I had no time to contact the original author to join both repos...

El mar, 29 jun 2021 a las 4:56, Esteban Maringolo (emaringolo@gmail.com) escribió:

Hi,

I'm rearchitecting a web app to perform updates only when necessary
(instead of computing them all the time) on each request, I can have a
global announcer and subscribers to know when to update within an
image, but is there a way to have something like that but for
inter-image coordination?

I'd only need to communicate the id and the class name (or a similar
identifier), so on other images they'll update accordingly, and if
there is an update in one image, it will notify the other images. The
common data is on the database, so this is just to avoid re-reading a
lot of things.

Is a message queue a good fit for this? Pub/Sub?
What is available in Pharo that works without having to set up a lot of things?

Thanks!

Esteban A. Maringolo

On 29 Jun 2021, at 20:10, Esteban Maringolo emaringolo@gmail.com wrote:

I already got MQTT running, It's not completely clear to me how the
QoS setting works (atLeastOnce, exactlyOnce, atMostOnce), I don't
remember using that setting in Java (it probably had a sensible
default).

On 30 Jun 2021, at 01:28, Esteban Maringolo emaringolo@gmail.com wrote:

I like the Ansible approach using RabbitMQ, in particular because it
would also work to delegate tasks to workers (even if in the same
image), and the task queue would be preserved, whilst in MQTT if there
are no subscribers for a topic, then all messages sent to it are
discarded.