July 2021 - Pharo-users - lists.pharo.org

Question about Spec2 and SpTablePresenter

by Mark O'Donoghue

Howdy all Iâve got stuck trying to manage Tables â over 12 hours now and Iâm out of ideas!. â¹ Any observations / suggestions are most welcomeâ¦ Iâve been loading small external files of transactions using NeoCSV into Spec2 tables. I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session. (Iâve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.) The idea is that transactions (and potentially some manual adjustments) will be processed over time. (This is preferable to having to reload all files from the beginning evert time I run the applicationâ¦) The Spec2 tables have been working well until I tried to persist them. I canât seem to fully re-load them to a previously saved state. For example - I can restore the essential contents of my table in most circumstances using: restoreObjects âfilePresenter1 is a SpTablePresenterâ | objects savedEntry | objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'. recentFileList := objects at: 1. currFileFilter := objects at: 2. savedEntry := objects at: 3. self updateFilterButton: currFileFilter. filteredFileList := self filterFilesUsing: currFileFilter. filePresenter1 items: filteredFileList. savedEntry ifNotNil: [filePresenter1 selectItem: savedEntry ]. However, if any of the table columns are re-sorted , the re-load operation gets confused and I canât get the saved a saved selected item to become selected again. (It seems to be confusing the index numbers of the sorted and unsorted lists â even when I match by contents rather than index.) (I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ). This all works fine - unless I sort a column! Since this approach was going to be used on several screens Iâd really like to find a solution. Cheers Mark Perth, Western Australia

July 4, 2021

Re: Communication between different images

by Jesus Mari Aguirre

On ZeroMQ it could be something like that...first you need a broker (a proxy) using XSUB and XPUB. In one pharo image... (I think this part better be done in C because it will freezy the image forever but it works) | frontend backend | frontend := ZmqXSubscriberSocket new. frontend bind: 'tcp://*:5559'. backend := ZmqXPublisherSocket new. backend bind: 'tcp://*:5560'. ZmqProxy frontend: frontend backend: backend capture: nil In any publisher or subscriber... publisher := ZmqPublisherSocket new connect: 'tcp://localhost:5559'. pr1 := [ |subscriber1| subscriber1 := ZmqSubscriberSocket new connect: 'tcp://localhost:5560'. [true] whileTrue:[ "receive is non blocking so this is an infinite loop" subscriber1 receiveIfAvailable: [ :messageData| messageData isEmpty ifFalse:[Transcript show: ('Subscriber 1 got stuff ' , messageData);cr]] ]] forkNamed: 'subscriber1'. "Publish doing that" publisher send: ('Hello Subscribers') asByteArray. El jue, 1 jul 2021 a las 21:03, Esteban Maringolo (<emaringolo(a)gmail.com>) escribiÃ³: > On Thu, Jul 1, 2021 at 3:00 PM Jesus Mari Aguirre > <jmariaguirre(a)gmail.com> wrote: > > > > As far as I know, zmq doesn't need a broker but subscribers should know > the address of the publisher, if the network increases its complexity with > more publishers you need a broker, that is a proxy on zmq. > > If I understand well you need any of them should be able to publish a > change to all of the other images? > > I want to broadcast notifications of updated objects so they can be > recomputed or reloaded from the database to reflect the latest > changes. > The change might happen in any of the worker images, either by user UI > or API call. > > Regards! > > Esteban A. Maringolo >

July 3, 2021

Re: Communication between different images

by Sven Van Caekenberghe

> On 1 Jul 2021, at 21:01, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > On Thu, Jul 1, 2021 at 3:00 PM Jesus Mari Aguirre > <jmariaguirre(a)gmail.com> wrote: >> >> As far as I know, zmq doesn't need a broker but subscribers should know the address of the publisher, if the network increases its complexity with more publishers you need a broker, that is a proxy on zmq. >> If I understand well you need any of them should be able to publish a change to all of the other images? > > I want to broadcast notifications of updated objects so they can be > recomputed or reloaded from the database to reflect the latest > changes. > The change might happen in any of the worker images, either by user UI > or API call. This reminds me that there is a notification mechanism in PostgreSQL. I have not used it though. The SQL commands are NOTIFY and (UN)LISTEN. I think these will arrive as P3Notifications. > > Regards! > > Esteban A. Maringolo

July 1, 2021

Re: Communication between different images

by Sven Van Caekenberghe

> On 1 Jul 2021, at 20:58, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > Hi Sven, > > Thanks, now I understand better the use of QoS and session ID with a > practical use case. > > How do you deal with the lifetime of each "listener" (subscriber) on > each image? I mean, how stable is it? > > E.g. > MQTTClient new > open; > subscribeToTopic: '/updates'; > runWith: [ :message | > Transcript cr; show: '[UPDATED]'; space; show: message contents > asString; cr. > ]; > yourself. > > You just fork that and it will continue to receive messages until you > close the connection? Basically, yes. > Is there a watchdog class/library to ensure > things like this continue working? (I'm used to having my images > living for months sometimes). I have an MQTTListener class that wraps the MQTT client and its process, the process's run method looks like this: run [ self log: 'Starting MQTT reading'. self ensureMQTTClient ifTrue: [ mqttClient runWith: [ :message | self handleMessage: message ] ]. self log: 'End of MQTT reading, restarting in 5s'. self closeMQTTClient. 5 seconds wait ] repeat It is a bit crude but it does help to restore the connection when the broker becomes temporarily unavailable. > Thanks in advance. > > Regards! > > > Esteban A. Maringolo > > > On Wed, Jun 30, 2021 at 5:00 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> >> >>> On 30 Jun 2021, at 01:28, Esteban Maringolo <emaringolo(a)gmail.com> wrote: >>> >>> I like the Ansible approach using RabbitMQ, in particular because it >>> would also work to delegate tasks to workers (even if in the same >>> image), and the task queue would be preserved, whilst in MQTT if there >>> are no subscribers for a topic, then all messages sent to it are >>> discarded. >> >> That is correct but less of a problem in practice than it seems to be at first sight. >> >> As a client, you normally have to be able to start from scratch and load everything that came before the moment you start up. >> >> When a client with a specific ID has connected once with clean session false, QoS 1 and a subscription, the persistence of the queue will be enabled for a as long as the server lives (and maybe even beyond restarts). >> >> There is absolutely no question that Rabbit MQ has much more functionality, I found Mosquitto MQTT very nice to work with, but as always, YMMV. >> >> Sven

July 1, 2021

Re: Communication between different images

by Esteban Maringolo

On Thu, Jul 1, 2021 at 3:00 PM Jesus Mari Aguirre <jmariaguirre(a)gmail.com> wrote: > > As far as I know, zmq doesn't need a broker but subscribers should know the address of the publisher, if the network increases its complexity with more publishers you need a broker, that is a proxy on zmq. > If I understand well you need any of them should be able to publish a change to all of the other images? I want to broadcast notifications of updated objects so they can be recomputed or reloaded from the database to reflect the latest changes. The change might happen in any of the worker images, either by user UI or API call. Regards! Esteban A. Maringolo

July 1, 2021

Re: Communication between different images

by Esteban Maringolo

Hi Sven, Thanks, now I understand better the use of QoS and session ID with a practical use case. How do you deal with the lifetime of each "listener" (subscriber) on each image? I mean, how stable is it? E.g. MQTTClient new open; subscribeToTopic: '/updates'; runWith: [ :message | Transcript cr; show: '[UPDATED]'; space; show: message contents asString; cr. ]; yourself. You just fork that and it will continue to receive messages until you close the connection? Is there a watchdog class/library to ensure things like this continue working? (I'm used to having my images living for months sometimes). Thanks in advance. Regards! Esteban A. Maringolo On Wed, Jun 30, 2021 at 5:00 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > > > On 30 Jun 2021, at 01:28, Esteban Maringolo <emaringolo(a)gmail.com> wrote: > > > > I like the Ansible approach using RabbitMQ, in particular because it > > would also work to delegate tasks to workers (even if in the same > > image), and the task queue would be preserved, whilst in MQTT if there > > are no subscribers for a topic, then all messages sent to it are > > discarded. > > That is correct but less of a problem in practice than it seems to be at first sight. > > As a client, you normally have to be able to start from scratch and load everything that came before the moment you start up. > > When a client with a specific ID has connected once with clean session false, QoS 1 and a subscription, the persistence of the queue will be enabled for a as long as the server lives (and maybe even beyond restarts). > > There is absolutely no question that Rabbit MQ has much more functionality, I found Mosquitto MQTT very nice to work with, but as always, YMMV. > > Sven

July 1, 2021

Re: Communication between different images

by Jesus Mari Aguirre

As far as I know, zmq doesn't need a broker but subscribers should know the address of the publisher, if the network increases its complexity with more publishers you need a broker, that is a proxy on zmq. If I understand well you need any of them should be able to publish a change to all of the other images? El mar., 29 jun. 2021 20:11, Esteban Maringolo <emaringolo(a)gmail.com> escribiÃ³: > As far as I understand, ZeroMQ does not require a broker to perform > the communication between publishers and subscribers, I don't know how > that is implemented (what do they connect to? how each client > discovers each other?). > > I already got MQTT running, It's not completely clear to me how the > QoS setting works (atLeastOnce, exactlyOnce, atMostOnce), I don't > remember using that setting in Java (it probably had a sensible > default). > > Regards! > > > Esteban A. Maringolo > > On Tue, Jun 29, 2021 at 2:45 PM Jesus Mari Aguirre > <jmariaguirre(a)gmail.com> wrote: > > > > Maybe ZeroMQ fits you, but unlucky is not documented and there are only > a few examples. I use ir in my jupyter kernel. You can install my port to > pharo64 uFFI doing: > > > > Metacello new > > baseline: 'JupyterTalk'; > > repository: 'github://jmari/JupyterTalk:master/repository'; > > load:'zmq' > > > > But it needs ZeroMQ previously installed on your system. > > It was ported from http://smalltalkhub.com/#!/~panuw/zeromq now it is > at: https://github.com/dellani/zeroMQ but I'm not sure if the original > repo works on newer Pharos....I had no time to contact the original author > to join both repos... > > > > > > > > El mar, 29 jun 2021 a las 4:56, Esteban Maringolo (<emaringolo(a)gmail.com>) > escribiÃ³: > >> > >> Hi, > >> > >> I'm rearchitecting a web app to perform updates only when necessary > >> (instead of computing them all the time) on each request, I can have a > >> global announcer and subscribers to know when to update within an > >> image, but is there a way to have something like that but for > >> inter-image coordination? > >> > >> I'd only need to communicate the id and the class name (or a similar > >> identifier), so on other images they'll update accordingly, and if > >> there is an update in one image, it will notify the other images. The > >> common data is on the database, so this is just to avoid re-reading a > >> lot of things. > >> > >> Is a message queue a good fit for this? Pub/Sub? > >> What is available in Pharo that works without having to set up a lot of > things? > >> > >> Thanks! > >> > >> Esteban A. Maringolo >

July 1, 2021