- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Berlin 2016-12-19

by Dimitris Chloupis

I don't see anywhere me mentioning my support for a political party or ideology or even mention politics at all. You must have a very abstract definition of "politics". Funny thing is that you could not quote the most inappropriate case because I am completely against any form of politics including anarchism. So please don't make assumptions about people who you know very little about. Accuse me that my reply is not Pharo development based , guilty as charged, but then so is not the topic of the discussion. On Thu, 22 Dec 2016 at 12:37, Norbert Hartl <norbert(a)hartl.name> wrote: > I don't think this list is the proper place for discussing politics. So > can we please not do it? > > thanks, > > Norbert > > Am 22.12.2016 um 11:20 schrieb Dimitris Chloupis <kilon.alios(a)gmail.com>: > > I completely agree with you Phil, the real tragedy is this Berlin tragedy > will be the excuse to funnel billion of euros into a war that will bring > zero results , billions that could be invested not only to save 30 or even > a thousand lives but millions of lives. > > To talk about regression is to not be aware of how extremely lucky and > privileged we are to live in a time that experts call "the long peace" . > Never in the recorded human history there were so few wars worldwide and as > much poverty and it continuously gets better even in this time of global > economic crisis. > > As a matter of fact the biggest problems we face are the result of this > long peace and prosperity . Top ones being over population and climate > change. > > In both cases of positives and negative we experience the profound effects > of exponential growth. This crazy curve of constant growth that start small > and much later increases in insane rates which has massive positive and > negative effects. Its a crazy ride and will only get massively more crazy > soon enough, especially now with the coming of the AI revolution. > > I wonder the role Pharo will play in all this :) > > On Thu, Dec 22, 2016 at 1:56 AM phil(a)highoctane.be <phil(a)highoctane.be> > wrote: > > Regression? Consider that 1 human on 5 is under the poverty line vs 1 on 2 > not so many years ago and that a large percentage of the human population > has been schooled in elementary matters (again, not the case not so far in > the past). > > That is a massive improvement! > > Despite terrorist casualties, which compared to flu or car crashes, or > even toddlers shooting relatives with guns is two orders of magnitude less, > we live in the safest environment we have ever had. Not a regression at all. > > http://www.fayard.fr/histoire-du-terrorisme-9782213687308 has a pretty > useful background on their logic. Terrorism is the weapon of the weak in > terms of fighting force, otherwise they would engage in a full on war to > conquer the territory. > > The psychological impact is just out of proportions vs the number of > casualties. > > I got rammed into by a truck last week and could have been killed. That's > just a mundane occurence that statistically kills more people than any > terrorist attack would. > > Ref: https://fr.wikipedia.org/wiki/Accident_de_la_route_en_France "Et > depuis 1945, les chiffres sont aussi Ã©difiants, car il faut parler d'au > moins 500 000 morts, soit l'Ã©quivalent d'un grand conflit majeur, ou > l'Ã©quivalent des pertes humaines subies en France lors de la Seconde Guerre > mondiale." > > Not to mention domestic accidents: "Chaque annÃ©e en France, prÃ¨s de > 20 000 personnes dÃ©cÃ¨dent dans un accident domestique, c'est trois fois > plus que sur la route."" Plus de la moitiÃ© des accidents de la vie > courante concerne des enfants de 0 Ã 16 ans. Et malgrÃ© les progrÃ¨s > accomplis depuis les annÃ©es 80, c'est pour eux une des principales causes > de dÃ©cÃ¨s prÃ©maturÃ©s. Produits mÃ©nagers ou mÃ©dicaments accessibles, chaise > haute non sanglÃ©e, table Ã langer instable. en quelques minutes, la vie > peut tourner au drame. Il faut savoir que chez les tout-petits, c'est Ã > l'intÃ©rieur de la maison que se situe le danger" > > Best course of action is to keep on trucking and be a productive element > of society. No matter what. YOLO after all. > > Phil > > > > > > > On Wed, Dec 21, 2016 at 9:44 PM, stepharong <stepharong(a)free.fr> wrote: > > Indeed, immensely sad. Humanity is under regression. > When you see that in Mali they destroyed the oldest world library. > What sadden me the most is that in case of attacks in France, these > attacks were made by brainwashed french > citezen against french citizen contrary to what ugly "politician" would > like that we think. > > https://en.wikipedia.org/wiki/2016_Berlin_attack > > It is really sad that these things happen, that so many innocent people > are killed and hurt. It makes me even more sad that I seem to be getting > used to this, that should not be the case, at all. > > Sven > > > > > -- > Using Opera's mail client: http://www.opera.com/mail/ > > > >

Dec. 22, 2016

Re: [Pharo-dev] Berlin 2016-12-19

by Norbert Hartl

I don't think this list is the proper place for discussing politics. So can we please not do it? thanks, Norbert > Am 22.12.2016 um 11:20 schrieb Dimitris Chloupis <kilon.alios(a)gmail.com>: > > I completely agree with you Phil, the real tragedy is this Berlin tragedy will be the excuse to funnel billion of euros into a war that will bring zero results , billions that could be invested not only to save 30 or even a thousand lives but millions of lives. > > To talk about regression is to not be aware of how extremely lucky and privileged we are to live in a time that experts call "the long peace" . Never in the recorded human history there were so few wars worldwide and as much poverty and it continuously gets better even in this time of global economic crisis. > > As a matter of fact the biggest problems we face are the result of this long peace and prosperity . Top ones being over population and climate change. > > In both cases of positives and negative we experience the profound effects of exponential growth. This crazy curve of constant growth that start small and much later increases in insane rates which has massive positive and negative effects. Its a crazy ride and will only get massively more crazy soon enough, especially now with the coming of the AI revolution. > > I wonder the role Pharo will play in all this :) > > On Thu, Dec 22, 2016 at 1:56 AM phil(a)highoctane.be <mailto:phil@highoctane.be> <phil(a)highoctane.be <mailto:phil@highoctane.be>> wrote: > Regression? Consider that 1 human on 5 is under the poverty line vs 1 on 2 not so many years ago and that a large percentage of the human population has been schooled in elementary matters (again, not the case not so far in the past). > > That is a massive improvement! > > Despite terrorist casualties, which compared to flu or car crashes, or even toddlers shooting relatives with guns is two orders of magnitude less, we live in the safest environment we have ever had. Not a regression at all. > > http://www.fayard.fr/histoire-du-terrorisme-9782213687308 <http://www.fayard.fr/histoire-du-terrorisme-9782213687308> has a pretty useful background on their logic. Terrorism is the weapon of the weak in terms of fighting force, otherwise they would engage in a full on war to conquer the territory. > > The psychological impact is just out of proportions vs the number of casualties. > > I got rammed into by a truck last week and could have been killed. That's just a mundane occurence that statistically kills more people than any terrorist attack would. > > Ref: https://fr.wikipedia.org/wiki/Accident_de_la_route_en_France <https://fr.wikipedia.org/wiki/Accident_de_la_route_en_France> "Et depuis 1945, les chiffres sont aussi Ã©difiants, car il faut parler d'au moins 500 000 morts, soit l'Ã©quivalent d'un grand conflit majeur, ou l'Ã©quivalent des pertes humaines subies en France lors de la Seconde Guerre mondiale." > > Not to mention domestic accidents: "Chaque annÃ©e en France, prÃ¨s de 20 000 personnes dÃ©cÃ¨dent dans un accident domestique, c'est trois fois plus que sur la route."" Plus de la moitiÃ© des accidents de la vie courante concerne des enfants de 0 Ã 16 ans. Et malgrÃ© les progrÃ¨s accomplis depuis les annÃ©es 80, c'est pour eux une des principales causes de dÃ©cÃ¨s prÃ©maturÃ©s. Produits mÃ©nagers ou mÃ©dicaments accessibles, chaise haute non sanglÃ©e, table Ã langer instable. en quelques minutes, la vie peut tourner au drame. Il faut savoir que chez les tout-petits, c'est Ã l'intÃ©rieur de la maison que se situe le danger" > > Best course of action is to keep on trucking and be a productive element of society. No matter what. YOLO after all. > > Phil > > > > > > > On Wed, Dec 21, 2016 at 9:44 PM, stepharong <stepharong(a)free.fr <mailto:stepharong@free.fr>> wrote: > Indeed, immensely sad. Humanity is under regression. > When you see that in Mali they destroyed the oldest world library. > What sadden me the most is that in case of attacks in France, these attacks were made by brainwashed french > citezen against french citizen contrary to what ugly "politician" would like that we think. > > https://en.wikipedia.org/wiki/2016_Berlin_attack <https://en.wikipedia.org/wiki/2016_Berlin_attack> > > It is really sad that these things happen, that so many innocent people are killed and hurt. It makes me even more sad that I seem to be getting used to this, that should not be the case, at all. > > Sven > > > > > -- > Using Opera's mail client: http://www.opera.com/mail/ <http://www.opera.com/mail/> > >

Dec. 22, 2016

Re: [Pharo-dev] Berlin 2016-12-19

by Dimitris Chloupis

I completely agree with you Phil, the real tragedy is this Berlin tragedy will be the excuse to funnel billion of euros into a war that will bring zero results , billions that could be invested not only to save 30 or even a thousand lives but millions of lives. To talk about regression is to not be aware of how extremely lucky and privileged we are to live in a time that experts call "the long peace" . Never in the recorded human history there were so few wars worldwide and as much poverty and it continuously gets better even in this time of global economic crisis. As a matter of fact the biggest problems we face are the result of this long peace and prosperity . Top ones being over population and climate change. In both cases of positives and negative we experience the profound effects of exponential growth. This crazy curve of constant growth that start small and much later increases in insane rates which has massive positive and negative effects. Its a crazy ride and will only get massively more crazy soon enough, especially now with the coming of the AI revolution. I wonder the role Pharo will play in all this :) On Thu, Dec 22, 2016 at 1:56 AM phil(a)highoctane.be <phil(a)highoctane.be> wrote: > Regression? Consider that 1 human on 5 is under the poverty line vs 1 on 2 > not so many years ago and that a large percentage of the human population > has been schooled in elementary matters (again, not the case not so far in > the past). > > That is a massive improvement! > > Despite terrorist casualties, which compared to flu or car crashes, or > even toddlers shooting relatives with guns is two orders of magnitude less, > we live in the safest environment we have ever had. Not a regression at all. > > http://www.fayard.fr/histoire-du-terrorisme-9782213687308 has a pretty > useful background on their logic. Terrorism is the weapon of the weak in > terms of fighting force, otherwise they would engage in a full on war to > conquer the territory. > > The psychological impact is just out of proportions vs the number of > casualties. > > I got rammed into by a truck last week and could have been killed. That's > just a mundane occurence that statistically kills more people than any > terrorist attack would. > > Ref: https://fr.wikipedia.org/wiki/Accident_de_la_route_en_France "Et > depuis 1945, les chiffres sont aussi Ã©difiants, car il faut parler d'au > moins 500 000 morts, soit l'Ã©quivalent d'un grand conflit majeur, ou > l'Ã©quivalent des pertes humaines subies en France lors de la Seconde Guerre > mondiale." > > Not to mention domestic accidents: "Chaque annÃ©e en France, prÃ¨s de > 20 000 personnes dÃ©cÃ¨dent dans un accident domestique, c'est trois fois > plus que sur la route."" Plus de la moitiÃ© des accidents de la vie > courante concerne des enfants de 0 Ã 16 ans. Et malgrÃ© les progrÃ¨s > accomplis depuis les annÃ©es 80, c'est pour eux une des principales causes > de dÃ©cÃ¨s prÃ©maturÃ©s. Produits mÃ©nagers ou mÃ©dicaments accessibles, chaise > haute non sanglÃ©e, table Ã langer instable. en quelques minutes, la vie > peut tourner au drame. Il faut savoir que chez les tout-petits, c'est Ã > l'intÃ©rieur de la maison que se situe le danger" > > Best course of action is to keep on trucking and be a productive element > of society. No matter what. YOLO after all. > > Phil > > > > > > > On Wed, Dec 21, 2016 at 9:44 PM, stepharong <stepharong(a)free.fr> wrote: > > Indeed, immensely sad. Humanity is under regression. > When you see that in Mali they destroyed the oldest world library. > What sadden me the most is that in case of attacks in France, these > attacks were made by brainwashed french > citezen against french citizen contrary to what ugly "politician" would > like that we think. > > https://en.wikipedia.org/wiki/2016_Berlin_attack > > It is really sad that these things happen, that so many innocent people > are killed and hurt. It makes me even more sad that I seem to be getting > used to this, that should not be the case, at all. > > Sven > > > > > -- > Using Opera's mail client: http://www.opera.com/mail/ > > >

Dec. 22, 2016

Re: [Pharo-dev] Berlin 2016-12-19

by phil＠highoctane.be

Regression? Consider that 1 human on 5 is under the poverty line vs 1 on 2 not so many years ago and that a large percentage of the human population has been schooled in elementary matters (again, not the case not so far in the past). That is a massive improvement! Despite terrorist casualties, which compared to flu or car crashes, or even toddlers shooting relatives with guns is two orders of magnitude less, we live in the safest environment we have ever had. Not a regression at all. http://www.fayard.fr/histoire-du-terrorisme-9782213687308 has a pretty useful background on their logic. Terrorism is the weapon of the weak in terms of fighting force, otherwise they would engage in a full on war to conquer the territory. The psychological impact is just out of proportions vs the number of casualties. I got rammed into by a truck last week and could have been killed. That's just a mundane occurence that statistically kills more people than any terrorist attack would. Ref: https://fr.wikipedia.org/wiki/Accident_de_la_route_en_France "Et depuis 1945, les chiffres sont aussi Ã©difiants, car il faut parler d'au moins 500 000 morts, soit l'Ã©quivalent d'un grand conflit majeur, ou l'Ã©quivalent des pertes humaines subies en France lors de la Seconde Guerre mondiale." Not to mention domestic accidents: "Chaque annÃ©e en France, prÃ¨s de 20 000 personnes dÃ©cÃ¨dent dans un accident domestique, c'est trois fois plus que sur la route."" Plus de la moitiÃ© des accidents de la vie courante concerne des enfants de 0 Ã 16 ans. Et malgrÃ© les progrÃ¨s accomplis depuis les annÃ©es 80, c'est pour eux une des principales causes de dÃ©cÃ¨s prÃ©maturÃ©s. Produits mÃ©nagers ou mÃ©dicaments accessibles, chaise haute non sanglÃ©e, table Ã langer instable. en quelques minutes, la vie peut tourner au drame. Il faut savoir que chez les tout-petits, c'est Ã l'intÃ©rieur de la maison que se situe le danger" Best course of action is to keep on trucking and be a productive element of society. No matter what. YOLO after all. Phil On Wed, Dec 21, 2016 at 9:44 PM, stepharong <stepharong(a)free.fr> wrote: > Indeed, immensely sad. Humanity is under regression. > When you see that in Mali they destroyed the oldest world library. > What sadden me the most is that in case of attacks in France, these > attacks were made by brainwashed french > citezen against french citizen contrary to what ugly "politician" would > like that we think. > > https://en.wikipedia.org/wiki/2016_Berlin_attack >> >> It is really sad that these things happen, that so many innocent people >> are killed and hurt. It makes me even more sad that I seem to be getting >> used to this, that should not be the case, at all. >> >> Sven >> >> >> > > -- > Using Opera's mail client: http://www.opera.com/mail/ > >

Dec. 21, 2016

Re: [Pharo-dev] MQTT for Pharo

by Sven Van Caekenberghe

> On 21 Dec 2016, at 18:19, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> >> On 21 Dec 2016, at 16:52 , Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >>> >>> On 21 Dec 2016, at 16:04, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>> >>>> >>>> On 21 Dec 2016, at 14:57 , Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hendrik, >>>> >>>> Thank you for this detailed feedback. >>>> >>>>> On 21 Dec 2016, at 12:44, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>>>> >>>>> Hi Sven! >>>>> One thing I noticed when testing the RabbitMQ client with keepalive > 0, was connection being closed and all subscriptions lost when receiving large payloads, due to timeout before the keepalive packet could be sent before waiting to receive next object. >>>> >>>> Indeed, I used my Stamp (STOMP) Rabbit MQ client as a model for the MQTT one - there is a lot of similarity, especially in concept. >>>> >>>> Keep alive processing is not that easy. I tried to do it by using read timeouts as a source of regular opportunities to check for the need to process keep alive logic. But of course, if you have no outstanding read (in a loop), that won't work. >>>> >>>> The fact that receiving a large payload would trigger an actual keep alive time out is not something that I have seen myself. It seems weird that the reading/transferring of incoming data would not count as activity against keep alive, no ? >>> >>> I never had a chance to investigate fully, but I distinctly remember having the same reaction! >>> It's quite awhile ago now, so my memory might be hazy, take the following with an appropriate amount of grains of salt. >>> >>> The first times I encountered it, it seemed quite random, occuring after extended periods of client inactivity after receiving only small payloads... >>> Setting a much shorter keepalive timeout than the default was/is very useful in reproducing/verifying if it is an issue. >>> The timeouts then occurred relatively shortly after I'd received a single payload with no other activity, and disappeared once I removed the resetting of lastActivity timestamp on reads, indicating that for at least rabbitmq (3.5 was the version at the time, I believe), receiving data was *not* being counted as keep-alive activity. >>> >>> The issue of receiving large payloads blocking writing in time was still unresolved, consistently cut off in the middle of (its own!) multi-MB payload transfers due to keepalive packet not being sent. >>> I couldn't see a solution other than abandoning the elegant single-threaded approach and do keepalive as a separate high-priority process, but the architectural choice for the app changed at this point to not include a MQ in first delivery, so the more involved rewrite for doing so before deploying in production, kinda got stranded :/ >> >> Instinctively it feels like messaging and multi-MB payloads would not be a good fit, at least I would suspect that they are an edge case. But maybe I am wrong. >> >> I think that the code in StampMedium>>#readBodyBytes: and StampMedium>>#readBodyString: could be refactored to use a loop with chunk buffers and at the same time check the elapsed time to fire back keep alive pings if necessary. A ping too much would not hurt, better safe than sorry. But is is hard to catch any/all network slowdowns, any IO operation could hang/timeout. >> >> But all this would assume that the problematic situation can be recreated. > > Absolutely, far from certain the MQTT server implementations work the same. > > This is the test code I used for rabbit: > > consumer1 := StampClient new > timeout:1; > heartbeat: 5000; > login: 'guest'; > passcode: 'guest'; > open; > subscribeTo: '/exchange/testall'; > yourself. > [[ consumer1 runWith:[:message | Transcript crShow: '1 read: ', message body]] ensure: [consumer1 close]] forkAt: Processor userInterruptPriority . > > producer := StampClient new > login: 'guest'; > passcode: 'guest'; > open; yourself. > > [1 to: 10000 do: [:i | producer sendText: i asString to: '/exchange/testall']] forkAt: Processor systemBackgroundPriority. > > Sometimes, but not always, this would fail, either during processing, or some time after. > > A screenshot of how it looks when a disconnect happens during processing due to heartbeat failing can be found at: > https://tresor.it/s#HN4RG9PEX1SYyKy4NcM31w I installed RabbitMQ 3.6.4 on my Mac and tried your example and it succeeded ;-) But I modified it slightly so that both threads have a stop condition (I also used /queue instead of /exchange but I don't know if that makes a difference). limit := 10000. consumer1 := StampClient new timeout:1; heartbeat: 5000; login: 'guest'; passcode: 'guest'; open; subscribeTo: '/queue/testall'; yourself. producer := StampClient new login: 'guest'; passcode: 'guest'; open; yourself. [ [ consumer1 runWith: [ :message | Transcript crShow: '1 read: ', message body. message body = limit asString ifTrue: [ ConnectionClosed signal ] ] ] ensure: [ consumer1 close ] ] forkAt: Processor userInterruptPriority. [ 1 to: limit do: [ :i | producer sendText: i asString to: '/queue/testall' ]. producer close ] forkAt: Processor systemBackgroundPriority. You did see the #testSimpleWorkQueue unit test ? And the StampBenchmarkTests ? You could also try running StampBenchmark manually with higher loads. What I mean is that I normally use the #writeWithReceipt: and #clientIndividualAck subscriptions with explicit ack/nack responses. The reason is that the way you do it (although allowed and the fastest way), there is no traffic flow regulation: the producer basically sends everything in one big go (because it does not wait for receipts from the server), the consumer gets all messages in one big batch from the broker - socket stream buffers at different levels will just fill up. All this does not yet explain your problem, I know. I also did a checkin because I still had this modification in my image: === Name: Stamp-SvenVanCaekenberghe.24 Author: SvenVanCaekenberghe Time: 21 December 2016, 9:54:27.217571 pm UUID: ee3a7276-6e00-4143-9c3e-03a320c14ea6 Ancestors: Stamp-HolgerHansPeterFreyther.23 StampClient - minor refactoring in #writeHeartbeatIfNeeded (#wasInactive ForTooLong) - add #touch call to #writeNoFlush: === Just to make sure we're on the same version (I am still using Pharo 4 for this, BTW). > Cheers, > Henry

Dec. 21, 2016

Re: [Pharo-dev] Berlin 2016-12-19

by stepharong

Indeed, immensely sad. Humanity is under regression. When you see that in Mali they destroyed the oldest world library. What sadden me the most is that in case of attacks in France, these attacks were made by brainwashed french citezen against french citizen contrary to what ugly "politician" would like that we think. > https://en.wikipedia.org/wiki/2016_Berlin_attack > > It is really sad that these things happen, that so many innocent people > are killed and hurt. It makes me even more sad that I seem to be getting > used to this, that should not be the case, at all. > > Sven > > -- Using Opera's mail client: http://www.opera.com/mail/

Dec. 21, 2016

Re: [Pharo-dev] MQTT for Pharo

by Henrik Johansen

> On 21 Dec 2016, at 16:52 , Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> >> On 21 Dec 2016, at 16:04, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> >>> On 21 Dec 2016, at 14:57 , Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hendrik, >>> >>> Thank you for this detailed feedback. >>> >>>> On 21 Dec 2016, at 12:44, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>>> >>>> Hi Sven! >>>> One thing I noticed when testing the RabbitMQ client with keepalive > 0, was connection being closed and all subscriptions lost when receiving large payloads, due to timeout before the keepalive packet could be sent before waiting to receive next object. >>> >>> Indeed, I used my Stamp (STOMP) Rabbit MQ client as a model for the MQTT one - there is a lot of similarity, especially in concept. >>> >>> Keep alive processing is not that easy. I tried to do it by using read timeouts as a source of regular opportunities to check for the need to process keep alive logic. But of course, if you have no outstanding read (in a loop), that won't work. >>> >>> The fact that receiving a large payload would trigger an actual keep alive time out is not something that I have seen myself. It seems weird that the reading/transferring of incoming data would not count as activity against keep alive, no ? >> >> I never had a chance to investigate fully, but I distinctly remember having the same reaction! >> It's quite awhile ago now, so my memory might be hazy, take the following with an appropriate amount of grains of salt. >> >> The first times I encountered it, it seemed quite random, occuring after extended periods of client inactivity after receiving only small payloads... >> Setting a much shorter keepalive timeout than the default was/is very useful in reproducing/verifying if it is an issue. >> The timeouts then occurred relatively shortly after I'd received a single payload with no other activity, and disappeared once I removed the resetting of lastActivity timestamp on reads, indicating that for at least rabbitmq (3.5 was the version at the time, I believe), receiving data was *not* being counted as keep-alive activity. >> >> The issue of receiving large payloads blocking writing in time was still unresolved, consistently cut off in the middle of (its own!) multi-MB payload transfers due to keepalive packet not being sent. >> I couldn't see a solution other than abandoning the elegant single-threaded approach and do keepalive as a separate high-priority process, but the architectural choice for the app changed at this point to not include a MQ in first delivery, so the more involved rewrite for doing so before deploying in production, kinda got stranded :/ > > Instinctively it feels like messaging and multi-MB payloads would not be a good fit, at least I would suspect that they are an edge case. But maybe I am wrong. > > I think that the code in StampMedium>>#readBodyBytes: and StampMedium>>#readBodyString: could be refactored to use a loop with chunk buffers and at the same time check the elapsed time to fire back keep alive pings if necessary. A ping too much would not hurt, better safe than sorry. But is is hard to catch any/all network slowdowns, any IO operation could hang/timeout. > > But all this would assume that the problematic situation can be recreated. Absolutely, far from certain the MQTT server implementations work the same. This is the test code I used for rabbit: consumer1 := StampClient new timeout:1; heartbeat: 5000; login: 'guest'; passcode: 'guest'; open; subscribeTo: '/exchange/testall'; yourself. [[ consumer1 runWith:[:message | Transcript crShow: '1 read: ', message body]] ensure: [consumer1 close]] forkAt: Processor userInterruptPriority . producer := StampClient new login: 'guest'; passcode: 'guest'; open; yourself. [1 to: 10000 do: [:i | producer sendText: i asString to: '/exchange/testall']] forkAt: Processor systemBackgroundPriority. Sometimes, but not always, this would fail, either during processing, or some time after. A screenshot of how it looks when a disconnect happens during processing due to heartbeat failing can be found at: https://tresor.it/s#HN4RG9PEX1SYyKy4NcM31w Cheers, Henry

Dec. 21, 2016

Berlin 2016-12-19

by Sven Van Caekenberghe

https://en.wikipedia.org/wiki/2016_Berlin_attack It is really sad that these things happen, that so many innocent people are killed and hurt. It makes me even more sad that I seem to be getting used to this, that should not be the case, at all. Sven

Dec. 21, 2016

Re: [Pharo-dev] MQTT for Pharo

by Sven Van Caekenberghe

> On 21 Dec 2016, at 15:47, Ben Coman <btc(a)openinworld.com> wrote: > > You might like to collaborate with Tim and Craig. > http://forum.world.st/Graphing-weather-data-td4927701.html > cheers -ben Hmm, that code seems to be quite incomplete (from what I can see publicly): not all packet types and their IO is fully implemented, there are no unit tests, there is no client, and none of the more complex quality of service levels 0, 1 & 2 logic/flow, and there are no functional unit tests against public sandbox/test servers. > On Wed, Dec 21, 2016 at 10:17 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >>> On 21 Dec 2016, at 15:11, phil(a)highoctane.be wrote: >>> >>> FWIW get yourself an account at http://enco.io >> >> OK, I'll put it on my list of things to look at. >> >>> The devportal has mqtt endpoints so that we can integrate with that and possibly have some market place solutions offered and running with Pharo. >>> >>> I have a small EnCoClient class in Pharo that works. Now making the API support more extensive. >>> >>> Docs: http://docs.enco.io/docs/device-sensor-apis >>> >>> Phil >>> >>> On Wed, Dec 21, 2016 at 2:59 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>>> On 21 Dec 2016, at 12:50, phil(a)highoctane.be wrote: >>>> >>>> Sven, >>>> >>>> I am currently working with IoT and MQTT is necessary. At this point I use mosquitto as the broker and your Pharo client is nice to have and I will test it and give you feedback. >>> >>> That is good to here, Phil. >>> >>> Yes, such real world testing is much needed. This is actually quite fun stuff to experiment with. Let me know how it goes. >>> >>> Sven >>> >>>> I've got a working microPython codebase so can compare what is in there (this for the LoPy device): >>>> >>>> Thx for the module! >>>> >>>> Phil >>>> >>>> On Wed, Dec 21, 2016 at 12:15 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> Hi, >>>> >>>> MQTT is a light-weight publish/subscribe messaging protocol, originally created around 1998. It is now an official open industry ISO standard. It is perfect for large-scale Internet of Things applications and high performance mobile messaging. >>>> >>>> The publish/subscribe messaging pattern requires a message broker. The broker is responsible for distributing messages to interested clients based on the topic of a message. Parties communicating with each other over MQTT would all be clients in different roles, like producers and consumers, using the broker as middleware. >>>> >>>> Many client libraries for different programming languages and multiple brokers/servers are available. Facebook Messenger and Amazon AWS IOT are two users, among many others. >>>> >>>> >>>> A good client library for Pharo was not yet available. I started a new MQTT project and I am looking for collaborators to help me finish it. The official specification is quite readable and there is a lot of information available (see the References/Links section at the end). >>>> >>>> >>>> Right now, a first version of the following features is available: >>>> >>>> - reading & writing of all 14 binary packet types >>>> >>>> - an experimental client with support for connection open/close, ping, subscribe/unsubscribe, QoS levels 0 (at most once), 1 (at least once) and 2 (exactly once) for application (publish) messages in both directions, message/package IDs and keep alive (heartbeat) >>>> >>>> - unit tests, for packet reading/writing and for clients against 3 publicly available sandbox/test brokers >>>> >>>> Basically, the code works but needs polishing and maturing. Also, it might be useful to experiment with alternative client API design. Not all features are yet implemented. It would also be nice to implement an actual server/broker, not to replace production quality servers, but as a proof of concept and tools during development. >>>> >>>> >>>> Code >>>> >>>> http://smalltalkhub.com/#!/~SvenVanCaekenberghe/MQTT/ >>>> >>>> Right now, documentation is limited, but there are class comments and the most important public API methods are commented too. There is no Metacello configuration yet, just load the 3 packages. >>>> >>>> >>>> References/Links >>>> >>>> - http://mqtt.org >>>> - https://en.wikipedia.org/wiki/MQTT >>>> - http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html >>>> - https://github.com/mqtt/mqtt.github.io/wiki/software >>>> - http://mosquitto.org >>>> - https://github.com/emqtt/emqttd >>>> - http://kamilfb.github.io/mqtt-spy/ >>>> - https://github.com/eclipse/paho.mqtt-spy/wiki >>>> - https://eclipse.org/paho/ >>>> - https://eclipse.org/paho/clients/c/embedded/ >>>> - http://www.rabbitmq.com/mqtt.html >>>> - https://vernemq.com >>>> - https://www.ibm.com/developerworks/community/blogs/c565c720-fe84-4f63-873f-… >>>> >>>> >>>> Sandbox/test Servers/Brokers >>>> >>>> iot.eclipse.org:1883 >>>> test.mosquitto.org:1883 >>>> broker.mqtt-dashboard.com:1883 >>>> >>>> >>>> Sven >>>> >>>> -- >>>> Sven Van Caekenberghe >>>> http://stfx.eu >>>> Smalltalk is the Red Pill >>>> >>>> >>>> >>> >>> >>> >> >> >

Dec. 21, 2016

Re: [Pharo-dev] MQTT for Pharo

by Sven Van Caekenberghe

> On 21 Dec 2016, at 16:04, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> >> On 21 Dec 2016, at 14:57 , Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Hendrik, >> >> Thank you for this detailed feedback. >> >>> On 21 Dec 2016, at 12:44, Henrik Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>> >>> Hi Sven! >>> One thing I noticed when testing the RabbitMQ client with keepalive > 0, was connection being closed and all subscriptions lost when receiving large payloads, due to timeout before the keepalive packet could be sent before waiting to receive next object. >> >> Indeed, I used my Stamp (STOMP) Rabbit MQ client as a model for the MQTT one - there is a lot of similarity, especially in concept. >> >> Keep alive processing is not that easy. I tried to do it by using read timeouts as a source of regular opportunities to check for the need to process keep alive logic. But of course, if you have no outstanding read (in a loop), that won't work. >> >> The fact that receiving a large payload would trigger an actual keep alive time out is not something that I have seen myself. It seems weird that the reading/transferring of incoming data would not count as activity against keep alive, no ? > > I never had a chance to investigate fully, but I distinctly remember having the same reaction! > It's quite awhile ago now, so my memory might be hazy, take the following with an appropriate amount of grains of salt. > > The first times I encountered it, it seemed quite random, occuring after extended periods of client inactivity after receiving only small payloads... > Setting a much shorter keepalive timeout than the default was/is very useful in reproducing/verifying if it is an issue. > The timeouts then occurred relatively shortly after I'd received a single payload with no other activity, and disappeared once I removed the resetting of lastActivity timestamp on reads, indicating that for at least rabbitmq (3.5 was the version at the time, I believe), receiving data was *not* being counted as keep-alive activity. > > The issue of receiving large payloads blocking writing in time was still unresolved, consistently cut off in the middle of (its own!) multi-MB payload transfers due to keepalive packet not being sent. > I couldn't see a solution other than abandoning the elegant single-threaded approach and do keepalive as a separate high-priority process, but the architectural choice for the app changed at this point to not include a MQ in first delivery, so the more involved rewrite for doing so before deploying in production, kinda got stranded :/ Instinctively it feels like messaging and multi-MB payloads would not be a good fit, at least I would suspect that they are an edge case. But maybe I am wrong. I think that the code in StampMedium>>#readBodyBytes: and StampMedium>>#readBodyString: could be refactored to use a loop with chunk buffers and at the same time check the elapsed time to fire back keep alive pings if necessary. A ping too much would not hurt, better safe than sorry. But is is hard to catch any/all network slowdowns, any IO operation could hang/timeout. But all this would assume that the problematic situation can be recreated. > Cheers, > Henry

Dec. 21, 2016