March 2019 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Resolving DNS in-image

by Ben Coman

On Fri, 29 Mar 2019 at 18:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Holger, > > I'll write most of my comments inline. > > Yesterday I moved my code to https://github.com/svenvc/NeoDNS using Tonel > format, to make it a bit easier to consume. I also did a couple of minor > updates while going over the functionality. Note the 'my code' is in the > package Net-Protocols-DNS-Experimental, which is using the object model in > 'Net-Protocols-DNS-MessageFormat' (that already existed before, I just > ported/cleaned it a little bit to my taste). > > Are you coming to the Pharo Days ? We could talk there. > > (More below) > > > On 28 Mar 2019, at 09:29, Holger Freyther <holger(a)freyther.de> wrote: > > > > > > > >> On 28. Mar 2019, at 08:02, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> > >> Hi Holger & Norbert, > >> > > > > great. Regardless of how many versions exist. We should get one into the > image with proper platform integration. > > > > I wasn't aware of your code but I assumed it is something you could > write, hence the Paleo prefix. Now that the Paleo code is the Neo one is > more funny... > > Yeah, naming is always fun, isn't it ? > > >> NeoSimplifiedDNSClient default addressForName: 'pharo.org'. > "104.28.27.35" > >> > >> One of my goals was to use it as a more reliable, non-blocking 'do we > have internet access' test: > >> > >> NeoNetworkState default hasInternetConnection. "true" > > > > > > What is internet access and how would this be used? Is this about > captive portals? With local network policy the big anycast services might > be blocked but the user can still reach services. Or with deployed > microservices they might reach other but not the outside? > > For years there is this issue in Pharo that if we build features that > require internet access (say for example automatic loading of the Catalog > when you start using Spotter, but there are many more places where this > could add lots of value), that people say "don't do this, because it won't > work when I have slow or no internet (like on a train)". > > The core cause of the problems is that the current NameResolver is totally > blocking, especially in failure cases, which gives a terrible experience. > Not that we want the everything in the kitchen sink in Pharo, but DNS is PERVASIVE!!! :) A lot of experimentation with new services goes on with RR records such that having a "live" DNS implementation would facilitate Pharo being a platform to play with leading edge services. Can we discuss putting an in-Image DNS on the Pharo 8 roadmap? First consideration - what are the argument against this? Second consideration - who would volunteer for a working party to make this happen? btw, years ago I found the BIND book really entertaining reading... http://shop.oreilly.com/product/9780596100575.do > > > Agreed that getting the OS DNS settings is hard but not impossible (go > seems to get away with its implementation on unix). It seems we manage to > honor platform settings for http proxies and I am confident we can do it > for DNS as well. > > I hacked a #getSystemDNS class method that goes via /etc/resolv.conf and > it works (but of course not on Windows). In the 'initialise' protocol, > there are #useCloudflareDNS, #useGoogleDNS, #useSystemDNS as well as > #useMulticastDNS methods. > > I would *very* much prefer not to depend on any obscure, hard to maintain > VM code (FFI would just be acceptable). > > > I planned to solve concurrency by creating one stub resolver per request > and having a shared cache. The internet can be a hostile place and DNS is > an easy victim. Cache poisoning does exist and random source port, 0x20 > randomization, random transaction ids, disrespecting PTMU ICMP messages are > the few mitigations we have. > > What I tried/implemented in NeoDNSClient (which inherits from the one-shot > NeoSimplifiedDNSClient) is a requestQueue and a cache (respecting ttl), > where clients put a request on the requestQueue and wait on a semaphore > inside the request (respecting their call's timeout). A single process > (that starts/stops as needed) handles the sending & receiving of the actual > protocol, signalling the matching request's semaphore. (The #beThreadSafe > option needs a bit more work though). > > But yes, ultimately, things might be more complex. There is of course a > problem in doing in-image DNS: it creates a (possibly) huge maintenance > burden with a lot of responsibility. Now, since we already do socket > streams, TLS/SSL and HTTP, the risk/challenge is probably acceptable. > The maintenance burden has been the argument against in the past. But the opportunity of "liveness" may be worth bit. My presumption is the main part of DNS changes slowly, so the big question for me is how feasible is it to fall back to the C-library for things the in-Image stuff doesn't handle. cheers -ben

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Esteban Maringolo

I would do it as an iterator (a circular one). The structure of the underlying list is irrelevant here (as long as it is sequenceable), IMO. Regards, El vie., 29 de mar. de 2019 07:21, Peter Kenny <peter(a)pbkresearch.co.uk> escribiÃ³: > Tim > > But the beauty of Smalltalk is that, if you need to use a particular > structure frequently, you can make it an object just by subclassing > something useful. In this case, you could create CircularList by > subclassing > SequenceableCollection and re-implementing #before and #after (and maybe > tidying up #add if you don't want the circle to be extendable). > > Pursuing this line of thought, we already have one specialized subclass in > LinkedList. If the last element is linked to the first, this becomes > circular. I glanced at some of the methods to see how well this would work, > but because it regards links as separate from the linked objects, it got > too > complicated. But it could give a starting point if you have lots of such > cases. > > HTH > > Peter Kenny > > > Tim Mackinnon wrote > > Hey thanks guys - while it certainly makes sense when you think about it, > > I was kind of hoping we had something that was much more readable and > > obvious. It seems strange that when we have lots of esoteric things in > > collection, that something which is quite common to do isnât there. I was > > kind of hoping we might have some circular list or something. > > > > As it stands, if you really want to communicate this clearly you are > > almost better off just doing, the more obvious: > > > > (Index := index - 1) = 0 ifTrue: [index := items size]. > > > > Anyway, an interesting one. > > > > Tim > > > >> On 28 Mar 2019, at 16:55, Peter Kenny < > > > peter@.co > > > > wrote: > >> > >> Tim > >> > >> I found myself puzzling as to *why* James's solution works. This longer > >> explanation helped me to understand. > >> > >> First, generalize moving forward to moving an arbitrary number of steps, > >> still with wrapping. > >> > >> ^list at: (index + move - 1 \\ size + 1 > >> > >> Second, realize that moving backward one step with wrapping is exactly > >> equivalent to moving forward (size - 1) steps - because of wrapping, a > >> move > >> of size steps is null. > >> > >> Finally, substitute move = size - 1. > >> > >> ^list at: (index + size - 2 \\ size + 1. > >> > >> This can of course easily generalize to backward moves of any size. > >> > >> HTH > >> > >> Peter Kenny > >> > >> > >> > >> -- > >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >> > > > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Tim Mackinnon

No problem Peter - Iâll raise a bug (as it happened, I had save my image a few hours before and epicea got back the few things I had changed since). Rather strangely - even just trying to debug âmyList nextâ (as in right click and pick debug it) seems to hang my image too (I didnât expect that). Tim > On 29 Mar 2019, at 11:36, Peter Kenny <peter(a)pbkresearch.co.uk> wrote: > > Hi Tim. Sorry you crashed your image. I owe you an apology, because when I > looked at the code for LinkedList, I thought there was something odd, but I > dismissed it thinking ' they must have thought of that!' I think the problem > is that LinkedList inherits #after: from SequenceableCollection, meaning it > uses #atIndex: rather than following the links. Even so I don't see why it > loops; surely going outside the range of the index should be a known error > condition. Anyway, between us we may have discovered a bug in LinkedList. Do > you feel stirred up enough to raise it as an issue? > > Peter > > > Tim Mackinnon wrote >> Hi Peter - yes of course you are right that its malleable, I guess Iâm >> often suprised about the things we leave out and then discover weird >> things weâve put in. >> >> I had a quick look at LinkedList (I should have thought of that) - I might >> be able to do something with that (and this is now more out of curiosity) >> - but gosh its an easy way to crash your image. Having linked the first to >> the last, and then tried a quick: list after: $n - Iâm now guessing Iâm >> stuck in an infinite loop that even Cmd . wonât break out of!!!! ouch. >> >> Tim >> >>> On 29 Mar 2019, at 10:20, Peter Kenny < > >> peter@.co > >> > wrote: >>> >> >>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >>> <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> > > > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >

March 29, 2019

Re: [Pharo-users] Text presenter autosave features

by Ben Coman

No guarantees, but it might help if you provided a detailed recipe for anyone to reproduce this. cheers -ben On Fri, 29 Mar 2019 at 00:24, Offray Vladimir Luna CÃ¡rdenas < offray.luna(a)mutabit.com> wrote: > Hi, > > Just a remainder if anyone can help with this... > > Thanks, > > Offray > > On 23/03/19 12:44 p. m., Offray Vladimir Luna CÃ¡rdenas wrote: > > Hi, > > > > While migrating Grafoscopio from Pharo 6.x to Pharo 7.x., I noticed > > that auto save for text fields is not enabled. Previously, when I > > wrote something in a text field, the dirty mark in the upper right > > corner appeared and if autoAccept was enabled, the text was > > automatically saved. Now with Pharo 7.x the dirty mark don't appear at > > all, and is not making any auto-saving. > > > > * I see that TextPresenter is the one who is responsible for such text > > objects. There is any minimal example I can use to test the proper > > behavior? > > > > * There is any way to use WelcomeHelp class to get a similar behavior? > > > > Thanks, > > > > Offray > > > > > > > >

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Peter Kenny

Hi Tim. Sorry you crashed your image. I owe you an apology, because when I looked at the code for LinkedList, I thought there was something odd, but I dismissed it thinking ' they must have thought of that!' I think the problem is that LinkedList inherits #after: from SequenceableCollection, meaning it uses #atIndex: rather than following the links. Even so I don't see why it loops; surely going outside the range of the index should be a known error condition. Anyway, between us we may have discovered a bug in LinkedList. Do you feel stirred up enough to raise it as an issue? Peter Tim Mackinnon wrote > Hi Peter - yes of course you are right that its malleable, I guess Iâm > often suprised about the things we leave out and then discover weird > things weâve put in. > > I had a quick look at LinkedList (I should have thought of that) - I might > be able to do something with that (and this is now more out of curiosity) > - but gosh its an easy way to crash your image. Having linked the first to > the last, and then tried a quick: list after: $n - Iâm now guessing Iâm > stuck in an infinite loop that even Cmd . wonât break out of!!!! ouch. > > Tim > >> On 29 Mar 2019, at 10:20, Peter Kenny < > peter@.co > > wrote: >> > >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >> <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

March 29, 2019

Re: [Pharo-users] Resolving DNS in-image

by CÃ©drick BÃ©ler

> The core cause of the problems is that the current NameResolver is totally blocking, especially in failure cases, which gives a terrible experience. > > One way to fix this would be with the concept of NetworkState, a cheap, reliable, totally non-blocking way to test if the image has a working internet connection. Related is the option of 'Airplane Mode', so that you can manually say: "consider the internet unreachable". > > The main scope was real internet access, not (possibly) limited internal network access. But that is a good point to think about. I totally agree on this point (and thanks again to push such network reliable dev). For my project (where I consider Internet as an optional connexion, the less trusted one btw.), this is number one priority to be have a Â« non-blocking way to test if the image has a working internet connection Â». As I see it, but this is conceptual. To have Internet, I either use: 1) a LAN in between with a gateway (which provides a WAN service) 2) an internal modem (GSM for instance) I focus on 1) meaning if I (re) connect to a (know) LAN, I know if it provides a gateway. Actually the access point/switch is another node in the system. So I kind of get around the problem. Still having a nice OS integration is important especially with all network interfaces we have (and I think that was a limitation of SSDP if I remember correctly what Henrik said). So +100 to you initiative guys :) >> Let's definitely talk. I hang out in the pharo discord group. :) >> >> >> holger > > I am only an occasional IM users, I more of an email guy. Not an expert as you, but definitely interested in such discussions. PharoDays ? This could include pushing SSDP from Henrik and maybe integrating Nouryâs (and others) dev on NetworkExtra ? My 2 cents, CÃ©drick > > Sven > > >

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Tim Mackinnon

Hi Peter - yes of course you are right that its malleable, I guess Iâm often suprised about the things we leave out and then discover weird things weâve put in. I had a quick look at LinkedList (I should have thought of that) - I might be able to do something with that (and this is now more out of curiosity) - but gosh its an easy way to crash your image. Having linked the first to the last, and then tried a quick: list after: $n - Iâm now guessing Iâm stuck in an infinite loop that even Cmd . wonât break out of!!!! ouch. Tim > On 29 Mar 2019, at 10:20, Peter Kenny <peter(a)pbkresearch.co.uk> wrote: > > Tim > > But the beauty of Smalltalk is that, if you need to use a particular > structure frequently, you can make it an object just by subclassing > something useful. In this case, you could create CircularList by subclassing > SequenceableCollection and re-implementing #before and #after (and maybe > tidying up #add if you don't want the circle to be extendable). > > Pursuing this line of thought, we already have one specialized subclass in > LinkedList. If the last element is linked to the first, this becomes > circular. I glanced at some of the methods to see how well this would work, > but because it regards links as separate from the linked objects, it got too > complicated. But it could give a starting point if you have lots of such > cases. > > HTH > > Peter Kenny > > > Tim Mackinnon wrote >> Hey thanks guys - while it certainly makes sense when you think about it, >> I was kind of hoping we had something that was much more readable and >> obvious. It seems strange that when we have lots of esoteric things in >> collection, that something which is quite common to do isnât there. I was >> kind of hoping we might have some circular list or something. >> >> As it stands, if you really want to communicate this clearly you are >> almost better off just doing, the more obvious: >> >> (Index := index - 1) = 0 ifTrue: [index := items size]. >> >> Anyway, an interesting one. >> >> Tim >> >>> On 28 Mar 2019, at 16:55, Peter Kenny < > >> peter@.co > >> > wrote: >>> >>> Tim >>> >>> I found myself puzzling as to *why* James's solution works. This longer >>> explanation helped me to understand. >>> >>> First, generalize moving forward to moving an arbitrary number of steps, >>> still with wrapping. >>> >>> ^list at: (index + move - 1 \\ size + 1 >>> >>> Second, realize that moving backward one step with wrapping is exactly >>> equivalent to moving forward (size - 1) steps - because of wrapping, a >>> move >>> of size steps is null. >>> >>> Finally, substitute move = size - 1. >>> >>> ^list at: (index + size - 2 \\ size + 1. >>> >>> This can of course easily generalize to backward moves of any size. >>> >>> HTH >>> >>> Peter Kenny >>> >>> >>> >>> -- >>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >>> > > > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html>

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Peter Kenny

Tim But the beauty of Smalltalk is that, if you need to use a particular structure frequently, you can make it an object just by subclassing something useful. In this case, you could create CircularList by subclassing SequenceableCollection and re-implementing #before and #after (and maybe tidying up #add if you don't want the circle to be extendable). Pursuing this line of thought, we already have one specialized subclass in LinkedList. If the last element is linked to the first, this becomes circular. I glanced at some of the methods to see how well this would work, but because it regards links as separate from the linked objects, it got too complicated. But it could give a starting point if you have lots of such cases. HTH Peter Kenny Tim Mackinnon wrote > Hey thanks guys - while it certainly makes sense when you think about it, > I was kind of hoping we had something that was much more readable and > obvious. It seems strange that when we have lots of esoteric things in > collection, that something which is quite common to do isnât there. I was > kind of hoping we might have some circular list or something. > > As it stands, if you really want to communicate this clearly you are > almost better off just doing, the more obvious: > > (Index := index - 1) = 0 ifTrue: [index := items size]. > > Anyway, an interesting one. > > Tim > >> On 28 Mar 2019, at 16:55, Peter Kenny < > peter@.co > > wrote: >> >> Tim >> >> I found myself puzzling as to *why* James's solution works. This longer >> explanation helped me to understand. >> >> First, generalize moving forward to moving an arbitrary number of steps, >> still with wrapping. >> >> ^list at: (index + move - 1 \\ size + 1 >> >> Second, realize that moving backward one step with wrapping is exactly >> equivalent to moving forward (size - 1) steps - because of wrapping, a >> move >> of size steps is null. >> >> Finally, substitute move = size - 1. >> >> ^list at: (index + size - 2 \\ size + 1. >> >> This can of course easily generalize to backward moves of any size. >> >> HTH >> >> Peter Kenny >> >> >> >> -- >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >> -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

March 29, 2019

Re: [Pharo-users] Resolving DNS in-image

by Sven Van Caekenberghe

Holger, I'll write most of my comments inline. Yesterday I moved my code to https://github.com/svenvc/NeoDNS using Tonel format, to make it a bit easier to consume. I also did a couple of minor updates while going over the functionality. Note the 'my code' is in the package Net-Protocols-DNS-Experimental, which is using the object model in 'Net-Protocols-DNS-MessageFormat' (that already existed before, I just ported/cleaned it a little bit to my taste). Are you coming to the Pharo Days ? We could talk there. (More below) > On 28 Mar 2019, at 09:29, Holger Freyther <holger(a)freyther.de> wrote: > > > >> On 28. Mar 2019, at 08:02, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Hi Holger & Norbert, >> > > great. Regardless of how many versions exist. We should get one into the image with proper platform integration. > > I wasn't aware of your code but I assumed it is something you could write, hence the Paleo prefix. Now that the Paleo code is the Neo one is more funny... Yeah, naming is always fun, isn't it ? >> NeoSimplifiedDNSClient default addressForName: 'pharo.org'. "104.28.27.35" >> >> One of my goals was to use it as a more reliable, non-blocking 'do we have internet access' test: >> >> NeoNetworkState default hasInternetConnection. "true" > > > What is internet access and how would this be used? Is this about captive portals? With local network policy the big anycast services might be blocked but the user can still reach services. Or with deployed microservices they might reach other but not the outside? For years there is this issue in Pharo that if we build features that require internet access (say for example automatic loading of the Catalog when you start using Spotter, but there are many more places where this could add lots of value), that people say "don't do this, because it won't work when I have slow or no internet (like on a train)". The core cause of the problems is that the current NameResolver is totally blocking, especially in failure cases, which gives a terrible experience. One way to fix this would be with the concept of NetworkState, a cheap, reliable, totally non-blocking way to test if the image has a working internet connection. Related is the option of 'Airplane Mode', so that you can manually say: "consider the internet unreachable". The main scope was real internet access, not (possibly) limited internal network access. But that is a good point to think about. > ... snip ... > > >> The main problems are concurrent and asynchronous requests, as well as error handling. >> >> I would be great if we could do this well in-image. (But getting OS DNS settings is hard too). >> >> We should talk ;-) > > Agreed that getting the OS DNS settings is hard but not impossible (go seems to get away with its implementation on unix). It seems we manage to honor platform settings for http proxies and I am confident we can do it for DNS as well. I hacked a #getSystemDNS class method that goes via /etc/resolv.conf and it works (but of course not on Windows). In the 'initialise' protocol, there are #useCloudflareDNS, #useGoogleDNS, #useSystemDNS as well as #useMulticastDNS methods. I would *very* much prefer not to depend on any obscure, hard to maintain VM code (FFI would just be acceptable). > I planned to solve concurrency by creating one stub resolver per request and having a shared cache. The internet can be a hostile place and DNS is an easy victim. Cache poisoning does exist and random source port, 0x20 randomization, random transaction ids, disrespecting PTMU ICMP messages are the few mitigations we have. What I tried/implemented in NeoDNSClient (which inherits from the one-shot NeoSimplifiedDNSClient) is a requestQueue and a cache (respecting ttl), where clients put a request on the requestQueue and wait on a semaphore inside the request (respecting their call's timeout). A single process (that starts/stops as needed) handles the sending & receiving of the actual protocol, signalling the matching request's semaphore. (The #beThreadSafe option needs a bit more work though). But yes, ultimately, things might be more complex. There is of course a problem in doing in-image DNS: it creates a (possibly) huge maintenance burden with a lot of responsibility. Now, since we already do socket streams, TLS/SSL and HTTP, the risk/challenge is probably acceptable. In any case, any solution would first need to prove itself in lots of real world situations. I actually added a #install class method to NeoDNSClient that hacks into the existing NetNameResolver and takes over responsibility for #addressForName:[timeout:] - for the whole image - of course this is still pure alpha. I am curious though, what was your initial motivation for starting PaleoDNS ? Which concrete issues did you encounter that you wanted to fix ? > Let's definitely talk. I hang out in the pharo discord group. :) > > > holger I am only an occasional IM users, I more of an email guy. Sven

March 29, 2019

Re: [Pharo-users] Easiest way to wrap around the indexed value in a list (mod doesn't quite work)

by Tim Mackinnon

Hey thanks guys - while it certainly makes sense when you think about it, I was kind of hoping we had something that was much more readable and obvious. It seems strange that when we have lots of esoteric things in collection, that something which is quite common to do isnât there. I was kind of hoping we might have some circular list or something. As it stands, if you really want to communicate this clearly you are almost better off just doing, the more obvious: (Index := index - 1) = 0 ifTrue: [index := items size]. Anyway, an interesting one. Tim > On 28 Mar 2019, at 16:55, Peter Kenny <peter(a)pbkresearch.co.uk> wrote: > > Tim > > I found myself puzzling as to *why* James's solution works. This longer > explanation helped me to understand. > > First, generalize moving forward to moving an arbitrary number of steps, > still with wrapping. > > ^list at: (index + move - 1 \\ size + 1 > > Second, realize that moving backward one step with wrapping is exactly > equivalent to moving forward (size - 1) steps - because of wrapping, a move > of size steps is null. > > Finally, substitute move = size - 1. > > ^list at: (index + size - 2 \\ size + 1. > > This can of course easily generalize to backward moves of any size. > > HTH > > Peter Kenny > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html >

March 29, 2019