[Pharo-dev] Catalog loading in Spotter
Sven Van Caekenberghe
sven at stfx.eu
Fri Feb 24 12:56:11 EST 2017
> On 24 Feb 2017, at 18:14, Ben Coman <btc at openinworld.com> wrote:
> On Mon, Feb 20, 2017 at 6:27 PM, Sven Van Caekenberghe <sven at stfx.eu> wrote:
> > On 20 Feb 2017, at 10:30, Pavel Krivanek <pavel.krivanek at gmail.com> wrote:
> > 2017-02-20 10:27 GMT+01:00 Guillermo Polito <guillermopolito at gmail.com>:
> > As far as I remember, the main problem was not bandwidth but a blocking and non-responsive UI.
> > Imagine I'm in the university, or in a company with a proxy. And I forget to set the proxy. Then, while spotter tries to fetch catalog entries, it would block the image and throw a timeout error some minutes after.
> > So, maybe for the future and besides Ben's C option (which I like) I have a
> > D) Load catalog entries in background in a non-blocking way
> > - and while catalog entries are not yet available, they should not be shown
> > AFIK the main problem was that there was not possilbe to do it non-blocking way.
> Yes that is true. (It is actually DNS resolving that is blocked in some freak case).
> In sqUnixSocket.c I read...
> * Notes:
> * Sockets are completely asynchronous, but the resolver is still synchronous.
> So does this mean that if for any reason the resolver primitive takes a long time to respond,
> the whole VM is blocked? Thus forking the Catalog loading in another thread would have no benefit?
Yes and yes.
> Would having the resolver coded purely in-Image behave better?
> So a slow name resolution would not hold up the whole image if it happened in a forked thread.
> I found such an implementation in OpenCobalt and after some minor cleanup basic operations work fine.
Well, I did one myself too (I have not looked at the one above, I will), but my conclusion was that it is not that difficult to get basic lookup working, but edge cases (like concurrent requests with intermixed answers) are not so easy.
There is also a chicken/egg problem: what DNS resolver (host) will you contact ? How to find out the one configured by your OS ? My idea was to use the Google ones (184.108.40.206), but there is also a more open one (220.127.116.11) and probably others.
We are often accused of reinventing the wheel and carrying the burden of maintaining all that infrastructure. Doing our own DNS client would add to that.
But I like the general idea, the question is one about development risk.
> cheers -ben
> But the thing is, the problematic situation only occurs for a very small percentage of people, like less than 1%, probably even much less. (Remember, nobody can produce a repeatable case for it).
> So the question is, is that enough reason to act as if we are still in the 90's when internet was a luxury and cut features that depend on it ? If you have no network today, you're in big trouble anyway. (Again, if you turn off your network, no blocking will happen at all, try it).
> So what is more important ? The general functionality of the 99.xxx % or the unreproducible freak case ?
More information about the Pharo-dev