My english sucks :P. It's "How can I help with that?" xD On Mon, Aug 30, 2010 at 5:15 PM, Guillermo Polito <guillermopolito@gmail.com

wrote:

On Mon, Aug 30, 2010 at 5:09 PM, Stéphane Ducasse < stephane.ducasse@inria.fr> wrote:

...

...

Stef, Noury,

Thanks for doing this, and for the preview!

Sometimes being a good friend means getting tough, and it's time for that. You are doing a great job of writing up how to create poorly designed socket applications. They are poorly designed because of what we inherit from Squeak. Servers should not listen for a time period; they need to listen until told otherwise, and trigger events (notifications if preferred) when a client tries to connect, at which point a dedicated process accepts the connection - that process more or less is the server. Clients should try to connect and read until told otherwise, either by a watchdog thread or by a user. Nothing should block in either case except the calling Smalltalk Process. If a client program does not hang because of a non-responsive server, an interacting user has the opportunity to hit a cancel button and put an end to wasted effort, or a watchdog can run and similarly #erminate the offending thread.

John pointed us to a kind of socket that raises events on data. Our problem is that we do not have manpower for that.

...

IMHO, we should not direct energy at documenting the current state of sockets; we should do the few remaining things to get something that really works. At the same time, we should try as much as possible to allow IrDA and OpenSSL to appear as options.

Noury and luc started to rewrite sockets using Alien and people can help. Nothing will happen magically :)

How should I help with that (a little)?

Cheers, Guille

...

...

Bill

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Hi, it just so happens I wrote that section on the colab wiki 1) primarily to play around with the wiki engine 2) I was debugging Magma socket issues and I could not find decent cookbook / snippets to illustrate some simple examples and how to test it outside of Pharo itself. I just thought as an exercise i would write it down.... so in response to this entire thread, huh? The example

| server client | server := Socket newTCP. server listenOn: 12345 backlogSize: 4. server waitForConnectionFor: 600. client := server accept. client receiveData

is only server side. If you follow the sections, you can see I used netcat as the client and illustrated some simple cmd line debugging techniques. the code above whilst it could of course contains bugs and not use the right API is consistent in what it tries to do.

so is this where we want to direct people first?

well, no, but sockets are fundamental and should be explained. I had only imagined it the seed for something bigger. the colab book will not expand unless people start somewhere. It is a work in progress as well, remember that... I agree it would be great if there could be some synergy between all the documentation efforts. I saw the colab book as a fairly free form area to write stuff down in a simple manner that was hosted on a wiki like space. PBE is a more coherent, dedicated and managed publishing effort. I had imagined the colab wiki could be a feeder for PBE but time will tell. It requires significant effort to produce something of the quality of PBE. cheers, Mike

The deeper question is whether we want to document how things are, or they they *should be* - which is VERY much different from the eventless and timeout ridden design that we have now. It desperately needs to be redesigned. ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Sean P. DeNigris [sean@clipperadams.com] Sent: Tuesday, August 31, 2010 10:41 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Sockets in Pharo CollaborActive Book Michael Roberts-2 wrote:

The example <snip> is only server side. If you follow the sections, you can see I used netcat as the client and illustrated some simple cmd line debugging techniques. the code above whilst it could of course contains bugs and not use the right API is consistent in what it tries to do.

Ah, okay. Now I got it. The client temporary variable, along with the fact that this snippet is a section to itself confused me and I didn't keep reading.

so is this where we want to direct people first?

Michael Roberts-2 wrote:

well, no, but sockets are fundamental and should be explained. I had only imagined it the seed for something bigger. the colab book will not expand unless people start somewhere. It is a work in progress as well, remember that...

Cool, I appreciate you getting the ball rolling - I am trying to push it further! (and I don't know too much about sockets) Sean -- View this message in context: http://forum.world.st/Sockets-in-Pharo-CollaborActive-Book-tp2399361p2402542... Sent from the Pharo Smalltalk mailing list archive at Nabble.com. _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

I am certainly willing to help with the work, and while I am by no means an expert on sockets, I have seen some good code. As we stand now, it is all but impossible to do things correctly; that will become clear by contrast, but only if people are willing to challenge their ideas. Timeouts are bad (unless provided by helper methods so they are explicitly requested); errors are helpful (silent failures are toxic); events are essential. It sounds like there are at least four of us who have strong interest in getting this right. We probably should have some discussion here, then describe a plan on the wiki and finally start coding. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Miguel Enrique Cobá Martínez [miguel.coba@gmail.com] Sent: Wednesday, September 01, 2010 12:10 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Sockets in Pharo CollaborActive Book El mar, 31-08-2010 a las 23:59 -0400, Schwab,Wilhelm K escribió:

The deeper question is whether we want to document how things are, or they they *should be* - which is VERY much different from the eventless and timeout ridden design that we have now. It desperately needs to be redesigned.

Maybe it needs someone to lead the effort. Why don't you begin with this? It appears that you know a lot about sockets. Certainly the pharo community will be grateful of your efforts. But saying again and again that sockets sucks doesn't fix it for anybody, including you. So, as Stef says, less saying, more coding. Very respectfully, show us your code :) Cheers

________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Sean P. DeNigris [sean@clipperadams.com] Sent: Tuesday, August 31, 2010 10:41 PM To: pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Sockets in Pharo CollaborActive Book

Michael Roberts-2 wrote:

...

The example <snip> is only server side. If you follow the sections, you can see I used netcat as the client and illustrated some simple cmd line debugging techniques. the code above whilst it could of course contains bugs and not use the right API is consistent in what it tries to do.

Ah, okay. Now I got it. The client temporary variable, along with the fact that this snippet is a section to itself confused me and I didn't keep reading.

...

so is this where we want to direct people first?

Michael Roberts-2 wrote:

...

well, no, but sockets are fundamental and should be explained. I had only imagined it the seed for something bigger. the colab book will not expand unless people start somewhere. It is a work in progress as well, remember that...

Cool, I appreciate you getting the ball rolling - I am trying to push it further! (and I don't know too much about sockets)

Sean -- View this message in context: http://forum.world.st/Sockets-in-Pharo-CollaborActive-Book-tp2399361p2402542... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

-- Miguel Cobá http://miguel.leugim.com.mx _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Noury, First, please don't assume that you have to change course just because we offered to help. If you are close to having a major improvement, then it might be best to just power through it. If there are things holding you up, we might be able to help there. Using Alien concerns me a little, as the last time I checked, it was not available on Linux?? Failure to support Linux would be a complete show-stopper for me. Regardless, I would like to see your code as I have read some comments suggesting that Alien is cumbersome to use; you might show me that all is well. My interest in Alien is primarily to support callbacks for numerical analysis. So far, I have been adding functions to a shared library, which is easy enough, ultimately translates formulas using C++ (which is better at it than Smalltalk), likely makes things go faster than they would with Smalltalk in the loop, but is far from elegant. Until Alien exists on Linux, it does not exist :( Back to sockets, if the current situation were to stand, for whatever reason, I would consider tackling the problem using a plugin to get OS threads, and use them to achieve the non-blocking operation that is essential for interactive software. In short, everything blocks its calling Smalltalk Process; nothing blocks the entire image. Hopefully you recognize that a server listening with timeouts is hideous design; just start and stop the server as needed, and it should be harmless to the image. If the server is not harmless to the image, it needs to be fixed. Clients having timeouts isn't much better. If you want to add helper methods with timeouts, who am I to tell others what to do, but we must not force timeouts on clients because there is no correct answer to how long something should wait, so we should not look for one. What we should do is make sure that we never block the entire image, and let people make their own decisions. Why would I consider a plugin? In large part because there are hints that I would find examples to follow, and because OpenSSL has a very cumbersome non-blocking interface, at least that was my impression of it. A plugin that I would write would hopefully support TCP and OpenSSL sockets, and hopefully IrDA. Running those things on OS threads and signaling semaphores in the image is likely to be simpler than working out the details of non-blocking calls in the separate worlds - I think. I can understand why you are not thrilled about creating a plugin. The sad part of this is that the current sockets system appears to demonstrate all of the essential pieces to make something that really works; it just hasn't been done. On attempting it, I would no doubt run across things that you found and led you to create a new system. Creating something new allows it to coexist with the current code until it is ready, at which point it could be integrated into Pharo. Thanks for working on it. Is the code on Squeaksource representative of your current progress? Please do not let my interest slow your efforts. If you are indeed doing all of the right things, it might be that my efforts should be directed toward getting Alien going on Linux. It might also be that I can be of most help simply by doing to your code what I did to DCOM long ago. Our network was a harsh landscape, and I got fairly good bringing code to its knees on intranets. This involves creating forked processes doing active communication across a hub that coincidentally gets unpowered w/o warning :) Threads must block or give errors, but the images must continue to be responsive and other threads must run as expected. Until we reliably pass that test, this is all just a game. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Noury Bouraqadi [bouraqadi@gmail.com] Sent: Saturday, September 04, 2010 5:31 AM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Sockets in Pharo CollaborActive Book Hi Bill, On 1 sept. 2010, at 05:59, Schwab,Wilhelm K wrote:

The deeper question is whether we want to document how things are, or they they *should be* - which is VERY much different from the eventless and timeout ridden design that we have now. It desperately needs to be redesigned.

I agree that the current status of the networking libraries is horrible. But, to learn the current status and think of the alternatives, I felt the need to understand how thinks work now. As Stéf said, Luc and I started the Ocean project. Thanks you Bill and to Guille for your proposal to help. It's a pitty that you don't attend to the ESUG conf (Your names are not on the registrations list) otherwise we could discuss this better. Cause Luc and I will give a short presentation describing the current status of Ocean and where it is headed. Now, we need to think how to split the project in order to have other contributions. Note BTW, our goal is to have a low-level Pharo wrappers of the networking facilities of the OS. We started using Alien to avoid using a plugin and ease portability. Once we get this low-level layer, we'll go on and use it in an OOP middleware like rST and UbiquiTalk. Which is after all, the right level to work with since we want to deal with objects and message sends. A direct use of Sockets isn't really OO most of the time. Noury _______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

On 5 sept. 2010, at 03:22, Schwab,Wilhelm K wrote:

Noury,

I was struggling a little with a stream for serial ports, and thought you might have tackled the problem of a generating stream (one that can grow in the background). Now that I type that, maybe Nile has the answer??

Re Ocean, I grabbed a .mcz from SqueakSource and simply browsed the main package. Some comments:

(1) no socket stream :( Streams are really slick things, and good read and write stream semantics should exist for sockets. FWIW, Dolphin separates the socket read and write streams, and we might do well to follow their lead.

Not yet ;-) So far, we want to have a clean socket. And then we'll rebuild a SocketStream on top of it.

(2) ByteArray>>asAlien loops over the bytes. Do all the bounds checking you want up front (once) and then use memcpy() or something to actually transfer the data. It recently came to my attention that Squeak has no formalized approach to access, copy and move external memory; Pharo needs to offer that, and this is a good place to start.

Thanks for the hint.

(3) You are planning to attack ipv4 vs. 6 with polymorphism - good call!!

Yes. One reason behind doing Ocean is that we found existing library too dirty to improve. So, we try to have a well designed tested library.

(4) OCNTcpSocket>>send: sizes a buffer to match the data provided to it. I'm not sure I like that. You might consider renaming your current #send: to #basicSend: and adding something like

send:blob | in | in := blob readStream. [ in atEnd ] whileFalse:[ self basicSend:( in nextAvailable:self alienDataBufferSize ). ].

so that a fixed buffer (which will have to be allocated when the socket is opened) is used to send large blobs in buffer-sized chunks. Better still would be to use array indexing to cope with the blob in pieces but in place to skip the copying.

We discussed this with Luc. And the rational behind our decision to do it this way is that Socket is but a basic wrapper for the actual OS socket. It shouldn't provide any extra features. Spliting a blob is something I'd rather do by using a SocketStream. Thanx for the feedback, Noury

would be cool to get inspired and produce material for smalltalkers. On Sep 1, 2010, at 9:03 AM, Michael Roberts wrote:

...

...

Cool, I appreciate you getting the ball rolling - I am trying to push it further! (and I don't know too much about sockets)

no worries. I have found this guide really useful: http://beej.us/guide/bgnet/

Even if you don't read C, there are lots of comments throughout.

cheers Mike

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Stef, Where does one go to sign up? I am not certain that Alien is appropriate or necessary to make a solid socket foundation, but I am certain that the existing code is very sub-optimal and needs a real jolt. If we are serious about fixing it (Linux, Windows and Mac), I am serious about helping. Bill ________________________________________ From: pharo-project-bounces@lists.gforge.inria.fr [pharo-project-bounces@lists.gforge.inria.fr] On Behalf Of Stéphane Ducasse [stephane.ducasse@inria.fr] Sent: Monday, August 30, 2010 4:09 PM To: Pharo-project@lists.gforge.inria.fr Subject: Re: [Pharo-project] Sockets in Pharo CollaborActive Book

Stef, Noury,

Thanks for doing this, and for the preview!

Sometimes being a good friend means getting tough, and it's time for that. You are doing a great job of writing up how to create poorly designed socket applications. They are poorly designed because of what we inherit from Squeak. Servers should not listen for a time period; they need to listen until told otherwise, and trigger events (notifications if preferred) when a client tries to connect, at which point a dedicated process accepts the connection - that process more or less is the server. Clients should try to connect and read until told otherwise, either by a watchdog thread or by a user. Nothing should block in either case except the calling Smalltalk Process. If a client program does not hang because of a non-responsive server, an interacting user has the opportunity to hit a cancel button and put an end to wasted effort, or a watchdog can run and similarly #erminate the offending thread.

John pointed us to a kind of socket that raises events on data. Our problem is that we do not have manpower for that.

IMHO, we should not direct energy at documenting the current state of sockets; we should do the few remaining things to get something that really works. At the same time, we should try as much as possible to allow IrDA and OpenSSL to appear as options.

Noury and luc started to rewrite sockets using Alien and people can help. Nothing will happen magically :)

Bill

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project