Am 19.08.2011 um 20:45 schrieb Sven Van Caekenberghe:

Hi Camillo,

On 19 Aug 2011, at 20:02, Camillo Bruni wrote:

...

I'm working on the various GitProtocols, amongst them is the normal HTTP version which in popular cases (like github) requires the SSL version.

So is there somewhere an SSL version for Zinc? (even partially working ;)

cheers cami

There is the 'Zodiac Alpha Release', http://lists.gforge.inria.fr/pipermail/pharo-project/2011-June/050539.html This is only for Mac though. I haven't really worked on this since then.

What is Zodiac based on again? I assume you didn't write to native stuff yourself. Norbert

Am 11.09.2011 um 21:05 schrieb Sven Van Caekenberghe:

On 11 Sep 2011, at 15:42, Norbert Hartl wrote:

...

What is Zodiac based on again? I assume you didn't write to native stuff yourself.

The idea is to use a new, more comprehensible SocketStream implementation and the use that to subclass a new SecureSocketStream that delegates the 4 key ssl methods (connect, accept, encode & decode) to an ssl session object. There could be 2 implementations of that session object: one based on the existing SSL plugin (a wrapper to the OpenSSL library) and one based on all the crypto work being done in Smalltalk.

Is the SSL plugin the one from Andreas that has license issues? Norbert

Am 12.09.2011 um 09:44 schrieb Sven Van Caekenberghe:

On 12 Sep 2011, at 09:21, Norbert Hartl wrote:

...

Is the SSL plugin the one from Andreas that has license issues?

Yes.

What is the license issue?

It is MIT.

Great, thanks. And the archive you provided only works on Mac because it needs a dedicated VM and you included only the Mac one, right? Is there a strategy for what plugins are in the standard VM? And which ones are built into a bigger one? Well, that might be a stupid question. But my knowledge about VMs is that for adding a plugin to a vm you have to compile the vm for it. thanks, Norbert

Am 12.09.2011 um 12:35 schrieb Sven Van Caekenberghe:

On 12 Sep 2011, at 12:28, Norbert Hartl wrote:

...

And the archive you provided only works on Mac because it needs a dedicated VM and you included only the Mac one, right?

No, it only works on the Mac (the snapshort image is the easiest way to test things).

The problem is that the new SocketStream implementation worked fine for me during development (on Mac), but later I found out that there are subtle but critical problems on the other platforms.

Subtle means they are not easy to reproduce? I think your initiative is an important one. SSL is gaining more momentum and it is a must-have these days. It would be good to build a little bit of infrastructure around and try to evolve it. What are your plans with Zodiac? What would you wish should happen to it? Norbert

Am 12.09.2011 um 14:45 schrieb Sven Van Caekenberghe:

Wish me luck.

I really really do! Norbert

On 12 Sep 2011, at 14:45, Sven Van Caekenberghe wrote:

Now, I still have to try the actual SSL tests by moving the plugin inside the One Click VM's on Linux and Windows. I'll report on how that goes later on when I have some more time.

I took a current Pharo One Click 1.3 (actually #13299) and placed the matching plugins from http://squeakssl.googlecode.com/files/SqueakSSL-bin-0.1.4.zip in the right places (I had to chmod +x the linux one) and tested the latest Zinc and Zodiac code, like this: Gofer it squeaksource: 'ZincHTTPComponents'; package: 'Zinc-HTTP'; package: 'Zinc-Tests'; package: 'Zinc-Patch-HTTPSocket'; load. Gofer it squeaksource: 'Zodiac'; package: 'Zodiac-Core'; package: 'Zodiac-Tests'; load. Gofer it squeaksource: 'ZincHTTPComponents'; package: 'Zinc-Zodiac'; load. "Switch to the Zn Zodiac socket factory" ZnNetworkingUtils default: ZnZodiacNetworkingUtils new. The good news is that, as I reported before, the non secure socket streams run without problems on all platforms. Also good is that the secure socket streams with the native plugin work, that is, on all platforms you can now do ZnClient get: 'https://encrypted.google.com' The bad news is that the 7 ZnHttpsTests behave differently on the different platforms: - Mac OS X: 7 green - Linux: 1 failure - Windos: 3 failures The problem of course is that this is pretty hard to debug: although the image, Zodiac and Zinc code is of course identical, the VM and the plugin differ for each platform. People interested in this can try to do this themselves; later on I might make a customized OneClick download as a second Zodiac Alpha release, where both the image and VM are ready to test. Sven

+1

...

...

And the archive you provided only works on Mac because it needs a dedicated VM and you included only the Mac one, right?

No, it only works on the Mac (the snapshort image is the easiest way to test things).

The problem is that the new SocketStream implementation worked fine for me during development (on Mac), but later I found out that there are subtle but critical problems on the other platforms.

Subtle means they are not easy to reproduce?

I think your initiative is an important one. SSL is gaining more momentum and it is a must-have these days. It would be good to build a little bit of infrastructure around and try to evolve it. What are your plans with Zodiac? What would you wish should happen to it?

Norbert

Göran, On 13 Sep 2011, at 09:38, Göran Krampe wrote:

Hi!

On 09/12/2011 12:35 PM, Sven Van Caekenberghe wrote:

...

The problem is that the new SocketStream implementation worked fine for me during development (on Mac), but later I found out that there are subtle but critical problems on the other platforms.

Just a few words about the SocketStream implementation. I wrote the current incarnation of SocketStream (unless it has changed under Pharo - haven't looked) and one important aspect of the rewrite was to make it much faster than the old code written originally by Bolot Kerimbaev back in... ancient times.

Basically two things make it fast: - As little copying as possible. - Making sure code lands on good primitives.

An example of the former is the logic involved in dealing with the buffer growing etc.

An example of the latter is upToAll: that could be written naively and end up using next, next for each byte. Ouch.

Anyway, there are indeed some methods in SocketStream that "hurts the eyes", like upToAll: but that is mainly because we are lacking a primitive for searching that has a "stopAt" argument. I never got around to extending the one we have with such an argument.

regards, Göran

I respect the current SocketStream: it is used by all of us everyday and it works reliably. The problem is that its code base has grown in all kinds of directions and in complexity. Today it is pretty hard to understand since it is not clear if all code that is there is relevant. Also, subclassing it to build a secure socket stream is pretty difficult. The Zodiac streams are an experiment in rewriting [secure] socket stream. I would be honored if you would take some time to look at the code and give some feedback http://www.squeaksource.com/Zodiac

From one of my previous mails:

Warning: this is of course not yet code for general consumption, use the source and the comments. This is a proof of concept. This package has no further dependencies and does not add anything to system classes, it is thus safe to load. There are four stream classes, ZdcAbstractSocketStream, ZdcSimpleSocketStream, ZdcOptimizedSocketStream and ZdcSocketStream, inheriting from each other, adding successively more optimized implementations of the standard Smalltalk stream primitives, especially for bulk IO. There is one helper class, ZdcIOBuffer to help in managing read and write buffers inside the streams. The interface towards Socket is also quite small (i.e. only a very small amount of the functionality in Socket is used). The 48 units tests as well as the functional test (see further) are all green on Pharo 1.2.x and 1.3 as well as on Squeak 4.2. The functional test is the following: Zinc HTTP Components recently got a its ZnNetworkingUtils class turned into a socket stream factory. By loading the new package Zinc-Zodiac from the ZincHTTPComponents repository, you can ask Zn to use the new socket stream implementation, like this: ZnNetworkingUtils default: ZnZodiacNetworkingUtils new. To reset, just do: ZnNetworkingUtils default: nil. The cool thing is: all of Zn's +170 unit tests, most doing actual HTTP client and server interactions (some over the internet) succeed using the new Zodiac socket stream implementation. Regards, Sven

On 13 Sep 2011, at 13:10, Göran Krampe wrote:

I could and will try to squeeze it in, but I am mainly hacking Jtalk these days :)

OK

...

The cool thing is: all of Zn's +170 unit tests, most doing actual HTTP client and server interactions (some over the internet) succeed using the new Zodiac socket stream implementation.

Have you also measured performance? Compared to old SocketStream? Just curious.

No, not yet, but it feels OK ;-) I think it is quite a challenge to measure socket stream performance objectively.

The earliest SocketStream by Bolot also used an internal object for each "direction" of the stream - a BufferStream I think it was called. And nuking those to make SocketStream operate directly on buffers with the ability to move "pointers" etc was one of the main optimizations in order to minimize copying etc.

Yes, one of my goals is to prevent copying and moving bytes around as much as possible. The most optimized version, the one at the bottom of the hierarchy, ZnSocketStream, never copies the data (say a large byte array) that you write, and reads directly into the buffers that you provide - provided you use the right methods of course, i.e. those based on #next: count putAll: collection startingAt: offset and #readInto: collection startingAt: offset count: requestedCount But the first goals are correctness and good unit and functional test coverage.

Exactly Thanks for precising it.

...

...

...

Great, thanks. And the archive you provided only works on Mac because it needs a dedicated VM and you included only the Mac one, right? Is there a strategy for what plugins are in the standard VM? And which ones are built into a bigger one? Well, that might be a stupid question. But my knowledge about VMs is that for adding a plugin to a vm you have to compile the vm for it.

A plugin can be built in two different ways: internal and external. If you want the plugin to be internal (which is not the case for SqueakSSL), then you have to rebuild the VM, otherwise the plugin is just a platform specific dynamic library file (.so, .dll, .dylib, etc) which you can copy to the VM or image directory.

Levente

...

thanks,

Norbert