2013/5/10 Ben Coman <btc@openinworld.com>

Luc Fabresse wrote:

...

Hi all,

I am *experimenting* with the replacement of the current network layer with OCEAN. My current plan is to make Zinc tests green while working on the top of OCEAN. And investigate the removal of all the current Network classes (does it play?). After looking at Zinc code, there are few dependencies. Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils. It seems to play so far (green tests).

But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream). Now remember the mails of Sven.

So my questions are:

1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?

2) why ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream? why the code isn't something like that: ZnNetworkingUtils>>initialize super initialize. self socketStreamClass: (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) . self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])

Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)). So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.

Thanks for giving me some light here,

Luc

I'm not even close to knowing the answers you'd like, but it looks really interesting. Slide 9 at [1] shows Alien being used to interact with the OS Network API. What are you thoughts on NativeBoost being used instead ?

It is old slides ;-) History: At the beginning we would like to replace the Network VM plugin by wraping the POSIX socket layer through Alien. At that time, we chose Alien because it supports callbacks. And it was a nightmare to use Nativeboost because it required a special VM (with NB plugin) but that VM had not some Network primitives about ipv6 and UDP. Now things are much much better ;-) one VM and everything inside. But now we focus on the image part (Network API) that still uses the plugin as backend (no need for Alien at all). When it will work we will implement another backend with NB probably. I know that Igor started one. But I would to investigate wrapping ZeroMQ (I saw some mails about that). Cheers, Luc

[1] http://www.slideshare.net/**esug/ocean-5305932<http://www.slideshare.net/esug/ocean-5305932>

cheers -ben

cool signature :)

Hi Luc,

On 09 May 2013, at 22:05, Luc Fabresse <luc.fabresse@gmail.com> wrote:

...

Hi all,

I am *experimenting* with the replacement of the current network layer with OCEAN. My current plan is to make Zinc tests green while working on the top of OCEAN. And investigate the removal of all the current Network classes (does it play?). After looking at Zinc code, there are few dependencies. Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils. It seems to play so far (green tests).

It is great that you are picking this up again.

If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !

...

But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream). Now remember the mails of Sven.

So my questions are:

1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?

2) why ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream? why the code isn't something like that:

ZnNetworkingUtils>>initialize super initialize. self socketStreamClass: (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) . self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])

Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).

So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.

Thanks for giving me some light here,

Luc

When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:

stream binary; shouldSignal: true; autoFlush: false

Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:

ZdcAbstractSocketStream ZdcSimpleSocketStream ZdcOptimizedSocketStream ZdcSocketStream

My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing

ZnNetworkingUtils default socketStreamClass: ZdcSocketStream

After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.

Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.

Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.

The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.

Regards,

Sven

-- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org

On 11 May 2013, at 06:34, Luc Fabresse <luc.fabresse@gmail.com> wrote:

Hi Sven,

Thanks for the explanation. You confirm what I was thinking, I will inegrate the ZdcAbstractSocketStream.

Cool. Is there a place where I can already see what you did up to now ?

I never used deeply SocketStream. So, I will also have a closer look at SocketStream to understand what you said about "abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues". My first impression is that we could have an "Encoder" hierarchy on top of a binary streams. Perhaps it is already there.

Yes, there are ZnCharacterReadStream and ZnCharacterWriteStream that convert a binary stream into a character read and write stream respectively, using a decoder/encoder. BTW, there are also ZnBufferedReadStream and ZnBufferedWriteStream. Sven

Thanks,

#Luc

#Luc

2013/5/10 Sven Van Caekenberghe <sven@stfx.eu> Hi Luc,

On 09 May 2013, at 22:05, Luc Fabresse <luc.fabresse@gmail.com> wrote:

...

Hi all,

I am *experimenting* with the replacement of the current network layer with OCEAN. My current plan is to make Zinc tests green while working on the top of OCEAN. And investigate the removal of all the current Network classes (does it play?). After looking at Zinc code, there are few dependencies. Basically, I implemented an OCNSOcketStream (inheriting from SocketStream) and redefined some methods of ZnNetworkingUtils. It seems to play so far (green tests).

It is great that you are picking this up again.

If you mean that your OCNSocketStream with OCEAN Sockets can now run the full Zn test suite, that would be very good news indeed !

...

But then, I discovered that Zodiac introduces a new SocketStream hierarchy (ZdcAbstractSocketStream). Now remember the mails of Sven.

So my questions are:

1) Is the intent of ZdcSocketStream to be a future replacement for SocketStream?

2) why ZnNetworkingUtils>>initialize makes use of SocketStream instead of ZdcSocketStream? why the code isn't something like that:

ZnNetworkingUtils>>initialize super initialize. self socketStreamClass: (Smalltalk globals at: #ZdcSocketStream ifAbsent: [ SocketStream ]) . self secureSocketStreamClass: (Smalltalk globals at: #ZdcSecureSocketStream ifAbsent: [ nil ])

Moreover, I guess that ZdcSocketStream should speed up things (it says optimized version in the name of the superclass ;-)).

So probably, my OCNSOcketStream should better inherit from ZdcSocketStream.

Thanks for giving me some light here,

Luc

When I started writing Zinc, now already more than 2 years ago, I used the regular SocketStreams. But I limited my use to the following semantic options and just the regular ReadStream API:

stream binary; shouldSignal: true; autoFlush: false

Some time later I did Zodiac. Before doing the TLS/SSL streams I wanted to make sure I correctly understood the SocketStream API and semantics that I was using. That is why I first reimplemented regular socket streams. Hence the hierarchy:

ZdcAbstractSocketStream ZdcSimpleSocketStream ZdcOptimizedSocketStream ZdcSocketStream

My goal too was to be able to run the full Zn test suite. That is still possible, you can switch the plain socket stream being used by doing

ZnNetworkingUtils default socketStreamClass: ZdcSocketStream

After that came the actual ZdcSecureSocketStream. Nowadays, Zodiac is a full part of Pharo, and thus used in the wild, which validates the implementation.

Yes, I think it would be way better (and easier for you) to plug into the hierarchy described above. If you need any help, just ask. I am very interested in that. Maybe the 'pluggability' can be improved.

Now the question 'should ZdcSocketStream replace SocketStream ?' can only be answered by users like yourself. As fas as I use socket streams I think it can, but I am sure some people will protest when they can no longer abuse this binary stream for writing characters or strings or reading lines while ignoring encoding issues.

The SocketStream that we have today works, but it is overly complex in API and implementation. It is also inefficient: try following what happens when you write a 1Mb ByteArray.

Regards,

Sven

-- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org