The idea is to have much simpler streams which can be composed to get more sophisticated behaviour. The most primitive streams should be binary read or write streams, like a raw file or network connection. To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders). If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them. And there are some other examples in the system too. Have a look at BinaryFileStream and ZdcSocketStream. Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change. The contract of a stream should be much, much simpler than it is today. For writing that means #nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt: the 3 last ones can be written in terms of of the first one, but the last one is key because it can be the most efficient. And maybe also #flush #close Some helpers for character writing are #space #tab #cr #crlf #lf Maybe #newline #<< is a handy method too. For reading that means #atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count: Again, they can all be written in terms of #next, but #readInto:startingAt:count: is the core, efficient one. Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs. #close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too. There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above. If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection). The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes. All this being said, there is no one, single correct answer. But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on. Sven

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote:

...

I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

Thanks Sven, we should take that list of selectors and make a document or Lint rules on them. On Mon, Nov 13, 2017 at 9:08 PM, Stephane Ducasse <stepharo.self@gmail.com> wrote:

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:

...

The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the

last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote:

...

I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

2017-11-13 23:51 GMT+01:00 Norbert Hartl <norbert@hartl.name>:

...

Am 13.11.2017 um 21:08 schrieb Stephane Ducasse <stepharo.self@gmail.com :

...

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote: The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

In bootstrap we already use the package 'Zinc-Character-Encoding-Core'. It is standalone so there is no dependency issue. -- Pavel

Norbert

...

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the

last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

...

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote: I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

To a package next to block? On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <norbert@hartl.name>:

...

...

Am 13.11.2017 um 21:08 schrieb Stephane Ducasse < stepharo.self@gmail.com>:

...

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote: The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

...

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the

last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

...

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote: I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

2017-11-14 9:53 GMT+01:00 Denis Kudriashov <dionisiydk@gmail.com>:

I look at the code, So Zinc provides only binary/character streams. Right?

About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?

yes, time to time we make sync with Zinc upstream. -- Pavel

2017-11-14 9:36 GMT+01:00 Guillermo Polito <guillermopolito@gmail.com>:

...

To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <norbert@hartl.name>:

...

...

Am 13.11.2017 um 21:08 schrieb Stephane Ducasse < stepharo.self@gmail.com>:

...

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote: The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

...

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the

last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

...

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote: I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

--

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>*

*Web:* *http://guillep.github.io* <http://guillep.github.io>

*Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

2017-11-14 14:00 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

On 14 Nov 2017, at 09:53, Denis Kudriashov <dionisiydk@gmail.com> wrote:

I look at the code, So Zinc provides only binary/character streams. Right?

Yes, Zn streams focus on classic binary(byte) / character streams.

Streaming over arbitrary data is very cool and well covered by the old ones.

While I really like traditional streams I can not agree here. Sometimes it reminds me poor java collections which force writing loops all the time. For example the most common task in my experience was writing contents of read stream into write stream. And the only possibility now is loop.

From this point of view XStreams really pushes streams to the level of smalltalk collections.

...

About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?

2017-11-14 9:36 GMT+01:00 Guillermo Polito <guillermopolito@gmail.com>: To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov <dionisiydk@gmail.com> wrote: What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <norbert@hartl.name>:

...

Am 13.11.2017 um 21:08 schrieb Stephane Ducasse < stepharo.self@gmail.com>:

...

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote: The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

...

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but the

last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

...

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote: I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

--

Guille Polito Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io Phone: +33 06 52 70 66 13

2017-11-14 14:25 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:

...

Yes, Zn streams focus on classic binary(byte) / character streams.

Streaming over arbitrary data is very cool and well covered by the old ones.

While I really like traditional streams I can not agree here. Sometimes it reminds me poor java collections which force writing loops all the time. For example the most common task in my experience was writing contents of read stream into write stream. And the only possibility now is loop. From this point of view XStreams really pushes streams to the level of smalltalk collections.

Yes, I agree, Xtreams is much better (but steep learning curve).

I just wanted to point out that my contributions in Zn streams focus and better/simpler byte/character IO.

Yes, and it is really nice. Interesting how many users we have in system for general streams? (created on arbitrary collections).

From first look the main users are #printOn: methods. But they use string based, so they are not general. What others we have?

Improvement on the classic streams are, of course, welcome.

...

...

About contribution: it is in external repository of Sven. Can we contribute with normal process, create pull request into Pharo repo?

2017-11-14 9:36 GMT+01:00 Guillermo Polito <guillermopolito@gmail.com : To a package next to block?

On Tue, Nov 14, 2017 at 9:16 AM, Denis Kudriashov < dionisiydk@gmail.com> wrote: What about contributing to zinc streams? Imaging that I will create block based streams, collecting:/selecting streams like in XSteam. Where I should put them?

2017-11-13 23:51 GMT+01:00 Norbert Hartl <norbert@hartl.name>:

...

Am 13.11.2017 um 21:08 schrieb Stephane Ducasse < stepharo.self@gmail.com>:

...

On Mon, Nov 13, 2017 at 8:27 PM, Sven Van Caekenberghe < sven@stfx.eu> wrote: The idea is to have much simpler streams which can be composed to get more sophisticated behaviour.

The most primitive streams should be binary read or write streams, like a raw file or network connection.

To add a character encoding/decoding you wrap them in a ZnCharacterReadStream or ZnCharacterWriteStream (these use the newer, cleaner ZnCharacterEncoders).

Yes really nice :)

And Guille started to use them and we are slowly rewriting all the stream internal users to use Zn and after we will be free.

No, you will depend on zinc classes. How is that supposed to work in bootstrap?

Norbert

...

...

If you want buffering, you wrap a ZnBufferedReadStream or ZnBufferedWriteStream around them.

And there are some other examples in the system too.

Have a look at BinaryFileStream and ZdcSocketStream.

Simply put, MultiByteFileStream and MultiByteBinaryOrTextStream must die, because they try to be everything at once and are impossible to change.

YES YES YES and celebrate. I could never understand anything. My brain is too limited for these kind of games :)

...

The contract of a stream should be much, much simpler than it is today.

Fully agree.

...

For writing that means

#nextPut: #nextPutAll: #next:putAll: #next:putAll:startingAt:

the 3 last ones can be written in terms of of the first one, but

the last one is key because it can be the most efficient.

...

And maybe also

#flush #close

Some helpers for character writing are

#space #tab #cr #crlf #lf

Maybe #newline

:)

...

#<< is a handy method too.

For reading that means

#atEnd #next #next: #next:into: #next:into:startingAt: #nextInto: #peek #skip: #upToEnd #upTo: #readInto:startingAt:count:

Again, they can all be written in terms of #next, but

#readInto:startingAt:count: is the core, efficient one.

...

Note that #peek allows a one character lookahead, which should be sufficient for almost all parsing needs.

#close is also a necessary operation, #peekFor: a handy one, #nextLine is popular too.

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

If you absolutely need these, read #upToEnd and use a regular ReadStream (over a fixed collection).

The collection based classic Streams should always remain in the system, they are too handy. But have you seen for example, #nextInt32 on PositionableStream ? Good luck with that when the the underlying collection is anything other than bytes.

All this being said, there is no one, single correct answer.

But if we all try to simplify what we expect of streams (use a more limited API), we'll be more nimble to make implementation changes later on.

Sven

...

On 13 Nov 2017, at 19:58, Stephane Ducasse < stepharo.self@gmail.com> wrote:

Hi Evan

I think that we will use the ZnStreams. If we use Xtreams we will transform their API because some messages are not really good. Stef

...

On Mon, Nov 13, 2017 at 7:54 PM, Evan Donahue <emdonahu@gmail.com> wrote: I've heard mention once or twice on this list and in some release notes of what sounded like possible coming changes to the stream API. Could anyone point me to any concrete details about that? I haven't been able to dig anything up myself by searching. I'm about to write something that I'd like to be polymorphic with the stream API, but if that's about to change, I'd like to plan ahead.

Thanks, Evan

--

Guille Polito Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io Phone: +33 06 52 70 66 13

On 14 Nov 2017, at 15:33, Steffen Märcker <merkste@web.de> wrote:

Hi!

...

...

Yes, I agree, Xtreams is much better (but steep learning curve).

I just wanted to point out that my contributions in Zn streams focus and better/simpler byte/character IO.

Yes, and it is really nice. Interesting how many users we have in system for general streams? (created on arbitrary collections).

I really think streams (in general) should focus on what they are best at. Namely, (stepwise) reading and writing from and to various sources, and buffering for efficiency, too. XStreams does an excellent job here. However, higher level operations - like collecting, selecting, splitting (map, filter, partition) and such - should be addressed by other means. Those operations apply to streams, collections, generators and other data structures. They can efficiently be implemented independent from the data structure. By doing so, code duplication can be avoided and the API of streams, etc. can be kept simple.

Although I won't have time to contribute code, before finishing my thesis, I'd like to point out, that transducers are here to address exactly this. The package already works with collections, streams and xstreams.

Are transducers the subject of your thesis ? Any pointers to more information ?

Best, Steffen

On 14 Nov 2017, at 16:00, Steffen Märcker <merkste@web.de> wrote:

Hi,

...

Are transducers the subject of your thesis ?

No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)

...

Any pointers to more information ?

http://www.cincomsmalltalk.com/publicRepository/Transducers.html https://clojure.org/reference/transducers Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.

Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

2017-11-14 16:30 GMT+01:00 Steffen Märcker <merkste@web.de>:

I forgot to mention, that the most recent code for Pharo is already on Github: https://github.com/Pharophile/Transducers

Reducers was the name of the first very first implementation.

(In fact, I was originally inspired by clojures Reducers lib. After implementing it in Smalltalk, I developed the concept further. Later I found out, that the clojure guys did the same in parallel and ended up with the same abstraction but a differnt name. Hence I decided change the name in order to make the relation clear.)

I like abstraction. But I think names and order of computation should be changed to be more Smalltalk friendly. Because now it looks like Haskell with right to left order: squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). fileOut writeStream <~ #isSeparator filter <~ fileIn readStream. Is there any reason to not change it? I would like to start expressions with source of data: squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set. fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.

Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <sven@stfx.eu>:

...

On 14 Nov 2017, at 16:00, Steffen Märcker <merkste@web.de> wrote:

...

Hi,

<---Schnitt--->

...

...

No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)

<---Schnitt--->

...

...

http://www.cincomsmalltalk.com/publicRepository/Transducers.html https://clojure.org/reference/transducers Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.

Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments

Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

Sven

From the thread discussion here my take.

- I do not think that we want to reuse any of the old streams. - So first Zn to replace the binary and MultiUglyStream and after we will have to check if we should port the design of Xtream (but not the API) to Pharo. Stef On Tue, Nov 14, 2017 at 5:59 PM, Stephane Ducasse <stepharo.self@gmail.com> wrote:

Denis I agree. I do not like to code in reverse order.

On Tue, Nov 14, 2017 at 4:49 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

...

2017-11-14 16:30 GMT+01:00 Steffen Märcker <merkste@web.de>:

...

I forgot to mention, that the most recent code for Pharo is already on Github: https://github.com/Pharophile/Transducers

Reducers was the name of the first very first implementation.

(In fact, I was originally inspired by clojures Reducers lib. After implementing it in Smalltalk, I developed the concept further. Later I found out, that the clojure guys did the same in parallel and ended up with the same abstraction but a differnt name. Hence I decided change the name in order to make the relation clear.)

I like abstraction. But I think names and order of computation should be changed to be more Smalltalk friendly. Because now it looks like Haskell with right to left order:

squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.

Is there any reason to not change it? I would like to start expressions with source of data:

squares := (1 to: 1000) ~> #squared map ~> 1000 take ~> Set. fileIn readStream ~> #isSeparator filter ~> fileOut writeStream.

...

Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <sven@stfx.eu>:

...

...

On 14 Nov 2017, at 16:00, Steffen Märcker <merkste@web.de> wrote:

Hi,

<---Schnitt--->

...

...

No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)

<---Schnitt--->

...

...

http://www.cincomsmalltalk.com/publicRepository/Transducers.html https://clojure.org/reference/transducers Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.

Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments

Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

I have a meta-comment here. As a community I feel sometimes that we are trying to make people converge. "Somebody already wrote a transducers library, I cannot write another transducers library, we have to merge". Now, I understand that sometimes we don't have the "manpower" to solve all the problems in the world, but let's not coerce people to "converge" just for converging. Most of the times, different libraries are implemented with different tradeoffs and different use case scenarios, and even with different aesthetic preferences. I think I'm saying nothing strange if I say that there is no silver bullet :). Some people may implement their own library for fun: - I don't want to prevent people from having fun - I don't want to prevent myself from having fun So Steffen, please deliver something nice, with nice docs, and have fun and thanks :) On Wed, Nov 15, 2017 at 11:12 AM, Steffen Märcker <merkste@web.de> wrote:

Short: No. =)

Actually, the <~ notation was originally proposed by another Smalltalker on the vwnc list. The idea was that <~ visualizes the flow into the variable, which I find nice (see first expression). However, I do understand that this alienates others. Hence, I am non-religious about the matter; I could even live with both messages coexisting.

Denis I agree. I do not like to code in reverse order.

...

I like abstraction. But I think names and order of computation should be

...

changed to be more Smalltalk friendly. Because now it looks like Haskell with right to left order:

squares := Set <~ 1000 take <~ #squared map <~ (1 to: 1000). fileOut writeStream <~ #isSeparator filter <~ fileIn readStream.

Is there any reason to not change it?

Best, Steffen

-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Please note, the port on GitHub is not the most recent version. I had trouble porting it to Pharo and code exchange in general. For now it's unfinished due to a lack of time. Am .11.2017, 16:30 Uhr, schrieb Steffen Märcker <merkste@web.de>:

I forgot to mention, that the most recent code for Pharo is already on Github: https://github.com/Pharophile/Transducers

Reducers was the name of the first very first implementation.

(In fact, I was originally inspired by clojures Reducers lib. After implementing it in Smalltalk, I developed the concept further. Later I found out, that the clojure guys did the same in parallel and ended up with the same abstraction but a differnt name. Hence I decided change the name in order to make the relation clear.)

Am .11.2017, 16:18 Uhr, schrieb Sven Van Caekenberghe <sven@stfx.eu>:

...

...

On 14 Nov 2017, at 16:00, Steffen Märcker <merkste@web.de> wrote:

Hi,

<---Schnitt--->

...

...

No. Transducers is my side project. I've implemented a package for VisualWorks. Unfortunately, I did not finish the port to Pharo yet, simply due to a lack of time. Originally, transducers evolved in the clojure community. I figured, the concept a good fit for Smalltalk and adapted it accordingly. (My thesis is on conditional probabilities in model-checking of probabilistic systems.)

<---Schnitt--->

...

http://www.cincomsmalltalk.com/publicRepository/Transducers.html https://clojure.org/reference/transducers Plus some mails on this list and more on the vwnc list. Feel free to ask; I understand, that the package comment has lots of potential for improvement to help understanding.

Best, Steffen

Some code seems to be here: http://smalltalkhub.com/#!/~cdlm/Experiments

Not sure if it is complete or what the relation is, or the difference between transducers and reducers ...

On 14 Nov 2017, at 10:20, Prof. Andrew P. Black <black@cs.pdx.edu> wrote:

...

On 13 Nov 2017, at 20:27 , Sven Van Caekenberghe <sven@stfx.eu> wrote:

There is a discussion about positioning (#position , #position: and related) but these cannot be supported _in general_ by the kind of streams described above.

I agree with that. But I think that general streams can, and should support:

# saveState “answers an opaque cookie” # restoreState: aCookieObtainedFromTheSameStream

This enables pone to read ahead, and then reset the state to the way that it was. The implementation of the cookie will depend on the implementation of the stream. In the simplest case it may be an integer position (sufficient for an in-core stream); in the more general case it may include a copy of the stream’s buffer, and the cookie obtained by saving the state of any wrapped stream.

Andrew

Yes and no: #saveState / #restoreState: need a possibly unlimited memory/buffer, similar to #position / #position: They cannot be offered in general (a socket stream is infinite, has no begin/end, in most cases). But I do think that a wrapper stream could add such feature, we only have to implement one and agree on the API (and the buffer size should probably be limited in size). I am curious though, why do you need it ? I mean, I got away (and try hard) to only use 1-element peeking and managed to implement STON, NeoJSON, NeoCSV, Stomp, PostgresSQL, Redis, and other protocols. A syntax that really requires multi element peeking is not so common. Sven