March 2018 - Pharo-dev - lists.pharo.org

Executive Summary of the recent FileStream Changes

by Sven Van Caekenberghe

Executive Summary of the recent FileStream Changes In Pharo 7 Guille Polito recently committed a heroic set of changes that we were planning to do for a long time but were afraid to take on. The idea is to replace a couple of fat, overly complex, multi-functional, do-all classes with a set of simpler single purpose classes that can be combined as needed. The classes that we want to get rid of can be found in the package DeprecatedFileSystem, in particular FileStream, StandardFileStream, MultiByteFileStream, MultiByteBinaryOrTextStream and RWBinaryOrTextStream. The replacements are can be found in packages Files and Zinc-Character-Encoding-Core. Encoding and decoding characters to and from bytes is done using classes that you wrap around a more primitive binary stream. The same goes for buffering or translating line endings. For example, '/Users/sven/Desktop/foo.txt' asFileReference binaryReadStream. gives you a ZnBufferedWriteStream wrapping a BinaryWriteStream. While, '/Users/sven/Desktop/foo.txt' asFileReference readStream. gives a ZnCharacterReadStream wrapping a ZnBufferedWriteStream wrapping a BinaryWriteStream. To translate line endings, we would wrap a ZnCharacterWriteStream using a ZnCrPortableWriteStream. There are a couple of more specialised streams to cover special cases (like read and writing at the same time). SocketStream remains another fat, overly complex, multi-functional, do-all class, for which usable replacements exist in the form of ZdcSocketStream and ZdcSecureSocketStream, which are simpler, cleaner and binary only. Of course, switching is more than replacing one class with a 100% compatible alternative, that would give us the same complex result. The challenge is to use a simpler API as well, to rethink how the streams are used. You know, KISS. Of course, we are far from done and need more testing, debugging and help from as many people as possible. Sven -- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org

March 15, 2018

FileHandle creates a buffered stream on a buffered stream

by Sven Van Caekenberghe

Hi (Guille), FileHandle creates a buffered stream on a buffered stream when sent #readStream. Note how #binaryReadStream adds a bufferer stream as well as #readStreamEncoded: Probably the latter should not do this Sven

March 15, 2018

Re: [Pharo-dev] Experiment: New Download page based on Pharo Launcher

by Marcus Denker

-> better structure -> simplified the large print (no need to talk about downloading zip files) TODO: - add back a section about downloading images. > On 14 Mar 2018, at 18:18, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > Ok, I will fix that in the next pass. > >> On 9 Mar 2018, at 18:10, Sean P. DeNigris <sean(a)clipperadams.com> wrote: >> >> Marcus Denker-4 wrote >>> comments welcomeâ¦ >> >> IHMO the structure is a little confusing because the first heading "Usage" >> (Launcher implied) is on the same level as other methods like "Linux >> Packages". I think there needs to be something like a header one level >> higher like "Other Options" between these two sections. >> >> >> >> ----- >> Cheers, >> Sean >> -- >> Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html >> >

March 15, 2018

Re: [Pharo-dev] Pharo 7 new file stream encoders

by Sven Van Caekenberghe

I think the idea is to use ZnCrPortableWriteStream for that. > On 15 Mar 2018, at 14:58, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Hi all, > > how does one create a file stream in Pharo 7 that forces a specific > line end convention ? For example, for forcing all end of lines to be > Character lf. > > The old way, > > fileStream lineEndConvention: #'lf'. > > does not work anymore. > > Thanks, > > Thierry >

March 15, 2018

Pharo 7 new file stream encoders

by Thierry Goubier

Hi all, how does one create a file stream in Pharo 7 that forces a specific line end convention ? For example, for forcing all end of lines to be Character lf. The old way, fileStream lineEndConvention: #'lf'. does not work anymore. Thanks, Thierry

March 15, 2018

Re: [Pharo-dev] Base64MimeConverter class>>mimeDecodeToBytes: change breaks image load

by Sven Van Caekenberghe

And in PNGReadWriter>>nextImage we should remove stream reset and stream binary. This probably goes for the other image readers as well. > On 15 Mar 2018, at 13:38, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Yeah, right, good catch. > > But what is this nonsense: > > PNGReadWriter>>on: aStream > (stream := aStream) reset. > stream binary > "Note that 'reset' makes a file be text. Must do this after." > > Of course you can only read an image from a binary stream, there should be nothing else there but > > stream := aStream > >> On 15 Mar 2018, at 13:31, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> Also: >> >> 'apicture.png' asFileReference inspect. >> >> Will raise a "image format not recognised" because: >> >> - AbstractFileReference>>binaryReadStreamDo: ultimately uses a >> ZnBufferedReadStream >> - ImageReadWriter>>on: sends #reset to the stream >> >> and ZnBufferedReadStream doesn't implement #reset. >> >> My guess is that ZnBufferedReadStream should implement #reset (it has >> #position:). But I'm not familiar with the current thinking on stream >> rationalisation. >> >> I'll raise some fogbugz issues a bit later. >> >> Image downloaded today: >> >> Pharo 7.0 >> Build information: >> Pharo-7.0+alpha.build.696.sha.a26248bb90ee8dfba131f2f4d71ba2bbbad5d00d >> (64 Bit) >> >> >> >> Thanks, >> Alistair >> >> >> On 15 March 2018 at 13:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> BTW, for the record, we should deprecate Base64MimeConverter, and remove/change the following convenience methods: >>> >>> Remove >>> >>> String>>#base64Encoded. >>> >>> Change >>> >>> ByteArray>>#base64Encoded >>> "Encode the receiver using Base64, see ZnBase64Encoder" >>> >>> ^ ZnBase64Encoder new encode: self >>> >>> String>>#base64Decoded >>> "Decode the receiver using Base64, see ZnBase64Encoder" >>> >>> ^ ZnBase64Encoder new decode: self >>> >>> The thing is, Base64 is defined as going from bytes -> characters and vice versa. Not as going from characters -> characters. For that you need to use an explicit character encoder as in ZnUtils class>>#decodeBase64: and ZnUtils>>#encodeBase64: >>> >>> So yes, this would be a breaking change, but an easy one to follow since Base64 is really very simple as a concept. >>> >>>> On 15 Mar 2018, at 12:47, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> >>>> >>>>> On 15 Mar 2018, at 12:28, Alistair Grant <akgrant0710(a)gmail.com> wrote: >>>>> >>>>> Hi Guille & Pavel, >>>>> >>>>> On 6 Feb Guille changed Base64MimeConverter class>>mimeDecodeToBytes: >>>>> so that it doesn't reset the dataStream position back to the start of >>>>> the stream. This breaks Pavel's PlayingCard, part of FreeCell (you >>>>> can see that I only load important stuff :-)). >>>>> >>>>> PlayingCard class>>initialize >>>>> "PlayingCard initialize" >>>>> | forms f | >>>>> "Read the stored forms from mime-encoded data in imageData." >>>>> f := Base64MimeConverter mimeDecodeToBytes: (ReadStream on: self imageData). >>>>> forms := OrderedCollection new. >>>>> f next = 2 ifFalse: [self error: 'corrupted imageData']. >>>>> [f atEnd] whileFalse: [forms add: (Form new readFrom: f)]. >>>>> ... >>>>> >>>>> >>>>> Previously, f would be at the start of the stream (which makes sense >>>>> given this usage), but now it is at the end of the stream, so returns >>>>> nil and "f next = 2" fails. >>>>> >>>>> Guille, can you explain the change. The commit log just says "make tests run". >>>>> >>>>> Note that a lot of users of this won't fail as they just call >>>>> #contents of the stream after #mimeDecodeToBytes:, which works >>>>> regardless of the current position. >>>> >>>> So doing >>>> >>>> f reset >>>> >>>> at the caller site would fix it then ? >>>> >>>> Note that there is the newer ZnBase64Encoder that can be used as follows: >>>> >>>> f := (ZnBase64Encoder new decode: self imageData) readStream. >>>> >>>> Sven >>> >>> >> >

March 15, 2018

Re: [Pharo-dev] Base64MimeConverter class>>mimeDecodeToBytes: change breaks image load

by Sven Van Caekenberghe

Yeah, right, good catch. But what is this nonsense: PNGReadWriter>>on: aStream (stream := aStream) reset. stream binary "Note that 'reset' makes a file be text. Must do this after." Of course you can only read an image from a binary stream, there should be nothing else there but stream := aStream > On 15 Mar 2018, at 13:31, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > Also: > > 'apicture.png' asFileReference inspect. > > Will raise a "image format not recognised" because: > > - AbstractFileReference>>binaryReadStreamDo: ultimately uses a > ZnBufferedReadStream > - ImageReadWriter>>on: sends #reset to the stream > > and ZnBufferedReadStream doesn't implement #reset. > > My guess is that ZnBufferedReadStream should implement #reset (it has > #position:). But I'm not familiar with the current thinking on stream > rationalisation. > > I'll raise some fogbugz issues a bit later. > > Image downloaded today: > > Pharo 7.0 > Build information: > Pharo-7.0+alpha.build.696.sha.a26248bb90ee8dfba131f2f4d71ba2bbbad5d00d > (64 Bit) > > > > Thanks, > Alistair > > > On 15 March 2018 at 13:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> BTW, for the record, we should deprecate Base64MimeConverter, and remove/change the following convenience methods: >> >> Remove >> >> String>>#base64Encoded. >> >> Change >> >> ByteArray>>#base64Encoded >> "Encode the receiver using Base64, see ZnBase64Encoder" >> >> ^ ZnBase64Encoder new encode: self >> >> String>>#base64Decoded >> "Decode the receiver using Base64, see ZnBase64Encoder" >> >> ^ ZnBase64Encoder new decode: self >> >> The thing is, Base64 is defined as going from bytes -> characters and vice versa. Not as going from characters -> characters. For that you need to use an explicit character encoder as in ZnUtils class>>#decodeBase64: and ZnUtils>>#encodeBase64: >> >> So yes, this would be a breaking change, but an easy one to follow since Base64 is really very simple as a concept. >> >>> On 15 Mar 2018, at 12:47, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> >>> >>>> On 15 Mar 2018, at 12:28, Alistair Grant <akgrant0710(a)gmail.com> wrote: >>>> >>>> Hi Guille & Pavel, >>>> >>>> On 6 Feb Guille changed Base64MimeConverter class>>mimeDecodeToBytes: >>>> so that it doesn't reset the dataStream position back to the start of >>>> the stream. This breaks Pavel's PlayingCard, part of FreeCell (you >>>> can see that I only load important stuff :-)). >>>> >>>> PlayingCard class>>initialize >>>> "PlayingCard initialize" >>>> | forms f | >>>> "Read the stored forms from mime-encoded data in imageData." >>>> f := Base64MimeConverter mimeDecodeToBytes: (ReadStream on: self imageData). >>>> forms := OrderedCollection new. >>>> f next = 2 ifFalse: [self error: 'corrupted imageData']. >>>> [f atEnd] whileFalse: [forms add: (Form new readFrom: f)]. >>>> ... >>>> >>>> >>>> Previously, f would be at the start of the stream (which makes sense >>>> given this usage), but now it is at the end of the stream, so returns >>>> nil and "f next = 2" fails. >>>> >>>> Guille, can you explain the change. The commit log just says "make tests run". >>>> >>>> Note that a lot of users of this won't fail as they just call >>>> #contents of the stream after #mimeDecodeToBytes:, which works >>>> regardless of the current position. >>> >>> So doing >>> >>> f reset >>> >>> at the caller site would fix it then ? >>> >>> Note that there is the newer ZnBase64Encoder that can be used as follows: >>> >>> f := (ZnBase64Encoder new decode: self imageData) readStream. >>> >>> Sven >> >> >

March 15, 2018

Re: [Pharo-dev] Base64MimeConverter class>>mimeDecodeToBytes: change breaks image load

by Alistair Grant

Also: 'apicture.png' asFileReference inspect. Will raise a "image format not recognised" because: - AbstractFileReference>>binaryReadStreamDo: ultimately uses a ZnBufferedReadStream - ImageReadWriter>>on: sends #reset to the stream and ZnBufferedReadStream doesn't implement #reset. My guess is that ZnBufferedReadStream should implement #reset (it has #position:). But I'm not familiar with the current thinking on stream rationalisation. I'll raise some fogbugz issues a bit later. Image downloaded today: Pharo 7.0 Build information: Pharo-7.0+alpha.build.696.sha.a26248bb90ee8dfba131f2f4d71ba2bbbad5d00d (64 Bit) Thanks, Alistair On 15 March 2018 at 13:08, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > BTW, for the record, we should deprecate Base64MimeConverter, and remove/change the following convenience methods: > > Remove > > String>>#base64Encoded. > > Change > > ByteArray>>#base64Encoded > "Encode the receiver using Base64, see ZnBase64Encoder" > > ^ ZnBase64Encoder new encode: self > > String>>#base64Decoded > "Decode the receiver using Base64, see ZnBase64Encoder" > > ^ ZnBase64Encoder new decode: self > > The thing is, Base64 is defined as going from bytes -> characters and vice versa. Not as going from characters -> characters. For that you need to use an explicit character encoder as in ZnUtils class>>#decodeBase64: and ZnUtils>>#encodeBase64: > > So yes, this would be a breaking change, but an easy one to follow since Base64 is really very simple as a concept. > >> On 15 Mar 2018, at 12:47, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> >> >>> On 15 Mar 2018, at 12:28, Alistair Grant <akgrant0710(a)gmail.com> wrote: >>> >>> Hi Guille & Pavel, >>> >>> On 6 Feb Guille changed Base64MimeConverter class>>mimeDecodeToBytes: >>> so that it doesn't reset the dataStream position back to the start of >>> the stream. This breaks Pavel's PlayingCard, part of FreeCell (you >>> can see that I only load important stuff :-)). >>> >>> PlayingCard class>>initialize >>> "PlayingCard initialize" >>> | forms f | >>> "Read the stored forms from mime-encoded data in imageData." >>> f := Base64MimeConverter mimeDecodeToBytes: (ReadStream on: self imageData). >>> forms := OrderedCollection new. >>> f next = 2 ifFalse: [self error: 'corrupted imageData']. >>> [f atEnd] whileFalse: [forms add: (Form new readFrom: f)]. >>> ... >>> >>> >>> Previously, f would be at the start of the stream (which makes sense >>> given this usage), but now it is at the end of the stream, so returns >>> nil and "f next = 2" fails. >>> >>> Guille, can you explain the change. The commit log just says "make tests run". >>> >>> Note that a lot of users of this won't fail as they just call >>> #contents of the stream after #mimeDecodeToBytes:, which works >>> regardless of the current position. >> >> So doing >> >> f reset >> >> at the caller site would fix it then ? >> >> Note that there is the newer ZnBase64Encoder that can be used as follows: >> >> f := (ZnBase64Encoder new decode: self imageData) readStream. >> >> Sven > >

March 15, 2018

Re: [Pharo-dev] Base64MimeConverter class>>mimeDecodeToBytes: change breaks image load

by Sven Van Caekenberghe

BTW, for the record, we should deprecate Base64MimeConverter, and remove/change the following convenience methods: Remove String>>#base64Encoded. Change ByteArray>>#base64Encoded "Encode the receiver using Base64, see ZnBase64Encoder" ^ ZnBase64Encoder new encode: self String>>#base64Decoded "Decode the receiver using Base64, see ZnBase64Encoder" ^ ZnBase64Encoder new decode: self The thing is, Base64 is defined as going from bytes -> characters and vice versa. Not as going from characters -> characters. For that you need to use an explicit character encoder as in ZnUtils class>>#decodeBase64: and ZnUtils>>#encodeBase64: So yes, this would be a breaking change, but an easy one to follow since Base64 is really very simple as a concept. > On 15 Mar 2018, at 12:47, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > >> On 15 Mar 2018, at 12:28, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> Hi Guille & Pavel, >> >> On 6 Feb Guille changed Base64MimeConverter class>>mimeDecodeToBytes: >> so that it doesn't reset the dataStream position back to the start of >> the stream. This breaks Pavel's PlayingCard, part of FreeCell (you >> can see that I only load important stuff :-)). >> >> PlayingCard class>>initialize >> "PlayingCard initialize" >> | forms f | >> "Read the stored forms from mime-encoded data in imageData." >> f := Base64MimeConverter mimeDecodeToBytes: (ReadStream on: self imageData). >> forms := OrderedCollection new. >> f next = 2 ifFalse: [self error: 'corrupted imageData']. >> [f atEnd] whileFalse: [forms add: (Form new readFrom: f)]. >> ... >> >> >> Previously, f would be at the start of the stream (which makes sense >> given this usage), but now it is at the end of the stream, so returns >> nil and "f next = 2" fails. >> >> Guille, can you explain the change. The commit log just says "make tests run". >> >> Note that a lot of users of this won't fail as they just call >> #contents of the stream after #mimeDecodeToBytes:, which works >> regardless of the current position. > > So doing > > f reset > > at the caller site would fix it then ? > > Note that there is the newer ZnBase64Encoder that can be used as follows: > > f := (ZnBase64Encoder new decode: self imageData) readStream. > > Sven

March 15, 2018

Re: [Pharo-dev] Base64MimeConverter class>>mimeDecodeToBytes: change breaks image load

by Alistair Grant

Hi Sven, On 15 March 2018 at 12:47, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > >> On 15 Mar 2018, at 12:28, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> Hi Guille & Pavel, >> >> On 6 Feb Guille changed Base64MimeConverter class>>mimeDecodeToBytes: >> so that it doesn't reset the dataStream position back to the start of >> the stream. This breaks Pavel's PlayingCard, part of FreeCell (you >> can see that I only load important stuff :-)). >> >> PlayingCard class>>initialize >> "PlayingCard initialize" >> | forms f | >> "Read the stored forms from mime-encoded data in imageData." >> f := Base64MimeConverter mimeDecodeToBytes: (ReadStream on: self imageData). >> forms := OrderedCollection new. >> f next = 2 ifFalse: [self error: 'corrupted imageData']. >> [f atEnd] whileFalse: [forms add: (Form new readFrom: f)]. >> ... >> >> >> Previously, f would be at the start of the stream (which makes sense >> given this usage), but now it is at the end of the stream, so returns >> nil and "f next = 2" fails. >> >> Guille, can you explain the change. The commit log just says "make tests run". >> >> Note that a lot of users of this won't fail as they just call >> #contents of the stream after #mimeDecodeToBytes:, which works >> regardless of the current position. > > So doing > > f reset > > at the caller site would fix it then ? Yes, but I was thinking more about whether it was intended to put this responsibility on the caller (changing behaviour). > Note that there is the newer ZnBase64Encoder that can be used as follows: > > f := (ZnBase64Encoder new decode: self imageData) readStream. That works too. Thanks, Alistair

March 15, 2018