[Pharo-dev] FileStreams and encoding ( opening a readStream on a FileRerefence)

Sven Van Caekenberghe sven at stfx.eu
Tue Feb 7 05:15:22 EST 2017


OK, I committed the following in Zn #bleedingEdge:

===
Name: Zinc-Character-Encoding-Core-SvenVanCaekenberghe.46
Author: SvenVanCaekenberghe
Time: 7 February 2017, 11:07:31.306364 am
UUID: a2928299-1dda-4fdf-b15e-e5d7bed2373e
Ancestors: Zinc-Character-Encoding-Core-SvenVanCaekenberghe.45

Add ZnCharacterEncoder class>>#knownEncodingIdentifiers to return the collection of all known encoding identifiers in the system - thx Nicolai Hess for the request

Make sure #null is resolved to ZnNullEncoder

Add ZnCharacterEncoderTests>>#testKnownEncodingIdentifiers
===
Name: Zinc-Character-Encoding-Tests-SvenVanCaekenberghe.28
Author: SvenVanCaekenberghe
Time: 7 February 2017, 11:07:50.782471 am
UUID: 938d1683-b8c4-4861-b1e1-f876b57405ef
Ancestors: Zinc-Character-Encoding-Tests-SvenVanCaekenberghe.27

Add ZnCharacterEncoder class>>#knownEncodingIdentifiers to return the collection of all known encoding identifiers in the system - thx Nicolai Hess for the request

Make sure #null is resolved to ZnNullEncoder

Add ZnCharacterEncoderTests>>#testKnownEncodingIdentifiers
===

Thanks again for the feedback, this is a useful addition.

> On 6 Feb 2017, at 22:33, Nicolai Hess <nicolaihess at gmail.com> wrote:
> 
> 
> 
> 2017-02-04 13:40 GMT+01:00 Sven Van Caekenberghe <sven at stfx.eu>:
> 
> > On 4 Feb 2017, at 13:01, Nicolai Hess <nicolaihess at gmail.com> wrote:
> >
> >
> >
> > 2017-02-04 12:49 GMT+01:00 Sven Van Caekenberghe <sven at stfx.eu>:
> > Hi Nicolai,
> >
> > The FileSystem API is a bit inconsistent, yes.
> >
> > This is how you can use it:
> >
> > (FileLocator temp / 'foo.txt') writeStreamDo: [ :out |
> >   out binary.
> >   (ZnCharacterWriteStream on: out encoding: #utf8) << 'élève' ].
> >
> > (FileLocator temp / 'foo.txt') readStreamDo: [ :in |
> >   in binary.
> >   ZnCharacterReadStream on: in encoding: #utf8) upToEnd ].
> >
> > (FileLocator temp / 'foo.txt') binaryReadStreamDo: [ :in |
> >   (ZnCharacterReadStream on: in encoding: #utf8) upToEnd ].
> >
> > There is no #binaryWriteStreamDo:
> >
> > The API around File is more correct, IMHO.
> >
> > Does this help ?
> >
> > Yes, thanks for the fast response.
> >
> > What exactly is your question ?
> >
> > I am looking at the issues with FileList, there are some parts that don't work anymore (see FileList>>#contents, it calls some unimplemted methods), and it uses TextConverter and somoe parts of the older File api.
> > It looks like most other (newer) parts are using ZnCharacterReadstream for encoding, but I couldn't find a way to use it together with FileReferences (most parts of the FileList already operate with the newer FileSystem API.
> 
> OK.
> 
> From my standpoint and understanding, I would always use binary streams with explicit Zn converters, this combination is much easier to understand and better implemented with more features.
> 
> If you encounter any problem or have any questions, I will gladly try to help you.
> 
> :-)
> Is there a way to list all supported encodings, similar to TextConverter class >> #allEncodingNames ?
> I only found 
> ZnSimplifiedByteEncoder class >> #knownEncodingIdentifiers
> and
> ZnByteEncoder class >> #knownEncodingIdentifiers
> and nothing for all supported utf-8/16 ... encodings.
> 
> nicolai
>  
> 
> > Sven
> >
> > > On 4 Feb 2017, at 12:09, Nicolai Hess <nicolaihess at gmail.com> wrote:
> > >
> > > Hi
> > > How can I specify the character encoding when opening a readStream on a FileRerefence.
> > >
> > > I found this, that works:
> > >
> > > | readStream fileContent |
> > > readStream := (File named: aFileName) openForRead.
> > > fileContent := ZnCharacterReadStream on: readStream encoding: encoding.
> > >  fileContent upToEnd asString.
> > >
> > > But if I try to do the same with a readStream from a FileReference
> > >
> > > | readStream fileContent |
> > > readStream := aFileName asFileReference readStream.
> > > fileContent := ZnCharacterReadStream on: readStream encoding: encoding.
> > >  fileContent upToEnd asString.
> > >
> > > I get an error SmallInteger DNU #asciiValue,
> > >
> > > this is because, in the first method, we create a binary filestream, and if we
> > > use readStream from a FileReference, the stream is a MultibyteFileStream.
> > >
> > > How can I us ZnEncoder for a readstream from a FileReference?
> > >
> > > (and is it on purpose that both readStream method (openForRead/readStream)
> > > return different kinds of binary streams?)
> > >
> > >
> > > nicolai




More information about the Pharo-dev mailing list