Replacement for deprected MultiByteFileStream?
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream. Can someone please point me to the right code? Bernhard
Hi Bernhard. The deprecation is still in progress. Now to setup encoding for MultiByteFileStream you should set #converter: (FileLocator imageDirectory / 'test.txt') asFileReference readStreamDo: [ :s | s converter: UTF8TextConverter new. s contents ] 2017-12-30 13:27 GMT+01:00 Bernhard Pieber <bernhard@pieber.com>:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
On Sun, Dec 31, 2017 at 12:55 PM, Stephan Eggermont <stephan@stack.nl> wrote:
Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi Bernhard.
The deprecation is still in progress.
Considering the amount of non-image code using this, I would be interested to hear how this is going to be supported and documented.
Step by step. :) We cannot continue to have so bad streams without taking action. So we start to make sure that Pharo itself does not use the old streams. Then it means that we will be able to have the old stream library as a separate package and people will slowly migrate. Now it would be nice to have a rewrite rule to help but it requires time. Stef
Hi Denis, Thank you for your answer. However, setting the UTF8TextConverter did not help because of a bug in upToAll:. I created a pull request to fix it: https://github.com/pharo-project/pharo/pull/632 I am afraid the CI failed for some reason I don't understand. :-/ The UTF8TextConverter was set by default, by the way. Bernhard
Am 30.12.2017 um 13:57 schrieb Denis Kudriashov <dionisiydk@gmail.com>:
Hi Bernhard.
The deprecation is still in progress. Now to setup encoding for MultiByteFileStream you should set #converter:
(FileLocator imageDirectory / 'test.txt') asFileReference readStreamDo: [ :s | s converter: UTF8TextConverter new. s contents ]
2017-12-30 13:27 GMT+01:00 Bernhard Pieber <bernhard@pieber.com>: I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
On 31 Dec 2017, at 16:01, Bernhard Pieber <bernhard@pieber.com> wrote:
I created a pull request to fix it: https://github.com/pharo-project/pharo/pull/632
This implementation (which is not using #position) is acceptable. Nice tests. I am still a bit puzzled by the usefulness of this method though.
Hi Bernhard Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.) He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream. writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag | extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. outputStream close. writeSourceCodeFrom: aStream toFileReference: aFileReference aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ]. self inform: 'Filed out to: ', String cr, aFileReference basename self inform: 'Filed out to: ', String cr, fileName. I hope it helps. Stef On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Hi Stef, Thank you for your answer pointing me to the ZnCharacter...Stream. This seems to be meant as the replacement I was looking for. I will have a closer look. Bernhard
Am 30.12.2017 um 17:35 schrieb Stephane Ducasse <stepharo.self@gmail.com>:
Hi Bernhard
Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.)
He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream.
writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag
| extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents.
outputStream close.
writeSourceCodeFrom: aStream toFileReference: aFileReference
aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ].
self inform: 'Filed out to: ', String cr, aFileReference basename
self inform: 'Filed out to: ', String cr, fileName.
I hope it helps.
Stef
On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Stephane Ducasse-3 wrote
Hi Bernhard
Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.)
He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream.
writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag
| extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents.
outputStream close.
writeSourceCodeFrom: aStream toFileReference: aFileReference
aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ].
self inform: 'Filed out to: ', String cr, aFileReference basename
self inform: 'Filed out to: ', String cr, fileName.
I hope it helps.
Stef
On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <
bernhard@
> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Maybe it's just me, but I'd a big fan of convenience methods over explicit stacking when it comes to multi-layer streams. - It documents the intended/valid ways to stack (such as which streams it makes sense to wrap in a ZnCrPortableWriteStream, or which streams one normally would want to #buffered:do:) - It makes the transition (in terms of conciseness) from "traditional" filestreams less jarring (provided convenience methods provide sensible defaults). Say, for instance, a #withEncoding:do:* on read/writeStream, so instead of the above, one can do: fileName := FileSystem disk checkName: fileName fixErrors: true. (outputStream :=(File named: fileName) writeStream) withEncoding: 'utf8' do: [:encodedWriteStream | encodedWriteStream nextPutAll: aStream contents]. outputStream close. and aFileReference writeStreamDo: [ :outputStream | outputStream withEncoding: 'utf8' do: [:encodedStream | encodedStream nextPutAll: aStream contents. ]]. or input := aFileReference readStreamDo: [ :inputStream | inputStream withEncoding: 'utf8' do: [:decodingStream | decodingStream contents]] It's more involved than the current multibytestreams, but takes away the burden of determining the correct stack of ZnStreams for the common case. For instance, you'd normally want file write streams to also be wrapped by a buffered stream, and have a single flush at end of withEncoding:do:, while the same isn't necessary for withEncoding:do: on the base WriteStream class. Cheers, Henry *Bad name, I know, but you get the point. -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
Hi henry I had problems to compare the two versions because they are probably on multiple mails. Could you email it? Because convenient methods are important but I do not see the point. Stef (in my constant quest for the better) On Tue, Jan 2, 2018 at 1:13 PM, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
Stephane Ducasse-3 wrote
Hi Bernhard
Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.)
He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream.
writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag
| extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents.
outputStream close.
writeSourceCodeFrom: aStream toFileReference: aFileReference
aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ].
self inform: 'Filed out to: ', String cr, aFileReference basename
self inform: 'Filed out to: ', String cr, fileName.
I hope it helps.
Stef
On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <
bernhard@
> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Maybe it's just me, but I'd a big fan of convenience methods over explicit stacking when it comes to multi-layer streams. - It documents the intended/valid ways to stack (such as which streams it makes sense to wrap in a ZnCrPortableWriteStream, or which streams one normally would want to #buffered:do:) - It makes the transition (in terms of conciseness) from "traditional" filestreams less jarring (provided convenience methods provide sensible defaults).
Say, for instance, a #withEncoding:do:* on read/writeStream, so instead of the above, one can do:
fileName := FileSystem disk checkName: fileName fixErrors: true. (outputStream :=(File named: fileName) writeStream) withEncoding: 'utf8' do: [:encodedWriteStream | encodedWriteStream nextPutAll: aStream contents]. outputStream close.
and aFileReference writeStreamDo: [ :outputStream | outputStream withEncoding: 'utf8' do: [:encodedStream | encodedStream nextPutAll: aStream contents. ]]. or input := aFileReference readStreamDo: [ :inputStream | inputStream withEncoding: 'utf8' do: [:decodingStream | decodingStream contents]]
It's more involved than the current multibytestreams, but takes away the burden of determining the correct stack of ZnStreams for the common case. For instance, you'd normally want file write streams to also be wrapped by a buffered stream, and have a single flush at end of withEncoding:do:, while the same isn't necessary for withEncoding:do: on the base WriteStream class.
Cheers, Henry
*Bad name, I know, but you get the point.
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
On Tue, Jan 2, 2018 at 1:13 PM, Henrik Sperre Johansen < henrik.s.johansen@veloxit.no> wrote:
Stephane Ducasse-3 wrote
Hi Bernhard
Have a look at how guille started to implement File package. (Guille is actively converting all the image users of the old hierarchy but last time he losts a couple of hours because changing epicea and other users of positionable streams and sourceArray... at the same time was leading to complex situations :). But Guille told me that he learned and will come back to this.)
He is using ZnCharacterWriteStream and wrapper like ZnCrPortableWriteStream.
writeSourceCodeFrom: aStream baseName: baseName isSt: stOrCsFlag
| extension fileName outputStream | extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. fileName := baseName, extension. fileName := FileSystem disk checkName: fileName fixErrors: true. outputStream := (File named: fileName) writeStream. (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents.
outputStream close.
writeSourceCodeFrom: aStream toFileReference: aFileReference
aFileReference writeStreamDo: [ :outputStream | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: outputStream encoding: 'utf8')) nextPutAll: aStream contents. ].
self inform: 'Filed out to: ', String cr, aFileReference basename
self inform: 'Filed out to: ', String cr, fileName.
I hope it helps.
Stef
On Sat, Dec 30, 2017 at 1:27 PM, Bernhard Pieber <
bernhard@
> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Maybe it's just me, but I'd a big fan of convenience methods over explicit stacking when it comes to multi-layer streams. - It documents the intended/valid ways to stack (such as which streams it makes sense to wrap in a ZnCrPortableWriteStream, or which streams one normally would want to #buffered:do:) - It makes the transition (in terms of conciseness) from "traditional" filestreams less jarring (provided convenience methods provide sensible defaults).
Say, for instance, a #withEncoding:do:* on read/writeStream, so instead of the above, one can do:
fileName := FileSystem disk checkName: fileName fixErrors: true. (outputStream :=(File named: fileName) writeStream) withEncoding: 'utf8' do: [:encodedWriteStream | encodedWriteStream nextPutAll: aStream contents]. outputStream close.
and aFileReference writeStreamDo: [ :outputStream | outputStream withEncoding: 'utf8' do: [:encodedStream | encodedStream nextPutAll: aStream contents. ]]. or input := aFileReference readStreamDo: [ :inputStream | inputStream withEncoding: 'utf8' do: [:decodingStream | decodingStream contents]]
It's more involved than the current multibytestreams, but takes away the burden of determining the correct stack of ZnStreams for the common case. For instance, you'd normally want file write streams to also be wrapped by a buffered stream, and have a single flush at end of withEncoding:do:, while the same isn't necessary for withEncoding:do: on the base WriteStream class.
It's not just you :). I was during last sprint (while seeing such idioms appear) adding convenience methods like writeStreamEncoded: anEncoding writeStreamDo: aBlock encoded: anEncoding readStreamEncoded: anEncoding readStreamDo: aBlock encoded: anEncoding But my image fantastically crashed, the save image failed and I lost everything...
Cheers, Henry
*Bad name, I know, but you get the point.
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html
-- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu... 1.4. Converting Streams
On 30 Dec 2017, at 13:27, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Which reminds me that we should publish these HTML in another location.... Just too many things to do :( But at least we get pdf and HTML and more if needed for our books. On Sat, Dec 30, 2017 at 6:03 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
1.4. Converting Streams
On 30 Dec 2017, at 13:27, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
Hi Sven, Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation. However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams? Bernhard
Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
1.4. Converting Streams
On 30 Dec 2017, at 13:27, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
On 31 Dec 2017, at 16:21, Bernhard Pieber <bernhard@pieber.com> wrote:
Hi Sven,
Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation.
However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams?
The Stream API is much too wide, it has to be reduced and split in a number of elementary streams that can be composed to add desired behaviour (like character encoding/decoding, buffering, line end translation, ...). Positioning in a stream (moving backwards without restrictions) is not generally possible (think of a network stream), that is why it is not implemented.
Bernhard
Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
1.4. Converting Streams
On 30 Dec 2017, at 13:27, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
IIUC that means that a stream implementation compatible with the standard streams cannot be implemented on top of the new streams, right? What is the plan about #upToAll:, #match: and similar messages which are very useful for parsing? Where should they go? Bernhard
Am 31.12.2017 um 17:07 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 31 Dec 2017, at 16:21, Bernhard Pieber <bernhard@pieber.com> wrote:
Hi Sven,
Thanks for the answer. This is exactly what I was looking for. And it is great that there is also documentation.
However, ZnCharacterReadStream does not implement #upToAll:. The implementation from PositionableStream cannot easily be borrowed because there seems to be no way to reset the position. Is there a plan to support the most commonly used messages from the normal streams?
The Stream API is much too wide, it has to be reduced and split in a number of elementary streams that can be composed to add desired behaviour (like character encoding/decoding, buffering, line end translation, ...). Positioning in a stream (moving backwards without restrictions) is not generally possible (think of a network stream), that is why it is not implemented.
Bernhard
Am 30.12.2017 um 18:03 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfu...
1.4. Converting Streams
On 30 Dec 2017, at 13:27, Bernhard Pieber <bernhard@pieber.com> wrote:
I just saw that the FileStream hierarchy including MultiByteFileStream is deprecated. However, I can't find what file stream class should be used to read a UTF-8 encoded file instead. FileReference>>#readStreamDo: also uses MultiByteFileStream.
Can someone please point me to the right code?
Bernhard
participants (7)
-
Bernhard Pieber -
Denis Kudriashov -
Guillermo Polito -
Henrik Sperre Johansen -
Stephan Eggermont -
Stephane Ducasse -
Sven Van Caekenberghe