In-memory FileSystem write streams not being polymorphic
Hi all! I've been working in a fix of PharoFilesOpener and decided to test the current behavior first before changing it, so I extracted the dependency to FileSystem and tried to use an instance created through FileSystem memory for testing. The code gets the write stream for a file and then sends the #isReadOnly message to do some stuff. The thing is that the streams returned form the in-memory file system are instances of WriteStream (see MemoryHandle >> writeStream) instead of StandardFileStream (see FileHandle >> writeStream) and so they don't respond the #isReadOnly message. Is this the expected way the in-memory file system should work? Shouldn't it return polymorphic file streams? If this is a bug, what change would you recommend to do? I was thinking about returning StandardFileStreams instances from MemoryHandle >> writeStream. Cheers! Nico PM
On 2013-11-11, at 14:19, Nicolás Papagna Maldonado <nicolas.papagna@gmail.com> wrote:
Hi all!
I've been working in a fix of PharoFilesOpener and decided to test the current behavior first before changing it, so I extracted the dependency to FileSystem and tried to use an instance created through FileSystem memory for testing.
The code gets the write stream for a file and then sends the #isReadOnly message to do some stuff.
The thing is that the streams returned form the in-memory file system are instances of WriteStream (see MemoryHandle >> writeStream) instead of StandardFileStream (see FileHandle >> writeStream) and so they don't respond the #isReadOnly message.
Is this the expected way the in-memory file system should work? Shouldn't it return polymorphic file streams?
You are absolutely right, it definitely should, but this requires a bit of work (see below)
If this is a bug, what change would you recommend to do? I was thinking about returning StandardFileStreams instances from MemoryHandle >> writeStream.
The memory filesystem works by definition in the image, whereas the StandardFileStream is bound to external file operations through the primitives (see for instance #basicNext). I think you have two choices here: 1. rewrite your code that it does not rely on the #isReadOnly but rather check up front if you can read the file 2. implement your own proper FileStream that works on the internal ByteArray of the memory FileSystem (maybe there is already a decent solution somewhere in the Zinc code) or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory :/
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A) The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream. An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches. Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite. 2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
And of course, I forgot all the usages of ReadWriteStream. I started rewriting some usage in Squeak, and good news, 95% of usage is unecessary, a WriteStream is sufficient. 2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
On Nov 12, 2013, at 2:33 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
And of course, I forgot all the usages of ReadWriteStream. I started rewriting some usage in Squeak, and good news, 95% of usage is unecessary, a WriteStream is sufficient.
:) if you want to push some of these changes please do. In the past we renamed all the hardcoded names with nsStream to be able to plug another but we never went further. Stef
2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
On Nov 12, 2013, at 2:31 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
if I understand correctly I prefer A too. Because like that we can decide to even throw away later the wrapper and a wrapper is like a clear API.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
Nicolas, Is it currently possible to load some latest version of Xtream into Pharo 3.0 ? If yes, from which repository using which Configuration ? I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync. It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about. The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge. The biggest gain is of course if clients switch to the newer API ;-) Sven On 12 Nov 2013, at 14:31, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo) A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/ Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found... I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository? 2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which
on the long run will
solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...) 2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo) A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/
Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...
I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?
2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier < nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which
on the long run will
solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
Yep, they do. build.squeak.org uses this repo. See also https://code.google.com/p/xtreams/issues/detail?id=2 About time for us to pick up this work? frank On 12 November 2013 21:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...)
2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo) A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/
Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...
I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?
2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor... So Xtreams-Terminals was thought as a dialect specific package... Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging. Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints. So I created a .Pharo3 branch in Terminals and TerminalsTests. There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle. Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed. Up to now, I diffed the versions in VW and manually ported to Squeak. The good thing is that I can try to learn and understand the changes. It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...). The bad things are that - there are many versions to integrate, and the history is not linear; - I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities. There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts. Or just put more brains, finger and eyes in the loop... 2013/11/12 Frank Shearar <frank.shearar@gmail.com>
Yep, they do. build.squeak.org uses this repo. See also https://code.google.com/p/xtreams/issues/detail?id=2
About time for us to pick up this work?
frank
On 12 November 2013 21:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...)
2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo) A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/
Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams)
because
ShortRunArray is not found...
I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?
2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into
Pharo
3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;),
which
on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
So the latest ConfigurationOfXtreams found either in http://www.squeaksource.org/Xtreams or http://www.squeaksource.org/MetacelloRepository more or less load in Pharo3.0... More or less means that it still tries to load FFI which fails to compile... And I don't know why it loads, and i don't know why it fails... FFI is required only by Xtreams-Xtra which is not (or should not be) in the 'default' group for #'pharo3.x' It's not very important, because most of the Xtreams packages are already loaded when the failure occurs. When i check with this, it does not tell me that it will load FFI (maybe because I have a version loaded?): ConfigurationOfXtreams project latestVersion fetch loadDirective. But when I load with this, it tries to compile FFI again: ConfigurationOfXtreams project latestVersion load. 2013/11/13 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor... So Xtreams-Terminals was thought as a dialect specific package... Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging.
Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints. So I created a .Pharo3 branch in Terminals and TerminalsTests. There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle.
Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed. Up to now, I diffed the versions in VW and manually ported to Squeak. The good thing is that I can try to learn and understand the changes. It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...). The bad things are that - there are many versions to integrate, and the history is not linear; - I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities. There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts. Or just put more brains, finger and eyes in the loop...
2013/11/12 Frank Shearar <frank.shearar@gmail.com>
Yep, they do. build.squeak.org uses this repo. See also https://code.google.com/p/xtreams/issues/detail?id=2
About time for us to pick up this work?
frank
On 12 November 2013 21:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...)
2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above
repo)
A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/
Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...
I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?
2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into
Pharo
3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;),
which
on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
Thanks nicolas for checking that. Stef
So the latest ConfigurationOfXtreams found either in http://www.squeaksource.org/Xtreams or http://www.squeaksource.org/MetacelloRepository more or less load in Pharo3.0... More or less means that it still tries to load FFI which fails to compile... And I don't know why it loads, and i don't know why it fails... FFI is required only by Xtreams-Xtra which is not (or should not be) in the 'default' group for #'pharo3.x' It's not very important, because most of the Xtreams packages are already loaded when the failure occurs.
When i check with this, it does not tell me that it will load FFI (maybe because I have a version loaded?): ConfigurationOfXtreams project latestVersion fetch loadDirective.
But when I load with this, it tries to compile FFI again: ConfigurationOfXtreams project latestVersion load.
2013/11/13 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> In original VW works, there is no FileDirectory nor FileReference nor FileSystem nor... So Xtreams-Terminals was thought as a dialect specific package... Of course, these terminals are still 95% compatible between Squeak4.x and Pharo3.x, so we can make a better packaging.
Before deciding what is a good package delimitation, and what's not, I wanted to experiment a bit without compatibility constraints. So I created a .Pharo3 branch in Terminals and TerminalsTests. There's a little more than FileDirectory references: I also used the FileHandle class provided by FileSystem package, rather than Xtreams-Support XTIOFileHandle.
Anyway, the Squeak/Pharo version is way behind the visualworks version, so yes, there is definitely some work needed. Up to now, I diffed the versions in VW and manually ported to Squeak. The good thing is that I can try to learn and understand the changes. It is also mandatory to cop with the small differences (some VW method might be absent in Squeak/Pharo and require a Xtreams-Support counterpart, the classes need a XT prefix, ...). The bad things are that - there are many versions to integrate, and the history is not linear; - I have only ten fingers, two eyes, and a single brain, the whole comes with very limited multitasking capabilities. There must be a better way, like trying to automate the port of shared implementations, and review only the dialect-specific parts. Or just put more brains, finger and eyes in the loop...
2013/11/12 Frank Shearar <frank.shearar@gmail.com> Yep, they do. build.squeak.org uses this repo. See also https://code.google.com/p/xtreams/issues/detail?id=2
About time for us to pick up this work?
frank
On 12 November 2013 21:29, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
And the tests pass, except the file tests which are based on creating a stream thru (FileDirectory default / ...)
2013/11/12 Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com>
Hi Sven, The last Squeak/Pharo locations I am aware of are: - http://www.squeaksource.org/Xtreams for the Xtreams package - http://www.squeaksource.org/MetacelloRepository for ConfigurationOfXtreams (but I just copied the last Config to above repo) A very important repository for resources of all kind from the original authors is https://code.google.com/p/xtreams/
Those versions above do not load in Pharo3.0 due to several problems - FileDirectory deprecation (there is an extension method that should be removed in Pharo branch) - Xtras package of Xtreams depends on FFI, and FFI does not load cleanly in Pharo (at least the one found from the ConfigurationOfXtreams) because ShortRunArray is not found...
I will try and publish a Pharo branch, but before I do so, is anyone aware of more recent work, or repository?
2013/11/12 Sven Van Caekenberghe <sven@stfx.eu>
Nicolas,
Is it currently possible to load some latest version of Xtream into Pharo 3.0 ?
If yes, from which repository using which Configuration ?
I know that at ESUG 2012, Sean and Martin worked a bit on getting all different versions better in sync.
It would be cool if we could at least load it, separate from the transition strategy (I agree with your proposal BTW), because many people do not know or have not seen what we are actually talking about.
The clean, start from scratch approach of Xtreams also includes a much tighter and semantically better defined API. IMHO, a consequence is that #get / #next and #put: / #nextPut: are not just plain aliases (a modern use of exception handling is one big difference). The compatibility layer might be more of a challenge.
The biggest gain is of course if clients switch to the newer API ;-)
Sven
On 12 Nov 2013, at 14:31, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
2013/11/12 Stéphane Ducasse <stephane.ducasse@inria.fr>
or of course, you start looking at porting XStreams to pharo ;), which on the long run will solve many more problems. The current situation is not that satisfactory
having experience with it and thinking about a plan for the beginning of 40 would be great. I know that nicolas ported XTream to pharo/squeak. Now understanding how integrate it would be nice. Stef
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API? This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
Yes, a Wrapper would provide the legacy API. And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more. If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever). 2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness. "All sort of un-needed complexity?" That's overstating it a bit, don't you think? So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but... what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API? IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code? On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API. So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not. As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names. frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
Exactly, every specialized stream has its specialized API and/or specialized implementation. File streams don't even have a name, because they need not to. You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing. The file has a name, the stream does not need any. Before adding anything to a stream, ask first, why am i going to need this? 2013/11/13 Frank Shearar <frank.shearar@gmail.com>
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API.
So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not.
As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names.
frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek
upTo:
...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Exactly, every specialized stream has its specialized API and/or specialized implementation. File streams don't even have a name, because they need not to. You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing. The file has a name, the stream does not need any. Before adding anything to a stream, ask first, why am i going to need this?
I agree with what Frank said, but not your suggestion that:
File streams don't even have a name, because they need not to. If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
because doing that means the FileStream is no longer fully-encapsulated. And now for the same method to support a non-FileStream stream, what will be passed as the new FileReference argument? nil? So the code went from stream-delegation to case-logic. This is why organic growth happened in the original Stream hierarchy -- because it was "needed", not random. Not strict, yes, but not "random" either. One final thing to consider about selector "uniformity" is how it can negate the ability to effectively trace code (via senders and implementors). Wow, how many senders of #next...? Stream-composition is cool, though, and I understand that to be a benefit of the stricter API.
2013/11/13 Frank Shearar <frank.shearar@gmail.com>
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API.
So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not.
As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names.
frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
Chris, I have to completely disagree. The idea of breaking a complex object into simpler parts is not just theories and extremism. Assigning several roles to an object is sometimes convenient for a start, but generally does scale very badly. It's a collective experience driven by pragmatic considerations in developping in all sort of languages. And it's my own experience too. A file is like the container, the stream is about reading/writing the contents. I think I remember the confusion was made at the beginning in st-80, because it was simple enough at that time. So it's rather an historical slag, not something that was added later for this special case. And it is something that should have been revised, because when we added pipes, sockets or other external streams, that ruined the idea of confusing the two things into a not well delimited blob without clear responsibilities. Though we kept the original implementation and started to distort the package by adding faked polymorphism like name ^'john doe'. Clearly, I need no stream on its contents to rename a file, query its permissions, or delete it, so why bother with internal states of a stream if I'm interested in handling the file? If I am interested about the contents, I don't need to ever know about the path of the file nor its name, or shall I change a $A read from it into a $B if the file is named 'turnAllAinB'? Where exactly are we going to need this encapsulated blob which pretend to be both a file and a stream? I would be happy to show that a rewrite is both simple and good looking at end application sites. Of course, inside the lava of multi year hacked stream hierarchy, it might be a bit less obvious to disentangle the spaghetti. Hence more radical solutions: table rase and complete rewrite. 2013/11/13 Chris Muller <asqueaker@gmail.com>
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Exactly, every specialized stream has its specialized API and/or specialized implementation. File streams don't even have a name, because they need not to. You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing. The file has a name, the stream does not need any. Before adding anything to a stream, ask first, why am i going to need this?
I agree with what Frank said, but not your suggestion that:
File streams don't even have a name, because they need not to. If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
because doing that means the FileStream is no longer fully-encapsulated. And now for the same method to support a non-FileStream stream, what will be passed as the new FileReference argument? nil? So the code went from stream-delegation to case-logic.
This is why organic growth happened in the original Stream hierarchy -- because it was "needed", not random. Not strict, yes, but not "random" either.
One final thing to consider about selector "uniformity" is how it can negate the ability to effectively trace code (via senders and implementors). Wow, how many senders of #next...?
Stream-composition is cool, though, and I understand that to be a benefit of the stricter API.
2013/11/13 Frank Shearar <frank.shearar@gmail.com>
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API.
So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not.
As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names.
frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or
whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error
if
accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility
selectors
B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
These are really excellent arguments, Nicolas. Good discussion. On Wed, Nov 13, 2013 at 3:32 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Chris, I have to completely disagree. The idea of breaking a complex object into simpler parts is not just theories and extremism. Assigning several roles to an object is sometimes convenient for a start, but generally does scale very badly. It's a collective experience driven by pragmatic considerations in developping in all sort of languages. And it's my own experience too.
A file is like the container, the stream is about reading/writing the contents. I think I remember the confusion was made at the beginning in st-80, because it was simple enough at that time. So it's rather an historical slag, not something that was added later for this special case. And it is something that should have been revised, because when we added pipes, sockets or other external streams, that ruined the idea of confusing the two things into a not well delimited blob without clear responsibilities. Though we kept the original implementation and started to distort the package by adding faked polymorphism like name ^'john doe'.
Clearly, I need no stream on its contents to rename a file, query its permissions, or delete it, so why bother with internal states of a stream if I'm interested in handling the file? If I am interested about the contents, I don't need to ever know about the path of the file nor its name, or shall I change a $A read from it into a $B if the file is named 'turnAllAinB'?
Where exactly are we going to need this encapsulated blob which pretend to be both a file and a stream? I would be happy to show that a rewrite is both simple and good looking at end application sites. Of course, inside the lava of multi year hacked stream hierarchy, it might be a bit less obvious to disentangle the spaghetti. Hence more radical solutions: table rase and complete rewrite.
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Exactly, every specialized stream has its specialized API and/or specialized implementation. File streams don't even have a name, because they need not to. You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing. The file has a name, the stream does not need any. Before adding anything to a stream, ask first, why am i going to need this?
I agree with what Frank said, but not your suggestion that:
File streams don't even have a name, because they need not to. If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
because doing that means the FileStream is no longer fully-encapsulated. And now for the same method to support a non-FileStream stream, what will be passed as the new FileReference argument? nil? So the code went from stream-delegation to case-logic.
This is why organic growth happened in the original Stream hierarchy -- because it was "needed", not random. Not strict, yes, but not "random" either.
One final thing to consider about selector "uniformity" is how it can negate the ability to effectively trace code (via senders and implementors). Wow, how many senders of #next...?
Stream-composition is cool, though, and I understand that to be a benefit of the stricter API.
2013/11/13 Frank Shearar <frank.shearar@gmail.com>
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with sterility borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API.
So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not.
As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names.
frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work and the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
Of course, since this is a question of style, all was invented way before the computer era :) https://en.wikipedia.org/wiki/Metonymy#Synecdoche 2013/11/13 Chris Muller <asqueaker@gmail.com>
These are really excellent arguments, Nicolas. Good discussion.
On Wed, Nov 13, 2013 at 3:32 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Chris, I have to completely disagree. The idea of breaking a complex object into simpler parts is not just theories and extremism. Assigning several roles to an object is sometimes convenient for a start, but generally does scale very badly. It's a collective experience driven by pragmatic considerations in developping in all sort of languages. And it's my own experience too.
A file is like the container, the stream is about reading/writing the contents. I think I remember the confusion was made at the beginning in st-80, because it was simple enough at that time. So it's rather an historical slag, not something that was added later for this special case. And it is something that should have been revised, because when we added pipes, sockets or other external streams, that ruined the idea of confusing the two things into a not well delimited blob without clear responsibilities. Though we kept the original implementation and started to distort the package by adding faked polymorphism like name ^'john doe'.
Clearly, I need no stream on its contents to rename a file, query its permissions, or delete it, so why bother with internal states of a stream if I'm interested in handling the file? If I am interested about the contents, I don't need to ever know about the path of the file nor its name, or shall I change a $A read from it into a $B if the file is named 'turnAllAinB'?
Where exactly are we going to need this encapsulated blob which pretend to be both a file and a stream? I would be happy to show that a rewrite is both simple and good looking at end application sites. Of course, inside the lava of multi year hacked stream hierarchy, it might be a bit less obvious to disentangle the spaghetti. Hence more radical solutions: table rase and complete rewrite.
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Wed, Nov 13, 2013 at 1:17 PM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Exactly, every specialized stream has its specialized API and/or specialized implementation. File streams don't even have a name, because they need not to. You can browse XTFileReadStream and XTFileWriteStream, you won't find such thing. The file has a name, the stream does not need any. Before adding anything to a stream, ask first, why am i going to need this?
I agree with what Frank said, but not your suggestion that:
File streams don't even have a name, because they need not to. If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
because doing that means the FileStream is no longer fully-encapsulated. And now for the same method to support a non-FileStream stream, what will be passed as the new FileReference argument? nil? So the code went from stream-delegation to case-logic.
This is why organic growth happened in the original Stream hierarchy -- because it was "needed", not random. Not strict, yes, but not "random" either.
One final thing to consider about selector "uniformity" is how it can negate the ability to effectively trace code (via senders and implementors). Wow, how many senders of #next...?
Stream-composition is cool, though, and I understand that to be a benefit of the stricter API.
2013/11/13 Frank Shearar <frank.shearar@gmail.com>
On 13 November 2013 16:48, Chris Muller <asqueaker@gmail.com> wrote:
I know nothing about Xtreams but, IMO, this obsession with
sterility
borders on mental illness.
"All sort of un-needed complexity?" That's overstating it a bit, don't you think?
So you must really feel stressed that ALL Object's, in fact, have a #name, huh? I admit this seems to push the limits but...
what's the point of having any kind of different streams at all then? It's the _differences_ between them that makes composing them useful. Compression, encryption, filtering, sockets, files, circular, etc. You think you'll be able to do all that and get away with all of them having exactly the same API?
They don't have the same API. And they shouldn't. And that's Nicolas' whole point. A Stream does not have a name (nor should it - what's the name of the compressed output of an audio stream from your microphone?). A FileStream can extend this API, and that's fine. But keep that out of the Stream API.
So Nicolas is arguing that _difference_ should be reflected in the API. File streams have names, but generic streams do not.
As it happens, Xtreams takes a very disciplined approach to this. Some streams have positions, and so you can go back. Some can't. This is reflected in the API. This is good.
IMHO working around that, passing extra objects around, sounds more stressful than letting a stream on a _file_ know its filename..
Mu. Streams have no names. File streams have names.
frank
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
How does that solution allow uniformity in stream-using code?
On Wed, Nov 13, 2013 at 9:58 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
Yes, a Wrapper would provide the legacy API.
And yes, the name of a stream should better not be part of the API. Most stream don't have a name, and adding such API adds all sort of un-needed complexity. It's an internal detail that can eventually help for reporting error if accessible, but nothing more.
If it really need it, the application certainly can retrieve the name from a higher level object (a FIleReference, FileDirectory or whatever).
2013/11/13 Chris Muller <asqueaker@gmail.com>
On Tue, Nov 12, 2013 at 7:31 AM, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote:
It's just a matter of selecting a strategy. I've proposed two: A) create a wrapper class for legacy Stream compatibility selectors B) create extensions for Legacy Stream compatibility selectors My preference goes to A)
By wrappers you mean the Xtreams are the innards doing the work
and
the wrappers providing the legacy API?
This would be a great way to test Xtreams.
The legacy support MUST be minimal (next nextPut: nextPutAll: peek upTo: ...), otherwise we will import all the cruft in Xtream and would go back to our starting point... Once the minimal support written (a few hours should be enough), we should gradually switch each every legacy Stream usage -> Xtream.
An area which require more work is those Streams that have mixed conventions (one portion is interpreted as text, another as binary). In theory that's easy, we just have two streams and they both wrap on a low level binary stream, but that means we have to be very cautious with buffers and caches.
Another area of work is usage of ugly selectors like name (we try to access the file name from the Stream API, arghh). Those usages are bad and require a rewrite.
Are you saying a FileStream knowing its #name or #filename is bad?
Hi Stef! You're thinking of Xstream as an alternative FileSytemStore or to replace an specific part of the Files package? Besides the specific issue I mentioned, how do the dev team feel about the current FileSystem-Core implementation? Cheers, Nico PM -- View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorph... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
On Nov 17, 2013, at 3:47 AM, Nicolás Papagna Maldonado <nicolas.papagna@gmail.com> wrote:
Hi Stef!
You're thinking of Xstream as an alternative FileSytemStore or to replace an specific part of the Files package?
I did not look at the code of both but from what the people are saying I think that replacing everything would be better :)
Besides the specific issue I mentioned, how do the dev team feel about the current FileSystem-Core implementation?
if you have specific feeling about it just let us know. I know that camille spent time to make it using faster primitive. Sometimes I do not like certin messages because they are not conveying what they really do. For example, I'm always confused by resolve: but for some people it is obvious (I'm too much into interpreter to not think how to resolve a variable with a context). Stef
Cheers,
Nico PM
-- View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorph... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
Thanks for the suggestions! As you said, I finally manages to get over the isReadOnly message. I've just commited the fix. Cheers, Nico PM -- View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorph... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
I have prototyped some of legacy Squeak/Pharo Stream API wrapped over Xtreams at http://www.squeaksource.com/Xtreams - see Xtreams-LegacyStreamAPI package. 2013/11/16 Nicolás Papagna Maldonado <nicolas.papagna@gmail.com>
Thanks for the suggestions!
As you said, I finally manages to get over the isReadOnly message.
I've just commited the fix.
Cheers,
Nico PM
-- View this message in context: http://forum.world.st/In-memory-FileSystem-write-streams-not-being-polymorph... Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.
participants (7)
-
Camillo Bruni -
Chris Muller -
Frank Shearar -
Nicolas Cellier -
Nicolás Papagna Maldonado -
Stéphane Ducasse -
Sven Van Caekenberghe