ZnBufferedReadStream and #upToAll:
Hi all, Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes. Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository? -- Eric
Hi eric
Hi all,
Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?
Right now to sven repo. In the future we want to have better tools to sync back from a project to its components. Stef
-- Eric
Thanks Stef, I imagine that sven did not add it because on infinite stream it does not
make sense.
Just to clarify my earlier email, I got here was by calling `'/path/to/some/file' asFileReference binaryReadStream` which responds with the ZnBufferedReadStream. I expected to be able to use something like #upToAll: to find binary-formatted headers etc within some encoding structure. It's entirely possible that this was the wrong expectation! On Sun, Mar 17, 2019 at 10:38 AM ducasse <stepharo@netcourrier.com> wrote:
Hi eric
Hi all,
Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
Sub-question: if I wanted to implement something like this and make a
PR, would I be submitting to the Pharo dev repository or the Zinc repository?
Right now to sven repo. In the future we want to have better tools to sync back from a project to its components.
Stef
-- Eric
-- Eric
Eric, Someone wrote a clean implementation of #upToAll: that does not use #position: - I forgot who - but it can be found in ZnCharacterReadStream>>#upToAll: I guess this could be copied over to ZnBufferedReadStream. The main reason that I was against #upToAll: was the implementation using #position: - but what Stef says it also very true: if you do not find what you are looking for, you will read up to end, which is not possible with network streams - you will hang. I also try to minimise the stream API in the newer streams, which is hard because the existing API is so broad. Personally, I never needed #upToAll: during parsing (I also always limit lookahead to 1). I am really curious why you need it ? Most users of #upToAll: search for CRLF, for which I would use a line reader class like ZnLineReader or ZnFastLineReader. Another point is that, if you absolutely need everything that is in the classic ReadStream (which is much more than a stream as it holds a collection of all its elements), in most cases, you could read your content first (assuming it is of known size) and wrap a classic stream around it. Sven
On 17 Mar 2019, at 15:45, Eric Gade <eric.gade@gmail.com> wrote:
Thanks Stef,
I imagine that sven did not add it because on infinite stream it does not make sense.
Just to clarify my earlier email, I got here was by calling `'/path/to/some/file' asFileReference binaryReadStream` which responds with the ZnBufferedReadStream. I expected to be able to use something like #upToAll: to find binary-formatted headers etc within some encoding structure. It's entirely possible that this was the wrong expectation!
On Sun, Mar 17, 2019 at 10:38 AM ducasse <stepharo@netcourrier.com> wrote: Hi eric
Hi all,
Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes.
I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
Sub-question: if I wanted to implement something like this and make a PR, would I be submitting to the Pharo dev repository or the Zinc repository?
Right now to sven repo. In the future we want to have better tools to sync back from a project to its components.
Stef
-- Eric
-- Eric
If that was the case, then it makes no sense to implement #upTo: but not #upToAll: , right ???? On 2019-03-17 10:37, ducasse wrote:
Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes. I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
On 17 Mar 2019, at 18:28, Benoit St-Jean <bstjean@yahoo.com> wrote:
If that was the case, then it makes no sense to implement #upTo: but not #upToAll: , right ????
Indeed :)
On 2019-03-17 10:37, ducasse wrote:
Was there a conscious decision not to include the #upToAll: method onZnBufferedReadStream?. This method is really useful for parsing files -- finding sub-patterns of bytes, etc. Perhaps #upToAll: is not the idiomatic way to do this, but I found it very useful to use in the plain old ReadStream classes. I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
ducasse <stepharo@netcourrier.com> wrote:
I imagine that sven did not add it because on infinite stream it does not make sense. Now it would be good to see how we can have useful extensions. But I let this to sven.
That sounds like something that would be useful to make explicit: isInfinite, with blocking, returning nil, returning null objects, or returning errors as possible expected behaviors? Stephan
Hi Sven, I am really curious why you need it ?
This came up for me when I was trying to parse out pieces of a PDF file. As you likely know, the PDF format has both text and binary information encoded within it. I had used #upToAll: in the past to find sequences of "magic bytes" that don't occur in the middle of a file. It seems that the buffered read stream is what you get if you send #binaryReadStream to a FileReference. But you're right â I could just convert streams and do it that way (if you have other techniques you use, please send along). #upToAll: would definitely be complicated to implement in the buffered version! On Mon, Mar 18, 2019 at 5:11 AM Stephan Eggermont <stephan@stack.nl> wrote:
ducasse <stepharo@netcourrier.com> wrote:
I imagine that sven did not add it because on infinite stream it does
not make sense.
Now it would be good to see how we can have useful extensions. But I let this to sven.
That sounds like something that would be useful to make explicit: isInfinite, with blocking, returning nil, returning null objects, or returning errors as possible expected behaviors?
Stephan
-- Eric
participants (5)
-
Benoit St-Jean -
ducasse -
Eric Gade -
Stephan Eggermont -
Sven Van Caekenberghe