[Pharo-project] [ENH]: Zinc - Streaming Download Progress
It would be /really/ nice if Zinc sent notifications of streaming download progress to log listeners and HTTPProgress handlers. The following method showed nice progress notifications via both routes until the file started to download, which is the bulk of the time spent... downloadLatestVmMakerImage | client | client := ZnClient new. client signalProgress: true. client log addListener: self. [client url: 'https://ci.lille.inria.fr/pharo/job/Cog%20Git%20Tracker/lastSuccessfulBuild/...'; downloadTo: '/Volumes/Fast/Squeak/Cog/vmmaker-image.zip'.] on: HTTPProgress do: [ :ex | Transcript show: ex asString; cr. ex resume ] -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Hi Sean, Yeah, that would be nice. But do you agree that the only solution is to use a dynamic variable ? Because signalling code has to known whether there is someone catching the notifications, right ? The #signalProgress option in ZnClient was just the first step, the interface. The real, interesting signalling has to happen in ZnUtils class>>#streamFrom:to:size: If you agree that the dynamic variable is the way to go, I can quite easily implement it. Sven On 12 Jul 2012, at 17:58, Sean P. DeNigris wrote:
It would be /really/ nice if Zinc sent notifications of streaming download progress to log listeners and HTTPProgress handlers.
The following method showed nice progress notifications via both routes until the file started to download, which is the bulk of the time spent...
downloadLatestVmMakerImage
| client | client := ZnClient new. client signalProgress: true. client log addListener: self.
[client url: 'https://ci.lille.inria.fr/pharo/job/Cog%20Git%20Tracker/lastSuccessfulBuild/...'; downloadTo: '/Volumes/Fast/Squeak/Cog/vmmaker-image.zip'.] on: HTTPProgress do: [ :ex | Transcript show: ex asString; cr. ex resume ]
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sven Van Caekenberghe wrote
But do you agree that the only solution is to use a dynamic variable ? Because signalling code has to known whether there is someone catching the notifications, right ?
Could you say more about that? Googling "dynamic variable" turned up academic stuff. Also, could Announcements be helpful here? Sean -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639695... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 12 July 2012 17:40, Sean P. DeNigris <sean@clipperadams.com> wrote:
Sven Van Caekenberghe wrote
But do you agree that the only solution is to use a dynamic variable ? Because signalling code has to known whether there is someone catching the notifications, right ?
Could you say more about that? Googling "dynamic variable" turned up academic stuff. Also, could Announcements be helpful here?
You've used them all the time: they're just like (shell) environment variables, only the environment variables can nest. So in a default Squeak image (which has the DynamicVariables package by default), MyDynVar value: 1 during: [ Transcript showln: (MyDynVar value printString). MyDynVar value: 2 during: [ Transcript showln: (MyDynVar value printString). ]. Transcript showln: (MyDynVar value printString). ] will output 1 2 1 There is a gotcha with naively implementing dynamic variables: they _do not_ play nicely with delimited continuations: http://www.lshift.net/blog/2012/06/27/resumable-exceptions-can-macro-express... shows a solution which should Just Work in Pharo ( and if it doesn't, let me know, and I'll make it work). The link also has links explaining in detail why the two don't work nicely. This bad interaction will bite you as soon as you do anything interesting to the stack, and given that you always have thisContext just a reference away, that could be any time you like! Oh. I see I _didn't_ put a link to the code. Sigh. http://ss3.gemstone.com/ss/Control.html is the place to get the code. frank
Sean
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639695... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sven Van Caekenberghe wrote
There is already a usecase in ZnConnectionTimeout.
Okay, I checked out the code. How does that concept apply here? Maybe a quick example? Thanks, Sean -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639711... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
It is better explained with code ;-) With: Sven Van Caekenberghe uploaded a new version of Zinc-HTTP to project Zinc HTTP Components: http://www.squeaksource.com/ZincHTTPComponents/Zinc-HTTP-SvenVanCaekenberghe... ==================== Summary ==================== Name: Zinc-HTTP-SvenVanCaekenberghe.285 Author: SvenVanCaekenberghe Time: 12 July 2012, 9:58 am UUID: e992fd76-efde-4b31-b4b4-bd468f8176f2 Ancestors: Zinc-HTTP-SvenVanCaekenberghe.284 enabled HTTPProgress signalling during streaming up/downloads. introduction of ZnSignalProgress with #enabled method ================================================= You should now be able to do something like: UIManager default informUserDuring: [ :bar | [ '/Users/sven/Desktop/test-99425.txt' asFileReference ensureDeleted. ^ ZnClient new signalProgress: true; url: 'http://s3-eu-west-1.amazonaws.com/public-stfx-eu/test-99425.txt'; downloadTo: '/Users/sven/Desktop/' ] on: HTTPProgress do: [ :progress | bar label: progress printString. progress total ifNotNil: [ 1 second asDelay wait. bar current: (progress amount / progress total) * 100 ]. progress resume ] ]. Sven On 12 Jul 2012, at 19:11, Sean P. DeNigris wrote:
Sven Van Caekenberghe wrote
There is already a usecase in ZnConnectionTimeout.
Okay, I checked out the code. How does that concept apply here? Maybe a quick example?
Thanks, Sean
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639711... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 12 July 2012 21:00, Sven Van Caekenberghe <sven@beta9.be> wrote:
It is better explained with code ;-)
With:
Sven Van Caekenberghe uploaded a new version of Zinc-HTTP to project Zinc HTTP Components: http://www.squeaksource.com/ZincHTTPComponents/Zinc-HTTP-SvenVanCaekenberghe...
==================== Summary ====================
Name: Zinc-HTTP-SvenVanCaekenberghe.285 Author: SvenVanCaekenberghe Time: 12 July 2012, 9:58 am UUID: e992fd76-efde-4b31-b4b4-bd468f8176f2 Ancestors: Zinc-HTTP-SvenVanCaekenberghe.284
enabled HTTPProgress signalling during streaming up/downloads. introduction of ZnSignalProgress with #enabled method
=================================================
You should now be able to do something like:
UIManager default informUserDuring: [ :bar | [ '/Users/sven/Desktop/test-99425.txt' asFileReference ensureDeleted. ^ ZnClient new signalProgress: true; url: 'http://s3-eu-west-1.amazonaws.com/public-stfx-eu/test-99425.txt'; downloadTo: '/Users/sven/Desktop/' ] on: HTTPProgress do: [ :progress | bar label: progress printString. progress total ifNotNil: [ 1 second asDelay wait. bar current: (progress amount / progress total) * 100 ]. progress resume ] ].
Sven
Yes: my Control library just provides a touch more formalism than arbitrary resumable exceptions, but it's otherwise just that. frank
On 12 Jul 2012, at 19:11, Sean P. DeNigris wrote:
Sven Van Caekenberghe wrote
There is already a usecase in ZnConnectionTimeout.
Okay, I checked out the code. How does that concept apply here? Maybe a quick example?
Thanks, Sean
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639711... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Frank, When I have more time, I sure have to read your stuff, it seems quite interesting. Sven On 12 Jul 2012, at 22:56, Frank Shearar wrote:
On 12 July 2012 21:00, Sven Van Caekenberghe <sven@beta9.be> wrote:
It is better explained with code ;-)
With:
Sven Van Caekenberghe uploaded a new version of Zinc-HTTP to project Zinc HTTP Components: http://www.squeaksource.com/ZincHTTPComponents/Zinc-HTTP-SvenVanCaekenberghe...
==================== Summary ====================
Name: Zinc-HTTP-SvenVanCaekenberghe.285 Author: SvenVanCaekenberghe Time: 12 July 2012, 9:58 am UUID: e992fd76-efde-4b31-b4b4-bd468f8176f2 Ancestors: Zinc-HTTP-SvenVanCaekenberghe.284
enabled HTTPProgress signalling during streaming up/downloads. introduction of ZnSignalProgress with #enabled method
=================================================
You should now be able to do something like:
UIManager default informUserDuring: [ :bar | [ '/Users/sven/Desktop/test-99425.txt' asFileReference ensureDeleted. ^ ZnClient new signalProgress: true; url: 'http://s3-eu-west-1.amazonaws.com/public-stfx-eu/test-99425.txt'; downloadTo: '/Users/sven/Desktop/' ] on: HTTPProgress do: [ :progress | bar label: progress printString. progress total ifNotNil: [ 1 second asDelay wait. bar current: (progress amount / progress total) * 100 ]. progress resume ] ].
Sven
Yes: my Control library just provides a touch more formalism than arbitrary resumable exceptions, but it's otherwise just that.
frank
On 12 Jul 2012, at 19:11, Sean P. DeNigris wrote:
Sven Van Caekenberghe wrote
There is already a usecase in ZnConnectionTimeout.
Okay, I checked out the code. How does that concept apply here? Maybe a quick example?
Thanks, Sean
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639711... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sven Van Caekenberghe wrote
You should now be able to do something like...
That didn't seem to work in my Pharo 1.4 image with 284 loaded. Does it require something else? I noticed the only method in my image that seems to set the progress total for downloading is HTTPSocket>>getRestOfBuffer:totalLength:, which has no senders in my image. Thanks! Sean -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639880... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Did you load Zinc-HTTP-SvenVanCaekenberghe.285 ? On 13 Jul 2012, at 18:22, Sean P. DeNigris wrote:
Sven Van Caekenberghe wrote
You should now be able to do something like...
That didn't seem to work in my Pharo 1.4 image with 284 loaded. Does it require something else? I noticed the only method in my image that seems to set the progress total for downloading is HTTPSocket>>getRestOfBuffer:totalLength:, which has no senders in my image.
Thanks! Sean
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639880... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On Jul 13, 2012, at 12:48 PM, "Sven Van Caekenberghe [via Smalltalk]" <ml-node+s1294792n4639882h30@n4.nabble.com> wrote:
Did you load Zinc-HTTP-SvenVanCaekenberghe.285 ? Ah, that's what happened. I loaded the latest version from sqs, which seems to be 284
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639889... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
It is hard to keep repos in sync manually⦠On 13 Jul 2012, at 19:04, Sean P. DeNigris wrote:
On Jul 13, 2012, at 12:48 PM, "Sven Van Caekenberghe [via Smalltalk]" <[hidden email]> wrote:
Did you load Zinc-HTTP-SvenVanCaekenberghe.285 ? Ah, that's what happened. I loaded the latest version from sqs, which seems to be 284 View this message in context: Re: [ENH]: Zinc - Streaming Download Progress Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sven Van Caekenberghe wrote
It is hard to keep repos in sync manuallyâ¦
Oh, I forgot about http://mc.stfx.eu/ZincHTTPComponents.... Is it better to try there first, in general? _________________________ Cool! It works great :) One question - why exceptions instead of announcements? -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639907... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 13 Jul 2012, at 21:21, Sean P. DeNigris wrote:
Oh, I forgot about http://mc.stfx.eu/ZincHTTPComponents.... Is it better to try there first, in general?
Yes, that is my primary one, but I normally copy immediately to SqS, unless under hacking stress
_________________________
Cool! It works great :) One question - why exceptions instead of announcements?
Uhh, HTTPProgress was already there. I believe all progress is done that way, no ? With Announcements the problem is that you have to specify an Announcer, Notifications are just thrown in the air ;-)
Sven Van Caekenberghe wrote
Uhh, HTTPProgress was already there. I believe all progress is done that way, no ?
Oh, okay. Using exceptions for non-exceptional conditions seems like a hack. My first cut at cleaning up the SystemProgressMorph was in preparation to re-implement using announcements instead. Sven Van Caekenberghe wrote
With Announcements the problem is that you have to specify an Announcer, Notifications are just thrown in the air ;-)
How about an announcer per client that also forwards to one for the whole Zinc library? -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639930... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Sean, On 13 Jul 2012, at 22:23, Sean P. DeNigris wrote:
Sven Van Caekenberghe wrote
Uhh, HTTPProgress was already there. I believe all progress is done that way, no ?
Oh, okay. Using exceptions for non-exceptional conditions seems like a hack. My first cut at cleaning up the SystemProgressMorph was in preparation to re-implement using announcements instead.
Yes, it seems like a hack.
Sven Van Caekenberghe wrote
With Announcements the problem is that you have to specify an Announcer, Notifications are just thrown in the air ;-)
How about an announcer per client that also forwards to one for the whole Zinc library?
I think I understand what you are trying to achieve, yes it makes sense. But this feels like a architectural decision - I don't feel qualified to decide. Anybody else care to comment ? Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill
Sven Van Caekenberghe wrote
With Announcements the problem is that you have to specify an Announcer, Notifications are just thrown in the air ;-)
How about an announcer per client that also forwards to one for the whole Zinc library?
I think I understand what you are trying to achieve, yes it makes sense. But this feels like a architectural decision - I don't feel qualified to decide.
Anybody else care to comment ?
I did not get it. Stef
Stéphane Ducasse wrote
I did not get it.
Besides being hackish, using exceptions to notify progress has other drawbacks. For example, if you use announcements, then everyone who is interested in receiving them will be notified. With exceptions, even if you want to be notified, the exception may get handled before you ever get a chance. My whole journey with cleaning SystemProgress started because I was trying to get progress notifications, but they were being handled by code that I didn't own before I got them. Announcements also allow elegant simplifications like e.g. for system progress, the progress bar might be just another subscriber to the announcements, so it's really easy to enable/disable it. Also, | client | client := ZnClient new. [ client signalProgress: true; url: fileUrl; downloadTo: self workspace fullName. ] on: HTTPProgress do: [ :progress | "handle progress" progress resume ]. vs. client := ZnClient new. client onProgressSend: #progress: to: self. url: fileUrl; downloadTo: self workspace fullName. -- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4640024... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
Notification/Exception vs Announcements is pretty easy to answer in my opinion: Notification/Exception should only be used when the original action can be modified, typical examples are UI dialogs which ask for something. Announcements can be used when there is no need for feedback. I would consider the following things be better using Announcements - inform: - progress bar Furthermore I would say each Notification/Exception should trigger an Announcement as well. Somehow they are the weak form of Notification/Exception because they are only an outgoing communication channel. Whereas Notification/Exception allow information to be injected back.. On 2012-07-14, at 21:12, Sean P. DeNigris wrote:
Stéphane Ducasse wrote
I did not get it.
Besides being hackish, using exceptions to notify progress has other drawbacks. For example, if you use announcements, then everyone who is interested in receiving them will be notified. With exceptions, even if you want to be notified, the exception may get handled before you ever get a chance. My whole journey with cleaning SystemProgress started because I was trying to get progress notifications, but they were being handled by code that I didn't own before I got them. Announcements also allow elegant simplifications like e.g. for system progress, the progress bar might be just another subscriber to the announcements, so it's really easy to enable/disable it.
Also, | client | client := ZnClient new. [ client signalProgress: true; url: fileUrl; downloadTo: self workspace fullName. ] on: HTTPProgress do: [ :progress | "handle progress" progress resume ].
vs.
client := ZnClient new. client onProgressSend: #progress: to: self. url: fileUrl; downloadTo: self workspace fullName.
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4640024... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
On 13 July 2012 21:23, Sean P. DeNigris <sean@clipperadams.com> wrote:
Sven Van Caekenberghe wrote
Uhh, HTTPProgress was already there. I believe all progress is done that way, no ?
Oh, okay. Using exceptions for non-exceptional conditions seems like a hack. My first cut at cleaning up the SystemProgressMorph was in preparation to re-implement using announcements instead.
Well, using Exceptions for non-exceptional conditions is a hack. But these aren't, they're Notifications, which are entirely different! Seriously though, resumable exceptions are at the core of how Squeak/Pharo work. They're really useful. frank
Sven Van Caekenberghe wrote
With Announcements the problem is that you have to specify an Announcer, Notifications are just thrown in the air ;-)
How about an announcer per client that also forwards to one for the whole Zinc library?
-- View this message in context: http://forum.world.st/ENH-Zinc-Streaming-Download-Progress-tp4639686p4639930... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
participants (5)
-
Camillo Bruni -
Frank Shearar -
Sean P. DeNigris -
Stéphane Ducasse -
Sven Van Caekenberghe