Downloading large files with a progress bar
Hi, I'm using myUrl asUrl saveContentsToFile: file to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used =============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. =============== but I don't know how to signal progress with "saveContentsToFile". Any hints? Thanks, Offray
Hi Offray, The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good. Alternatively, you could wrap the first expression as follows: ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ] ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested. HTH, Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
Hi Sven, Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. I get a progress bar message, but not the advancements on it. What I'm missing? Cheers, Offray ps: The file I'm trying to download is covered by the Open Data Base License. On 12/05/16 17:24, Sven Van Caekenberghe wrote:
Hi Offray,
The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good.
Alternatively, you could wrap the first expression as follows:
ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ]
ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested.
HTH,
Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
Hi again, This implementatin http://ws.stfx.eu/FSNU7TV3O5FY also shows a static downloading progress bar without any download, but the download works. Ofcourse any example with another url for a smilar large file could work for me. I'm just testing with the file for this [1] (Soon a proper announcement). [1] http://mutabit.com/offray/blog/en/entry/panama-papers-1 Cheers, Offray On 12/05/16 18:00, Offray Vladimir Luna Cárdenas wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. I get a progress bar message, but not the advancements on it. What I'm missing?
Cheers,
Offray
ps: The file I'm trying to download is covered by the Open Data Base License.
On 12/05/16 17:24, Sven Van Caekenberghe wrote:
Hi Offray,
The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good.
Alternatively, you could wrap the first expression as follows:
ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ]
ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested.
HTH,
Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
When the UI thread is really busy, updating the UI can be problematic. An extra fork seems to help: (FileLocator temp / 'PharoV30.sources') ensureDelete. [ [ :bar | bar title: 'Downloading Sources...'. [ ZnClient new url: 'http://files.pharo.org/sources/PharoV30.sources'; signalProgress: true; downloadTo: FileLocator temp ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ] ] asJob run ] fork.
On 13 May 2016, at 01:00, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. I get a progress bar message, but not the advancements on it. What I'm missing?
Cheers,
Offray
ps: The file I'm trying to download is covered by the Open Data Base License.
On 12/05/16 17:24, Sven Van Caekenberghe wrote:
Hi Offray,
The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good.
Alternatively, you could wrap the first expression as follows:
ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ]
ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested.
HTH,
Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
Hi, This is the code that finally did it: http://ws.stfx.eu/458U3WRKXAW1. I was really puzzled by the progress bar not being updated despite of the download being made (it took me several days to find what happen!). The thing is that the total size of the file was not being delivered by the datahub.io site, so there was no way to put it into the progress bar. I used an alternative approach reading the actual size against the total expected size (once you know the cause the solution took me a couple of hours, mostly learning about SystemProgressBar and JobProgressBar, the actual script took minutes.). Thanks Sven and Johan for helping me to figure this out. Cheers, Offray On 13/05/16 00:35, Sven Van Caekenberghe wrote:
When the UI thread is really busy, updating the UI can be problematic.
An extra fork seems to help:
(FileLocator temp / 'PharoV30.sources') ensureDelete.
[ [ :bar | bar title: 'Downloading Sources...'. [ ZnClient new url: 'http://files.pharo.org/sources/PharoV30.sources'; signalProgress: true; downloadTo: FileLocator temp ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ] ] asJob run ] fork.
On 13 May 2016, at 01:00, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. I get a progress bar message, but not the advancements on it. What I'm missing?
Cheers,
Offray
ps: The file I'm trying to download is covered by the Open Data Base License.
On 12/05/16 17:24, Sven Van Caekenberghe wrote:
Hi Offray,
The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good.
Alternatively, you could wrap the first expression as follows:
ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ]
ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested.
HTH,
Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
On 13 May 2016, at 01:00, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one.
That last code makes no sense and is not how the API is designed to be used.
I get a progress bar message, but not the advancements on it. What I'm missing?
Cheers,
Offray
ps: The file I'm trying to download is covered by the Open Data Base License.
On 12/05/16 17:24, Sven Van Caekenberghe wrote:
Hi Offray,
The actual implementation of #saveContentsToFile: can be found in ZnHttpSaveContentsToFile>>#performOperation where you can see how ZnClient is invoked. Just copy that with signalProgress: true added and you should be good.
Alternatively, you could wrap the first expression as follows:
ZnSignalProgress value: true during: [ myUrl asUrl saveContentsToFile: file ]
ZnSignalProgress is a DynamicVariable that is used by ZnClient to figure out if progress should be signalled. It takes precedence over the option #signalProgress. See #withProgressDo: if you are interested.
HTH,
Sven
On 13 May 2016, at 00:14, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi,
I'm using
myUrl asUrl saveContentsToFile: file
to download large files and is working fine. But I would like to add a progress bar to show the progress to the user. Previously I used
=============== [: bar | bar title: aString. [ "A classical ZnClient download with signalProgress: true;" ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ]. ] asJob run. ===============
but I don't know how to signal progress with "saveContentsToFile".
Any hints?
Thanks,
Offray
Hi, On 13/05/16 00:35, Sven Van Caekenberghe wrote:
On 13 May 2016, at 01:00, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. That last code makes no sense and is not how the API is designed to be used.
Sorry. I was tired and poking the code to see what was wrong. Your example with fork and Pharo.sources work pretty well. I will implement it. Thanks, Offray
Hi again, Umm.... this is kind of strange. When I try with the Sven code downloading the Pharo, downloading progress bar works, but when I try the follow one, download works and forking lets me use the environment, as with Sven's but progress bar don't advance: ================= OffshoreLeaksDB class >>downloadDatabase "I download the data of the panama papers from its page at the DataHub community repository: https://datahub.io/dataset/panama-papers " [[ :bar | bar title: 'Downloading Database...'. [ ZnClient new url: 'https://datahub.io/dataset/06f27df3-ec88-47ea-b428-7ec138f7835e/resource/50a...'; signalProgress: true; downloadTo: FileLocator temp ] on: HTTPProgress do: [ :progress | progress isEmpty ifFalse: [ bar current: progress percentage ]. progress resume ] ] asJob run ] fork. ================= As an alternative I'm thinking in creating a Roassal animation for background processes, similar to the rolling circles in the browser, tablet of phone, to show that something is advancing, but you still need to wait... Anyway, if someone can tell me about why this is happening I will really appreciate it. Thanks, Offray On 13/05/16 13:47, Offray Vladimir Luna Cárdenas wrote:
Hi,
On 13/05/16 00:35, Sven Van Caekenberghe wrote:
On 13 May 2016, at 01:00, Offray Vladimir Luna Cárdenas <offray.luna@mutabit.com> wrote:
Hi Sven,
Thanks for your quick answer. I tried both approaches but I'm missing something. See my last attempt at http://ws.stfx.eu/KBVBAP72IWOO with the last one. That last code makes no sense and is not how the API is designed to be used.
Sorry. I was tired and poking the code to see what was wrong. Your example with fork and Pharo.sources work pretty well. I will implement it.
Thanks,
Offray
participants (2)
-
Offray Vladimir Luna Cárdenas -
Sven Van Caekenberghe