Re: [Pharo-dev] ZnClient send big data + progress

June 6, 2014

      Hi Roberto,

Let's see. First we start an HTTP server in Pharo 3:

(ZnServer defaultOn: 1701)
  onRequestRespond: [ :request | 
    request inspect. 
    ZnResponse ok: (ZnEntity text: 'OK') ];
  maximumEntitySize: 100*1024*1024;
  logToTranscript ;
  start.

The handler code will always reply 'OK', but also inspect the incoming request. I raise the maximum entity size to 100Mb.

Here is a simple request:

ZnClient new
  url: 'http://localhost:1701';
  entity: (ZnEntity with: (0 to: 255) asByteArray);
  post.

Next is a bigger one (10Mb):

ZnClient new
  url: 'http://localhost:1701';
  entity: (ZnEntity with: (ByteArray new: 10*1024*1023 withAll: 123));
  post.

Finally the big one (50Mb):

ZnClient new
  url: 'http://localhost:1701';
  timeout: 900;
  entity: (ZnEntity with: (ByteArray new: 50*1024*1023 withAll: 123));
  post.

That fails indeed, and I don't know why (yet). The weird thing is, it fails with only 6Mb written, always at the same position, while the 10Mb upload did work OK. Raising the timeout does indeed not make a difference. Data is written using 16Kb chunks, with progress notifications (see ZnUtils class>>#nextPutAll:on:). 

Luckily, there is a workaround, using the classic SocketStream somehow fixes the issue:

  ZnNetworkingUtils default socketStreamClass: SocketStream.

Now, for the progress indication, Zinc certainly does it correctly, but somehow the UI does not always follow. You can verify this by writing to the Transcript:

[ :bar |
  bar title: 'Uploading...'.
  [
    ZnClient new 
      signalProgress: true;	
      url: 'http://localhost:1701';
      entity: (ZnEntity with: (ByteArray new: 10*1024*1023 withAll: 123));
      post ]
    on: HTTPProgress 
    do: [ :progress |
      progress isEmpty ifFalse: [ bar current: progress percentage ]. 
      Transcript crShow: progress.
      progress resume ] ] asJob run.

Experiment with adding/commenting out the #crShow: you will see that it works, sometimes.

HTH,

Sven

On 06 Jun 2014, at 10:47, roberto.minelli@usi.ch wrote:
...
Hi guys,
I have a problem with Zinc, I need to send a huge amount of data (bytes) that translates to ~50Mb.
When I do something like:
ZnEasy 
  post: 'http://localhost:15432' 
  data: (ZnEntity bytes: aHugeByteArray.]
I encounter a âConnectionTimedOut: Data send timed out.â, I attach the stack trace:
ZdcSocketStream(ZdcSimpleSocketStream)>>flushBytes:startingAt:count:
ZdcSocketStream>>next:putAll:startingAt:
ZnBivalentWriteStream>>next:putAll:startingAt:
ZnUtils class>>nextPutAll:on:
ZnByteArrayEntity>>writeOn:
ZnEntityWriter>>writeEntity:
ZnRequest(ZnMessage)>>writeOn:
ZnRequest>>writeOn:
ZnClient>>executeRequestResponse
ZnClient>>getConnectionAndExecute in Block: [ self executeRequestResponse ]
BlockClosure>>ensure:
ZnClient>>getConnectionAndExecute
ZnClient>>executeWithRedirectsRemaining:
ZnClient>>executeWithRetriesRemaining: in Block: [ self executeWithRedirectsRemaining: self maxNumb...etc...
BlockClosure>>on:do:
ZnClient>>executeWithRetriesRemaining:
ZnClient>>executeWithTimeout in Block: [ self executeWithRetriesRemaining: self numberOfR...etc...
BlockClosure>>on:do:
ZnClient>>executeWithTimeout in Block: [ ...
ZnClient>>withTimeoutDo: in Block: [ ^ block value ]
ZnConnectionTimeout(DynamicVariable)>>value:during: in Block: [ ...
BlockClosure>>ensure:
ZnConnectionTimeout(DynamicVariable)>>value:during:
ZnConnectionTimeout class(DynamicVariable class)>>value:during:
ZnClient>>withTimeoutDo:
ZnClient>>executeWithTimeout
ZnClient>>execute in Block: [ result := self executeWithTimeout ]
ZnClient>>withProgressDo: in Block: [ ^ block value ]
ZnSignalProgress(DynamicVariable)>>value:during: in Block: [ ...
BlockClosure>>ensure:
I tried to:
- Augment the client timeout to a huge value (10 mins) but it has no effect: The data send still goes into timeout, after 10-15 sec
- I replaced the ZnEntity with a ZnMultiPartFormDataEntity, still no luck
I am wondering what else can I do.
Moreover, I discovered there is a #signalProgress: message to signal process. The following examples do not work, any hint?
################################################################################
Example 1, from the book @ http://zn.stfx.eu/zn/zinc-http-components-paper.html:
UIManager default informUserDuring: [ :bar |
 [ ^ ZnClient new
       signalProgress: true;
       get: 'http://zn.stfx.eu/echo?delay=2' ]
   on: HTTPProgress 
   do: [ :progress | bar value: progress printString. progress resume ] ]
################################################################################
Example 2, from ZnClientTests>>#testProgress works, but not when issuing a POST request
[ :bar |
      bar title: 'Downloading Sources...'.
      [
      	ZnClient new 
      		url: 'http://files.pharo.org/sources/PharoV30.sources'; 
      		signalProgress: true;	
      		downloadTo: FileLocator temp ]
      	on: HTTPProgress 
      	do: [ :progress |
      		progress isEmpty ifFalse: [ bar current: progress percentage ]. 
      		progress resume ] ] asJob run.
â¦works but only when using #downloadTo: if you do a #post:contents: or a #get: as in the example with #get: 'http://zn.stfx.eu/echo?delay=10' no process is shown. Any hint?
Thanks in advance,
Roberto

Re: [Pharo-dev] ZnClient send big data + progress

Sven Van Caekenberghe