[Pharo-dev] Random corrupted data when copying from very large byte array

Stephane Ducasse stepharo.self at gmail.com
Thu Jan 18 12:41:09 EST 2018


Tx Cyril We know that there is some heap corruption and this is super that
you get a
reproducible scenario.

Stef


On Thu, Jan 18, 2018 at 4:51 PM, Thierry Goubier <thierry.goubier at gmail.com>
wrote:

> Hi Cyril,
>
> try with the last vms available at:
>
> https://bintray.com/opensmalltalk/vm/cog/
>
> For example, the last Ubuntu 64bits vm is at:
>
> https://bintray.com/opensmalltalk/vm/cog/201801170946#files
>
> Regards,
>
> Thierry
>
> 2018-01-18 16:42 GMT+01:00 Cyrille Delaunay <cy.delaunay at gmail.com>:
>
>> Hi everyone,
>>
>> I just added a new bug entry for an issue we are experimenting since some
>> times:
>>
>> https://pharo.fogbugz.com/f/cases/20982/Random-corrupted-dat
>> a-when-copying-from-very-large-byte-array
>>
>> Here is the description:
>>
>>
>> *History:*
>>
>> This issue has been spotted after experimenting strange behavior with
>> seaside upload.
>> After uploading a big file from a web browser, the modeled file generated
>> within pharo image begins with 4 unexpected bytes.
>> This issue occurs randomly: sometimes the first 4 bytes are right.
>> Sometimes the first 4 bytes are wrong.
>> This issue only occurs with Pharo 6.
>> This issue occurs for all platforms (Mac, Ubuntu, Windows)
>>
>> *Steps to reproduce:*
>>
>> I have been able to set up a small scenario that highlight the issue.
>>
>> Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/we
>> b/download
>> Then, iterate over this process till spotting the issue:
>>
>> => start the pharo image
>> => execute this piece of code in a playground
>>
>> 1:
>> 2:
>> 3:
>> 4:
>> 5:
>> 6:
>>
>> ZnServer startDefaultOn: 1701.ZnServer default maximumEntitySize: 80* 1024 * 1024.'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].18202065 timesRepeat: [ out nextPut: 0 ]].
>>
>> => Open a web browser page on: http://localhost:1701/form-test-3
>> => Upload the file zip file previously generated ('myzip.zip').
>> => If the web page displays: "contents=000000000a00
>> <https://pharo.kilnhg.com/Search?search=000000000a00>..." (or anything
>> unexpected), THIS IS THE ISSUE !
>> => If the web page displays: "contents=504b03040a00
>> <https://pharo.kilnhg.com/Search?search=504b03040a00>..", the upload
>> worked fine. You can close the image (without saving)
>>
>>
>>
>> *Debugging:*
>>
>>
>>
>> Bob Arning has been able to reproduce the issue with my scenario.
>> He dived into the code involved during this process, till reaching some
>> "basic" methods where he saw the issue occuring.
>>
>> Here are the conclusion till there:
>> => A corruption occurs while reading an input stream with method ZnUtils
>> class>>readUpToEnd:limit:
>> The first 4 bytes may be altered randomely.
>> => The first 4 bytes are initially correctly written to an outputStream.
>>
>> But, the first 4 bytes of this outputStream gets altered (corrupted),
>> sometimes when the inner byte array grows OR when performing the
>> final "outputStream contents"
>> => Here is a piece of code that reproduce the issue (still randomely.
>> stopping an restarting the image may change the behavior)
>>
>> 1:
>> 2:
>> 3:
>> 4:
>> 5:
>> 6:
>> 7:
>> 8:
>> 9:
>> 10:
>> 11:
>> 12:
>> 13:
>> 14:
>> 15:
>> 16:
>> 17:
>> 18:
>> 19:
>> 20:
>>
>> test4"self test4"    | species bufferSize buffer totalRead outputStream answer inputStream ba byte1 |            ba := ByteArray new: 18202085.        ba atAllPut: 99.        1 to: 20 do: [  :i | ba at: i put: (#[80 75 3 4 10 7 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ].    inputStream := ba readStream.    bufferSize := 16384.    species := ByteArray.
>>     buffer := species new: bufferSize.
>>     totalRead := 0.
>>     outputStream := nil.
>>     [ inputStream atEnd ] whileFalse: [ | readCount |
>>         readCount := inputStream readInto: buffer startingAt: 1 count: bufferSize.
>>         totalRead = 0 ifTrue: [
>>             byte1 := buffer first.
>>         ].
>>         totalRead := totalRead + readCount.
>>
>>         outputStream ifNil: [
>>             inputStream atEnd
>>                 ifTrue: [ ^ buffer copyFrom: 1 to: readCount ]
>>                 ifFalse: [ outputStream := (species new: bufferSize) writeStream ] ].
>>         outputStream next: readCount putAll: buffer startingAt: 1.
>>         byte1 = outputStream contents first ifFalse: [ self halt ].
>>     ].
>>     answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream contents ].
>>     byte1 = answer first ifFalse: [ self halt ].    ^answer
>>
>>
>>
>> *suspicions*
>>
>> This issue appeared with Pharo 6.
>>
>> Some people suggested that it could be a vm issue, and to try my little
>> scenario with the last available vm.
>>
>> I am not sure where to find the last available vm.
>>
>> I did the test using these elements:
>> https://files.pharo.org/image/60/latest.zip
>>
>> https://files.pharo.org/get-files/70/pharo-mac-latest.zip/
>> <https://files.pharo.org/get-files/70/>
>>
>>
>>
>> The issue is still present
>>
>>
>>
>> --
>> Cyrille Delaunay
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pharo.org/pipermail/pharo-dev_lists.pharo.org/attachments/20180118/bfddcf04/attachment-0002.html>


More information about the Pharo-dev mailing list