January 2018 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [IMPORTANT] PHARO CONTRIBUTION PROCESS UPDATED! PLEASE READ!

by Esteban Lorenzano

> On 19 Jan 2018, at 10:17, Luke Gorrie <luke(a)snabb.co> wrote: > > Hi Esteban, > > I am really happy that everything is working so well for you and I hope that other people are having a similar experience. > > For me it's another story. I'm on an unsupported platform (NixOS Linux), I'm building the VM from random git commits because the source releases are all antiquated, Iceberg segfaults the moment I start it, and epica+monticello+metacello+iceberg+fogbugz+jenkins feels like a series of obstacles between me and maintaining my application. > > The way I am coping is to scale back my ambitions. I spent a lot of time making a complete packaging for NixOS but without proper source releases from upstream that was too much work so I abandoned it. I take the non-reproducible builds from Jenkins and import them as binary-blobs into the build environment that I actually want to use. I accumulate fixes as changesets in a patches/ directory instead of sending them upstream. > > To me it's a slap in the face when you tell me that it's so simple, there is only one true way to contribute to Pharo, and the first step of that procedure is to *install a binary that is not compatible with my Linux distribution.* because you are mixing âcontribute to pharoâ with âbuilding the VM in another platformâ :) the image itself (hence, the pharo process) does not has anything to do with using NixOS as target platform. In this last effort (that I salute), you are basically by your ownâ¦ nevertheless, I have to point: - there are proper source for the VM: https://github.com/OpenSmalltalk/opensmalltalk-vm <https://github.com/OpenSmalltalk/opensmalltalk-vm> (releases is another point, but you can take the Cog branch as âstableâ. This has all sources you need to compile the VM. I have no idea how packaging for NixOS works, but I guess you can adapt from there. - once VM is built, the Pharo image will runâ¦ you do not have to build an image for being able to use it. Now, if you want to do it, here: https://github.com/pharo-project/pharo <https://github.com/pharo-project/pharo> are âproperâ Pharo sources along with the way to bootstrap. We are doing that in linux and macOS every day, so is a proven script that works. Again no idea how to do it on NixOS, but again, no idea why you care. - once you have all that working, for *your own sources*: use monticello or iceberg, the one you prefer. - for contribute to pharo, follow the process pointed. cheers! Esteban ps: but all you describe as problematic has *nothing* to do with pharo process but with your building of the VM (and probably the image) in macOS. > >

Jan. 19, 2018

Re: [Pharo-dev] [IMPORTANT] PHARO CONTRIBUTION PROCESS UPDATED! PLEASE READ!

by Luke Gorrie

Hi Esteban, I am really happy that everything is working so well for you and I hope that other people are having a similar experience. For me it's another story. I'm on an unsupported platform (NixOS Linux), I'm building the VM from random git commits because the source releases are all antiquated, Iceberg segfaults the moment I start it, and epica+monticello+metacello+iceberg+fogbugz+jenkins feels like a series of obstacles between me and maintaining my application. The way I am coping is to scale back my ambitions. I spent a lot of time making a complete packaging for NixOS but without proper source releases from upstream that was too much work so I abandoned it. I take the non-reproducible builds from Jenkins and import them as binary-blobs into the build environment that I actually want to use. I accumulate fixes as changesets in a patches/ directory instead of sending them upstream. To me it's a slap in the face when you tell me that it's so simple, there is only one true way to contribute to Pharo, and the first step of that procedure is to *install a binary that is not compatible with my Linux distribution.*

Jan. 19, 2018

[Pharo 7.0-dev] Build #437: 20992-Use-Stdio-class-instead-of-VTermOutputDriver

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #437 was: SUCCESS. The Pull Request #691 was integrated: "20992-Use-Stdio-class-instead-of-VTermOutputDriver" Pull request url: https://github.com/pharo-project/pharo/pull/691 Issue Url: https://pharo.fogbugz.com/f/cases/20992 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Jan. 19, 2018

Re: [Pharo-dev] [IMPORTANT] PHARO CONTRIBUTION PROCESS UPDATED! PLEASE READ!

by Norbert Hartl

Is there a summary what and why the model changed? Iâm really interested as you know. And I can tell that there are still things very fishy in the whole process. Norbert > Am 19.01.2018 um 08:39 schrieb Esteban Lorenzano <estebanlm(a)gmail.com>: > > hi :) > >> On 18 Jan 2018, at 18:39, Stephane Ducasse <stepharo.self(a)gmail.com <mailto:stepharo.self@gmail.com>> wrote: >> >> Thanks Luke for your feedback >> Guillermo and Pablo are working on a new Iceberg model and UI. > > Model is done, we will integrate it as 0.7 version. > UIâ¦ is harder. We could do a simple UI that allows just ONE path but at the end, no one will use it, because almost no use-case follows that. Believe me, I made a research on thatâ¦ and if you do not believe me, go and try github desktop, which basically does that (and nobody uses it ;) ) > > true story: actually the UI of iceberg was pretty simple when we started. It became the mess it is now just adding what people asked to have. > but also true story: it *is* a mess, so we have to get it right before release of Pharo7 :) > >> May be >> we can ask you if you want to give them feedback >> and try the new version. >> It is important for us that people can contribute. >> >> Stef >> >> On Wed, Jan 17, 2018 at 12:20 PM, Luke Gorrie <luke(a)snabb.co <mailto:luke@snabb.co>> wrote: >>> Just one data point... >>> >>> I am completely overwhelmed by all of the different ways to manage source >>> code in the Pharo universe. I would love to contribute to upstream but this >>> seems way out of reach: I'm not even managing to keep track of the code in >>> my own home directory properly yet. > > there are just two ways: > > - iceberg > - monticello > > compare with the amount of ways you have in other languages. > you can use either way you want to keep yours sources. > > but to contribute to Pharo there is just *one* way, following this instructions: https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo <https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo> > > people has reported this version is a lot easier than previous ones (and is not much different on how to contribute to other languages) but we are still looking for ways to simplify it (I want to test github issue tracker soon, for example, to enhance communication that today is broken in two partsâ¦ and honest reality check: people finds easier to discuss on github over the PR than over fogbugz). > >>> Just now I am looking at upstream code that should be fixed: CairoLibrary >>> unix32ModuleName and unix64ModuleName are giving higher priority to guessed >>> hard-coded paths than to LD_LIBRARY_PATH and this is backwards i.e. the >>> carefully selected paths a user supplies with LD_LIBRARY_PATH should come >>> first (imho.) > > follow the contribution instructions ;) > >>> On the one hand it would be fine to dive into the upstream contribution >>> process to work out how to do this. On the other hand I am still trying to >>> master tools like Epica to recover useful code that I have written and lost >>> track of when images crash and get rebuilt etc. I really need to get this >>> under control before spending time learning Iceberg, FogBugz, etc. > > wellâ¦ not so much to say here. My only advice is to save often you image, when you are working with things that you know may crash the VM (like using FFI). > > cheers! > Esteban > >>> >>> >>> >>> >>> On 14 December 2017 at 13:19, Esteban Lorenzano <estebanlm(a)gmail.com <mailto:estebanlm@gmail.com>> wrote: >>>> >>>> Hi! >>>> >>>> Iâm working on simplifying the contribution process, after collecting >>>> opinions/experiences last couple of months. >>>> As you know, Pharo contribution process is still WIP and we aim to have it >>>> as smooth as possible for Pharo 7.0 release. Now, after observe the idea of >>>> the âsystem repositoriesâ was a bad idea because it introduced extra and non >>>> standard âpathâ to contribution, I managed to remove that to reestablish >>>> âthe regular wayâ: you will now need to add pharo repository just as any >>>> other repository you add, by cloning or adding local repository. >>>> >>>> I took Guilleâs doc and moved it to pharo project (it does not has sense >>>> to have it living in a contributorâs repository when is so important). You >>>> can find it here: >>>> >>>> https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo <https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo> >>>> >>>> This document is also updated to reveal this new process, please read it. >>>> >>>> How to update your startup scripts? >>>> Some people has added startup scripts to easy the first part of >>>> contribution. Instead enabling system repositories, etc. you now need to >>>> replace that with this: >>>> >>>> (IceRepositoryCreator new >>>> location: '/path/to/pharo-project/pharo' asFileReference; >>>> subdirectory: 'src'; >>>> createRepository) >>>> register >>>> >>>> PLEASE, PLEASE, PLEASEâ¦ take a moment to read and try the document. Is >>>> very important that document reflects new process and works reliable in >>>> different scenarios (I validated it on macOS and Windows, and assumed it >>>> worked fine on linux but you knowâ¦ bad assumptions is the base of failure ;) >>>> ) >>>> >>>> Iâm eager to hear your feedback and continue enhancing the process. >>>> >>>> (yes, Stef, I know UI is still cumbersomeâ¦ Iâm working on that :) ) >>>> >>>> cheers! >>>> Esteban

Jan. 19, 2018

Re: [Pharo-dev] [IMPORTANT] PHARO CONTRIBUTION PROCESS UPDATED! PLEASE READ!

by Esteban Lorenzano

hi :) > On 18 Jan 2018, at 18:39, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > > Thanks Luke for your feedback > Guillermo and Pablo are working on a new Iceberg model and UI. Model is done, we will integrate it as 0.7 version. UIâ¦ is harder. We could do a simple UI that allows just ONE path but at the end, no one will use it, because almost no use-case follows that. Believe me, I made a research on thatâ¦ and if you do not believe me, go and try github desktop, which basically does that (and nobody uses it ;) ) true story: actually the UI of iceberg was pretty simple when we started. It became the mess it is now just adding what people asked to have. but also true story: it *is* a mess, so we have to get it right before release of Pharo7 :) > May be > we can ask you if you want to give them feedback > and try the new version. > It is important for us that people can contribute. > > Stef > > On Wed, Jan 17, 2018 at 12:20 PM, Luke Gorrie <luke(a)snabb.co> wrote: >> Just one data point... >> >> I am completely overwhelmed by all of the different ways to manage source >> code in the Pharo universe. I would love to contribute to upstream but this >> seems way out of reach: I'm not even managing to keep track of the code in >> my own home directory properly yet. there are just two ways: - iceberg - monticello compare with the amount of ways you have in other languages. you can use either way you want to keep yours sources. but to contribute to Pharo there is just *one* way, following this instructions: https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo <https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo> people has reported this version is a lot easier than previous ones (and is not much different on how to contribute to other languages) but we are still looking for ways to simplify it (I want to test github issue tracker soon, for example, to enhance communication that today is broken in two partsâ¦ and honest reality check: people finds easier to discuss on github over the PR than over fogbugz). >> Just now I am looking at upstream code that should be fixed: CairoLibrary >> unix32ModuleName and unix64ModuleName are giving higher priority to guessed >> hard-coded paths than to LD_LIBRARY_PATH and this is backwards i.e. the >> carefully selected paths a user supplies with LD_LIBRARY_PATH should come >> first (imho.) follow the contribution instructions ;) >> On the one hand it would be fine to dive into the upstream contribution >> process to work out how to do this. On the other hand I am still trying to >> master tools like Epica to recover useful code that I have written and lost >> track of when images crash and get rebuilt etc. I really need to get this >> under control before spending time learning Iceberg, FogBugz, etc. wellâ¦ not so much to say here. My only advice is to save often you image, when you are working with things that you know may crash the VM (like using FFI). cheers! Esteban >> >> >> >> >> On 14 December 2017 at 13:19, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >>> >>> Hi! >>> >>> Iâm working on simplifying the contribution process, after collecting >>> opinions/experiences last couple of months. >>> As you know, Pharo contribution process is still WIP and we aim to have it >>> as smooth as possible for Pharo 7.0 release. Now, after observe the idea of >>> the âsystem repositoriesâ was a bad idea because it introduced extra and non >>> standard âpathâ to contribution, I managed to remove that to reestablish >>> âthe regular wayâ: you will now need to add pharo repository just as any >>> other repository you add, by cloning or adding local repository. >>> >>> I took Guilleâs doc and moved it to pharo project (it does not has sense >>> to have it living in a contributorâs repository when is so important). You >>> can find it here: >>> >>> https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo >>> >>> This document is also updated to reveal this new process, please read it. >>> >>> How to update your startup scripts? >>> Some people has added startup scripts to easy the first part of >>> contribution. Instead enabling system repositories, etc. you now need to >>> replace that with this: >>> >>> (IceRepositoryCreator new >>> location: '/path/to/pharo-project/pharo' asFileReference; >>> subdirectory: 'src'; >>> createRepository) >>> register >>> >>> PLEASE, PLEASE, PLEASEâ¦ take a moment to read and try the document. Is >>> very important that document reflects new process and works reliable in >>> different scenarios (I validated it on macOS and Windows, and assumed it >>> worked fine on linux but you knowâ¦ bad assumptions is the base of failure ;) >>> ) >>> >>> Iâm eager to hear your feedback and continue enhancing the process. >>> >>> (yes, Stef, I know UI is still cumbersomeâ¦ Iâm working on that :) ) >>> >>> cheers! >>> Esteban >> >> >

Jan. 19, 2018

Re: [Pharo-dev] [Ann] Calypso scoped refactoring

by Stephane Ducasse

Tx denis!!! Stef On Mon, Jan 15, 2018 at 12:10 PM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > I finally implemented scopes support for refactorings in Calypso to be > able restrict system changes to particular set of packages or classes. > The new preview tool is copy of old ChangesBrowser with additional scopes > list widget: > > > > In this picture I requested Announcer>>initialize rename and switched > scope to the Announcer package. > > Scopes are filled by browser where refactoring was requested. It is same > logic how method browser receives scopes from original browser. > > When user changes scope all refactoring changes are recomputed. > > It was last dependency of Calypso from Nautilus related packages. Now it > can be loaded without Nautilus. > > Best regards, > Denis >

Jan. 18, 2018

Re: [Pharo-dev] Random corrupted data when copying from very large byte array

by ClÃ©ment Bera

I would suspect a bug in primitive 105 on byte objects (it was changed recently in the VM), called by copyFrom: 1 to: readCount. The bug would likely by due to specific alignment in readCount or something like that. (Assuming you're in 32 bits since the 4 bytes are corrupted). When I get better I can have a look (I am currently quite sick). On Thu, Jan 18, 2018 at 4:51 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > Hi Cyril, > > try with the last vms available at: > > https://bintray.com/opensmalltalk/vm/cog/ > > For example, the last Ubuntu 64bits vm is at: > > https://bintray.com/opensmalltalk/vm/cog/201801170946#files > > Regards, > > Thierry > > 2018-01-18 16:42 GMT+01:00 Cyrille Delaunay <cy.delaunay(a)gmail.com>: > >> Hi everyone, >> >> I just added a new bug entry for an issue we are experimenting since some >> times: >> >> https://pharo.fogbugz.com/f/cases/20982/Random-corrupted-dat >> a-when-copying-from-very-large-byte-array >> >> Here is the description: >> >> >> *History:* >> >> This issue has been spotted after experimenting strange behavior with >> seaside upload. >> After uploading a big file from a web browser, the modeled file generated >> within pharo image begins with 4 unexpected bytes. >> This issue occurs randomly: sometimes the first 4 bytes are right. >> Sometimes the first 4 bytes are wrong. >> This issue only occurs with Pharo 6. >> This issue occurs for all platforms (Mac, Ubuntu, Windows) >> >> *Steps to reproduce:* >> >> I have been able to set up a small scenario that highlight the issue. >> >> Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/we >> b/download >> Then, iterate over this process till spotting the issue: >> >> => start the pharo image >> => execute this piece of code in a playground >> >> 1: >> 2: >> 3: >> 4: >> 5: >> 6: >> >> ZnServer startDefaultOn: 1701.ZnServer default maximumEntitySize: 80* 1024 * 1024.'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].18202065 timesRepeat: [ out nextPut: 0 ]]. >> >> => Open a web browser page on: http://localhost:1701/form-test-3 >> => Upload the file zip file previously generated ('myzip.zip'). >> => If the web page displays: "contents=000000000a00 >> <https://pharo.kilnhg.com/Search?search=000000000a00>..." (or anything >> unexpected), THIS IS THE ISSUE ! >> => If the web page displays: "contents=504b03040a00 >> <https://pharo.kilnhg.com/Search?search=504b03040a00>..", the upload >> worked fine. You can close the image (without saving) >> >> >> >> *Debugging:* >> >> >> >> Bob Arning has been able to reproduce the issue with my scenario. >> He dived into the code involved during this process, till reaching some >> "basic" methods where he saw the issue occuring. >> >> Here are the conclusion till there: >> => A corruption occurs while reading an input stream with method ZnUtils >> class>>readUpToEnd:limit: >> The first 4 bytes may be altered randomely. >> => The first 4 bytes are initially correctly written to an outputStream. >> >> But, the first 4 bytes of this outputStream gets altered (corrupted), >> sometimes when the inner byte array grows OR when performing the >> final "outputStream contents" >> => Here is a piece of code that reproduce the issue (still randomely. >> stopping an restarting the image may change the behavior) >> >> 1: >> 2: >> 3: >> 4: >> 5: >> 6: >> 7: >> 8: >> 9: >> 10: >> 11: >> 12: >> 13: >> 14: >> 15: >> 16: >> 17: >> 18: >> 19: >> 20: >> >> test4"self test4" | species bufferSize buffer totalRead outputStream answer inputStream ba byte1 | ba := ByteArray new: 18202085. ba atAllPut: 99. 1 to: 20 do: [ :i | ba at: i put: (#[80 75 3 4 10 7 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ]. inputStream := ba readStream. bufferSize := 16384. species := ByteArray. >> buffer := species new: bufferSize. >> totalRead := 0. >> outputStream := nil. >> [ inputStream atEnd ] whileFalse: [ | readCount | >> readCount := inputStream readInto: buffer startingAt: 1 count: bufferSize. >> totalRead = 0 ifTrue: [ >> byte1 := buffer first. >> ]. >> totalRead := totalRead + readCount. >> >> outputStream ifNil: [ >> inputStream atEnd >> ifTrue: [ ^ buffer copyFrom: 1 to: readCount ] >> ifFalse: [ outputStream := (species new: bufferSize) writeStream ] ]. >> outputStream next: readCount putAll: buffer startingAt: 1. >> byte1 = outputStream contents first ifFalse: [ self halt ]. >> ]. >> answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream contents ]. >> byte1 = answer first ifFalse: [ self halt ]. ^answer >> >> >> >> *suspicions* >> >> This issue appeared with Pharo 6. >> >> Some people suggested that it could be a vm issue, and to try my little >> scenario with the last available vm. >> >> I am not sure where to find the last available vm. >> >> I did the test using these elements: >> https://files.pharo.org/image/60/latest.zip >> >> https://files.pharo.org/get-files/70/pharo-mac-latest.zip/ >> <https://files.pharo.org/get-files/70/> >> >> >> >> The issue is still present >> >> >> >> -- >> Cyrille Delaunay >> > > -- ClÃ©ment BÃ©ra Pharo consortium engineer https://clementbera.wordpress.com/ BÃ¢timent B 40, avenue Halley 59650 Villeneuve d'Ascq

Jan. 18, 2018

Re: [Pharo-dev] Random corrupted data when copying from very large byte array

by Alistair Grant

As a separate data point: - I've seen corruption twice out of several runs (the process hits "self halt") in a 32 bit VM. -- In both cases the first four bytes of the array were 0. - I haven't yet seen a crash or corruption in the 64 bit VM (it's in the process of looping 1000 times). OS: Ubuntu 16.04. Images: - Pharo-7.0+alpha.build.425.sha.eb0a6fb140ac4a42b1f158ed37717e0650f778b4 (32 Bit) - Pharo-7.0+alpha.build.436.sha.7e0f6d30dca546f3859b32764483b42f4fdcd63c (64 Bit) VM: 5.0-201801110739 Thursday 11 January 09:22:24 CET 2018 gcc 4.8.5 [Production Spur 64-bit VM] CoInterpreter VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 StackToRegisterMappingCogit VMMaker.oscog-eem.2302 uuid: 55ec8f63-cdbe-4e79-8f22-48fdea585b88 Jan 11 2018 VM: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Date: Wed Jan 10 23:39:30 2018 -0800 $ Plugins: 201801110739 alistair@alistair-xps13:snap/pharo-snap/pharo-vm/opensmalltalk-vm $ Linux b07d7880072c 4.13.0-26-generic #29~16.04.2-Ubuntu SMP Tue Jan 9 22:00:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux plugin path: /snap/pharo7/x1/usr/bin/pharo-vm/5.0-201801110739 [default: /snap/pharo7/x1/usr/bin/pharo-vm/5.0-201801110739/] The 32 bit VM is the same version as above. Cheers, Alistair On 18 January 2018 at 16:42, Cyrille Delaunay <cy.delaunay(a)gmail.com> wrote: > Hi everyone, > > I just added a new bug entry for an issue we are experimenting since some > times: > > https://pharo.fogbugz.com/f/cases/20982/Random-corrupted-data-when-copying-… > > Here is the description: > > > History: > > This issue has been spotted after experimenting strange behavior with > seaside upload. > After uploading a big file from a web browser, the modeled file generated > within pharo image begins with 4 unexpected bytes. > This issue occurs randomly: sometimes the first 4 bytes are right. Sometimes > the first 4 bytes are wrong. > This issue only occurs with Pharo 6. > This issue occurs for all platforms (Mac, Ubuntu, Windows) > > Steps to reproduce: > > I have been able to set up a small scenario that highlight the issue. > > Download Pharo 6.1 on my Mac (Sierra 10.12.6): > https://pharo.org/web/download > Then, iterate over this process till spotting the issue: > > => start the pharo image > => execute this piece of code in a playground > > 1: > 2: > 3: > 4: > 5: > 6: > > ZnServer startDefaultOn: 1701. > ZnServer default maximumEntitySize: 80* 1024 * 1024. > '/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out | > out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0]. > 18202065 timesRepeat: [ out nextPut: 0 ] > ]. > > => Open a web browser page on: http://localhost:1701/form-test-3 > => Upload the file zip file previously generated ('myzip.zip'). > => If the web page displays: "contents=000000000a00..." (or anything > unexpected), THIS IS THE ISSUE ! > => If the web page displays: "contents=504b03040a00..", the upload worked > fine. You can close the image (without saving) > > > > Debugging: > > > > Bob Arning has been able to reproduce the issue with my scenario. > He dived into the code involved during this process, till reaching some > "basic" methods where he saw the issue occuring. > > Here are the conclusion till there: > => A corruption occurs while reading an input stream with method ZnUtils > class>>readUpToEnd:limit: > The first 4 bytes may be altered randomely. > => The first 4 bytes are initially correctly written to an outputStream. > But, the first 4 bytes of this outputStream gets altered (corrupted), > sometimes when the inner byte array grows OR when performing the final > "outputStream contents" > => Here is a piece of code that reproduce the issue (still randomely. > stopping an restarting the image may change the behavior) > > 1: > 2: > 3: > 4: > 5: > 6: > 7: > 8: > 9: > 10: > 11: > 12: > 13: > 14: > 15: > 16: > 17: > 18: > 19: > 20: > > test4"self test4" | species bufferSize buffer totalRead outputStream > answer inputStream ba byte1 | ba := ByteArray new: 18202085. > ba atAllPut: 99. 1 to: 20 do: [ :i | ba at: i put: (#[80 75 3 4 10 7 > 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ]. inputStream := ba readStream. > bufferSize := 16384. species := ByteArray. > buffer := species new: bufferSize. > totalRead := 0. > outputStream := nil. > [ inputStream atEnd ] whileFalse: [ | readCount | > readCount := inputStream readInto: buffer startingAt: 1 count: > bufferSize. > totalRead = 0 ifTrue: [ > byte1 := buffer first. > ]. > totalRead := totalRead + readCount. > > outputStream ifNil: [ > inputStream atEnd > ifTrue: [ ^ buffer copyFrom: 1 to: readCount ] > ifFalse: [ outputStream := (species new: bufferSize) > writeStream ] ]. > outputStream next: readCount putAll: buffer startingAt: 1. > byte1 = outputStream contents first ifFalse: [ self halt ]. > ]. > answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream > contents ]. > byte1 = answer first ifFalse: [ self halt ]. ^answer > > > > suspicions > > This issue appeared with Pharo 6. > > Some people suggested that it could be a vm issue, and to try my little > scenario with the last available vm. > > I am not sure where to find the last available vm. > > I did the test using these elements: > > https://files.pharo.org/image/60/latest.zip > > https://files.pharo.org/get-files/70/pharo-mac-latest.zip/ > > > > The issue is still present > > > > > -- > Cyrille Delaunay

Jan. 18, 2018

Re: [Pharo-dev] Random corrupted data when copying from very large byte array

by Stephane Ducasse

Tx Cyril We know that there is some heap corruption and this is super that you get a reproducible scenario. Stef On Thu, Jan 18, 2018 at 4:51 PM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > Hi Cyril, > > try with the last vms available at: > > https://bintray.com/opensmalltalk/vm/cog/ > > For example, the last Ubuntu 64bits vm is at: > > https://bintray.com/opensmalltalk/vm/cog/201801170946#files > > Regards, > > Thierry > > 2018-01-18 16:42 GMT+01:00 Cyrille Delaunay <cy.delaunay(a)gmail.com>: > >> Hi everyone, >> >> I just added a new bug entry for an issue we are experimenting since some >> times: >> >> https://pharo.fogbugz.com/f/cases/20982/Random-corrupted-dat >> a-when-copying-from-very-large-byte-array >> >> Here is the description: >> >> >> *History:* >> >> This issue has been spotted after experimenting strange behavior with >> seaside upload. >> After uploading a big file from a web browser, the modeled file generated >> within pharo image begins with 4 unexpected bytes. >> This issue occurs randomly: sometimes the first 4 bytes are right. >> Sometimes the first 4 bytes are wrong. >> This issue only occurs with Pharo 6. >> This issue occurs for all platforms (Mac, Ubuntu, Windows) >> >> *Steps to reproduce:* >> >> I have been able to set up a small scenario that highlight the issue. >> >> Download Pharo 6.1 on my Mac (Sierra 10.12.6): https://pharo.org/we >> b/download >> Then, iterate over this process till spotting the issue: >> >> => start the pharo image >> => execute this piece of code in a playground >> >> 1: >> 2: >> 3: >> 4: >> 5: >> 6: >> >> ZnServer startDefaultOn: 1701.ZnServer default maximumEntitySize: 80* 1024 * 1024.'/Users/cdelaunay/myzip.zip' asFileReference writeStreamDo: [ :out |out binary; nextPutAll: #[80 75 3 4 10 0 0 0 0 0 125 83 67 73 0 0 0 0 0 0].18202065 timesRepeat: [ out nextPut: 0 ]]. >> >> => Open a web browser page on: http://localhost:1701/form-test-3 >> => Upload the file zip file previously generated ('myzip.zip'). >> => If the web page displays: "contents=000000000a00 >> <https://pharo.kilnhg.com/Search?search=000000000a00>..." (or anything >> unexpected), THIS IS THE ISSUE ! >> => If the web page displays: "contents=504b03040a00 >> <https://pharo.kilnhg.com/Search?search=504b03040a00>..", the upload >> worked fine. You can close the image (without saving) >> >> >> >> *Debugging:* >> >> >> >> Bob Arning has been able to reproduce the issue with my scenario. >> He dived into the code involved during this process, till reaching some >> "basic" methods where he saw the issue occuring. >> >> Here are the conclusion till there: >> => A corruption occurs while reading an input stream with method ZnUtils >> class>>readUpToEnd:limit: >> The first 4 bytes may be altered randomely. >> => The first 4 bytes are initially correctly written to an outputStream. >> >> But, the first 4 bytes of this outputStream gets altered (corrupted), >> sometimes when the inner byte array grows OR when performing the >> final "outputStream contents" >> => Here is a piece of code that reproduce the issue (still randomely. >> stopping an restarting the image may change the behavior) >> >> 1: >> 2: >> 3: >> 4: >> 5: >> 6: >> 7: >> 8: >> 9: >> 10: >> 11: >> 12: >> 13: >> 14: >> 15: >> 16: >> 17: >> 18: >> 19: >> 20: >> >> test4"self test4" | species bufferSize buffer totalRead outputStream answer inputStream ba byte1 | ba := ByteArray new: 18202085. ba atAllPut: 99. 1 to: 20 do: [ :i | ba at: i put: (#[80 75 3 4 10 7 7 7 7 7 125 83 67 73 7 7 7 7 7 7] at: i) ]. inputStream := ba readStream. bufferSize := 16384. species := ByteArray. >> buffer := species new: bufferSize. >> totalRead := 0. >> outputStream := nil. >> [ inputStream atEnd ] whileFalse: [ | readCount | >> readCount := inputStream readInto: buffer startingAt: 1 count: bufferSize. >> totalRead = 0 ifTrue: [ >> byte1 := buffer first. >> ]. >> totalRead := totalRead + readCount. >> >> outputStream ifNil: [ >> inputStream atEnd >> ifTrue: [ ^ buffer copyFrom: 1 to: readCount ] >> ifFalse: [ outputStream := (species new: bufferSize) writeStream ] ]. >> outputStream next: readCount putAll: buffer startingAt: 1. >> byte1 = outputStream contents first ifFalse: [ self halt ]. >> ]. >> answer := outputStream ifNil: [ species new ] ifNotNil: [ outputStream contents ]. >> byte1 = answer first ifFalse: [ self halt ]. ^answer >> >> >> >> *suspicions* >> >> This issue appeared with Pharo 6. >> >> Some people suggested that it could be a vm issue, and to try my little >> scenario with the last available vm. >> >> I am not sure where to find the last available vm. >> >> I did the test using these elements: >> https://files.pharo.org/image/60/latest.zip >> >> https://files.pharo.org/get-files/70/pharo-mac-latest.zip/ >> <https://files.pharo.org/get-files/70/> >> >> >> >> The issue is still present >> >> >> >> -- >> Cyrille Delaunay >> > >

Jan. 18, 2018

Re: [Pharo-dev] [IMPORTANT] PHARO CONTRIBUTION PROCESS UPDATED! PLEASE READ!

by Stephane Ducasse

Thanks Luke for your feedback Guillermo and Pablo are working on a new Iceberg model and UI. May be we can ask you if you want to give them feedback and try the new version. It is important for us that people can contribute. Stef On Wed, Jan 17, 2018 at 12:20 PM, Luke Gorrie <luke(a)snabb.co> wrote: > Just one data point... > > I am completely overwhelmed by all of the different ways to manage source > code in the Pharo universe. I would love to contribute to upstream but this > seems way out of reach: I'm not even managing to keep track of the code in > my own home directory properly yet. > > Just now I am looking at upstream code that should be fixed: CairoLibrary > unix32ModuleName and unix64ModuleName are giving higher priority to guessed > hard-coded paths than to LD_LIBRARY_PATH and this is backwards i.e. the > carefully selected paths a user supplies with LD_LIBRARY_PATH should come > first (imho.) > > On the one hand it would be fine to dive into the upstream contribution > process to work out how to do this. On the other hand I am still trying to > master tools like Epica to recover useful code that I have written and lost > track of when images crash and get rebuilt etc. I really need to get this > under control before spending time learning Iceberg, FogBugz, etc. > > > > > On 14 December 2017 at 13:19, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >> >> Hi! >> >> Iâm working on simplifying the contribution process, after collecting >> opinions/experiences last couple of months. >> As you know, Pharo contribution process is still WIP and we aim to have it >> as smooth as possible for Pharo 7.0 release. Now, after observe the idea of >> the âsystem repositoriesâ was a bad idea because it introduced extra and non >> standard âpathâ to contribution, I managed to remove that to reestablish >> âthe regular wayâ: you will now need to add pharo repository just as any >> other repository you add, by cloning or adding local repository. >> >> I took Guilleâs doc and moved it to pharo project (it does not has sense >> to have it living in a contributorâs repository when is so important). You >> can find it here: >> >> https://github.com/pharo-project/pharo/wiki/Contribute-a-fix-to-Pharo >> >> This document is also updated to reveal this new process, please read it. >> >> How to update your startup scripts? >> Some people has added startup scripts to easy the first part of >> contribution. Instead enabling system repositories, etc. you now need to >> replace that with this: >> >> (IceRepositoryCreator new >> location: '/path/to/pharo-project/pharo' asFileReference; >> subdirectory: 'src'; >> createRepository) >> register >> >> PLEASE, PLEASE, PLEASEâ¦ take a moment to read and try the document. Is >> very important that document reflects new process and works reliable in >> different scenarios (I validated it on macOS and Windows, and assumed it >> worked fine on linux but you knowâ¦ bad assumptions is the base of failure ;) >> ) >> >> Iâm eager to hear your feedback and continue enhancing the process. >> >> (yes, Stef, I know UI is still cumbersomeâ¦ Iâm working on that :) ) >> >> cheers! >> Esteban > >

Jan. 18, 2018