Pharo-dev

By thread

pharo-dev@lists.pharo.org

2 participants
144614 messages

Re: [Pharo-dev] Problem with zinc 2.9.2

by Sven Van Caekenberghe

Good! > On 22 Jul 2018, at 13:53, Norbert Hartl <norbert(a)hartl.name> wrote: > > Sven, > > thank you very much for quickly providing a new version. I changed the dependencies of my project for 2.9.4 and everything is fine now. So I can start using pharo7 on my laptop and keep 6.1 images for building the deployment artefact. Great! > > thanks, > > Norberr > >> Am 22.07.2018 um 12:59 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >> >> Done: >> >> === >> Name: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.117 >> Author: SvenVanCaekenberghe >> Time: 22 July 2018, 12:58:12.362 pm >> UUID: 949d5a24-f62d-0d00-a12c-ead9078b1897 >> Ancestors: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.116 >> >> stable v 2.9.4 >> === >> >> Please tell us if this works for you >> >>> On 22 Jul 2018, at 12:10, Norbert Hartl <norbert(a)hartl.name> wrote: >>> >>> Thank you guys! It is really great to see that there are people like Max jumping onto problems like this and solving it. >>> >>> @sven Iâm eager to test when the new configuration is available. Please drop us a note when you are done! >>> >>> thanks again, >>> >>> Norbert >>> >>>> Am 22.07.2018 um 11:06 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >>>> >>>> Ah, that makes total sense. Great catch. Thank you. >>>> >>>> === >>>> Name: Zinc-FileSystem-SvenVanCaekenberghe.17 >>>> Author: SvenVanCaekenberghe >>>> Time: 22 July 2018, 11:04:40.148339 am >>>> UUID: e97a508e-f42d-0d00-a127-f016078b1897 >>>> Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.16 >>>> >>>> Bugfix to ZnFileSystemUtils class>>#newBinaryFileNamed:do: we must ensure that the binaryStream opened here is closed as well (thx Max Leske for the bug hunting and fix) >>>> === >>>> >>>> The method in question now look as follows: >>>> >>>> newBinaryFileNamed: fileName do: block >>>> | fileReference | >>>> fileReference := fileName asFileReference. >>>> ^ (fileReference respondsTo: #binaryWriteStreamDo:ifPresent:) >>>> ifTrue: [ >>>> fileName asFileReference >>>> binaryWriteStreamDo: block >>>> ifPresent: [ FileExists signalWith: fileReference ] ] >>>> ifFalse: [ >>>> fileReference isFile >>>> ifTrue: [ FileExists signalWith: fileReference ] >>>> ifFalse: [ | binaryStream | >>>> binaryStream := self binaryFileStreamFor: fileName. >>>> [ block value: binaryStream ] ensure: [ binaryStream close ] ] ] >>>> >>>> I create a new 2.9.4 of the config later on. >>>> >>>>> On 21 Jul 2018, at 22:00, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>> Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: >>>>> >>>>> block value: (self binaryFileStreamFor: fileName) >>>>> >>>>> This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: >>>>> >>>>> | s | >>>>> s := self binaryFileStreamFor: fileName. >>>>> [ block value: s ] ensure: [ s close ] >>>>> >>>>> ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On 21 Jul 2018, at 17:21, Max Leske wrote: >>>>> >>>>> Just to be safe, I'll redo my experiment and get back to you. >>>>> >>>>> On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: >>>>> I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), >>>>> >>>>> ZnClient new >>>>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>>> downloadTo: '/tmp/foobar.zip'. >>>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>>>> Transcript open; show: bytes size; cr. >>>>> 5 seconds asDelay wait. >>>>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>>>> >>>>> And it gives me, >>>>> >>>>> 318420 >>>>> 318420 >>>>> >>>>> For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. >>>>> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >>>>> Max, >>>>> >>>>> what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. >>>>> >>>>> In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. >>>>> I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. >>>>> >>>>> Cheers, >>>>> Max >>>>> Norbert >>>>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>>>> >>>>> Hi Nicolas, >>>>> >>>>> The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. >>>>> >>>>> Thanks for the pointer though. >>>>> >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> >>>>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>>>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>>>> >>>>> Isn't there a recent PR on opensmalltalk that address just this? >>>>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : >>>>> >>>>> Hi Norbert, >>>>> >>>>> I was able to reproduce the problem and then identify the culprit, >>>>> although what I don't yet understand is how this is related to the changes >>>>> in Zinc. >>>>> >>>>> Anyway, the problem is that the file stream isn't properly flushed in >>>>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last >>>>> statement fixes your problem. Apparently, StandardFileStream / >>>>> MultiByteFileStream perform a simple close() on the file which, according >>>>> to the man page on close(), does *not* guarantee that the contents have >>>>> all been written to file. In this case a flush is necessary because the >>>>> entire file is immediately read again. >>>>> >>>>> Here's a smaller test case for you to play with Sven: >>>>> >>>>> ZnClient new >>>>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>>> downloadTo: '/tmp/foobar.zip'. >>>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>>>> Transcript open; show: bytes size; cr. >>>>> 5 seconds asDelay wait. >>>>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>>>> >>>>> The output in the Transcript should be: >>>>> >>>>> 315392 >>>>> 318420 >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>>>> >>>>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> I constructed a case that fails exactly like I experience it. Could you >>>>> try? Just unpack the attachment on a linux, set PHARO_VM env to your >>>>> executable and execute build.sh >>>>> >>>>> I will. Might take a couple of days though. >>>>> >>>>> No problem. Iâm happy if you find time. >>>>> >>>>> Norbert >>>>> >>>>> Max >>>>> >>>>> Norbert >>>>> >>>>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>>>> >>>>> Max, >>>>> >>>>> thanks for taking the effort. >>>>> >>>>> No worries. >>>>> >>>>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> I did my best to reproduce the issue but didn't have any luck. I really >>>>> need access to a Linux VM to debug this. So I'm praying that Apple fixes >>>>> the access restrictions to kernel extensions in their next beta... >>>>> >>>>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is >>>>> indeed a chance that something goes wrong during download of the zip >>>>> archive and that something may be tied to a difference in the versions of >>>>> Zinc (although I still think that's unlikely). >>>>> >>>>> Yes there is potential but I donât see it. I take a fresh 6.1 image and >>>>> load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather >>>>> early in that process. So I wonder why it does not go wrong in the first >>>>> phase. And also not if I load the test group within the first phase. >>>>> It must be either the second Metacello invocation or the stopping, copying >>>>> and starting of the image. >>>>> I try to isolate this case more and provide a script that goes wrong on my >>>>> machine. But it will take some time because I needed to stop trying to >>>>> solve this as I wasted nearly two days on that already. >>>>> >>>>> Let me know once you have something and I'll try to help out. >>>>> >>>>> Norbert >>>>> >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>>>> >>>>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> I checked the Parasol Archive and it does not appear to be in Zip64 format >>>>> (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can >>>>> read the Parasol zip). So my next guess is that there's either a problem >>>>> with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness >>>>> problem or non-binary stream data). It would therefore be helpful to find >>>>> out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is >>>>> passed / opened, is it binary, does the file exist and is it still the >>>>> correct file. >>>>> >>>>> I couldnât see anything obvious. The file in the debug message exists, it >>>>> is a readable zip file. The way Metacello uses it it is a >>>>> StandardFileStream. It switches it to binary in the code. >>>>> But the only difference between a working and non-working build is the >>>>> upgrade to zinc 2.9.2. >>>>> >>>>> Norbert >>>>> >>>>> I'd debug it myself but I can't run VirtualBox at the moment because I'm >>>>> on the macOS beta and it won't start... >>>>> >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> Hi Norbert, >>>>> >>>>> This is a bit of a guess, but it's possible that the archive that is >>>>> downloaded from github is in Zip64 format and that the libraries for >>>>> extracting Zip64 are missing on your Linux. That would of course contradict >>>>> the experience that the same operation appears to work in 6.1. >>>>> >>>>> to be honest I donât know what Zip64 format is. I thought the Zip classes >>>>> are pure smalltalk for unpacking. >>>>> >>>>> Try extracting the archive manually on your Linux machine with the same >>>>> method that Metacello uses (I assume, Metacello uses the ZipPlugin, which >>>>> will probably use the system libraries). >>>>> >>>>> I have no ZipPlugin as library in any of my vms. >>>>> >>>>> But there are zips downloaded and unpacked. I start the image the first >>>>> time loading all the code of my project. Then it is saved, copied to a new >>>>> name and reopened to load the tests and then it fails. I did try to load >>>>> the tests in the first run and then it works. >>>>> >>>>> Iâm running out of ideas >>>>> >>>>> thanks, >>>>> >>>>> Norbert >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>>>> >>>>> With the help of Esteban I got one step further. If I do >>>>> >>>>> MCWorkingCopy >>>>> managersForClass: ZnFileSystemUtils >>>>> do: [ :each | each ancestry initialize ] >>>>> >>>>> before loading my project it updates Zinc-FileSystem as well. Sadly it >>>>> still does not work for me because I get >>>>> >>>>> Loading baseline of BaselineOfMobilityMap... >>>>> ...RETRY->BaselineOfParasol >>>>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>>>> >>>>> and I donât know why. zip file is downloaded from github and present but >>>>> it fails and only on my jenkins on linux. On my Mac this works. >>>>> >>>>> I try a few things but then Iâm back on pharo6.1 for the time being :( >>>>> >>>>> Norbert >>>>> >>>>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>>>> >>>>> Really? I thought the same but then I didnât believe it works like that. >>>>> >>>>> Anyway this would be very easy to solve. We just need to ask Sven if he is >>>>> fine with doing an empty .16 version for Zinc-FileSystem and does an >>>>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced >>>>> that will solve it but the cost wonât be very high. >>>>> >>>>> Norbert >>>>> >>>>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: >>>>> >>>>> Need to investigate more. There are two .15 versions but there is 1 year >>>>> in between (if you didnât notice TheIntegratior.15 is from 2017). Just want >>>>> to have more context information because I can only see that this is >>>>> strange but lacking insight. >>>>> >>>>> Iâm trying to figure out why it does not update Zinc-FileSystem. No matter >>>>> what I do I cannot get Metacello to load the newer package. That would fix >>>>> the issue because Iâm loading 2.9.2 which should have >>>>> Zinc-FileSystem-SVC.15 and not stay on the one included in the image. >>>>> >>>>> I think this is important for everyone that has a product based on 6.1 and >>>>> that want to migrate someday to pharo7. This way it is impossible to do it >>>>> step by step. >>>>> >>>>> If there is a package .15 in the image and a package .15 in the repo, I >>>>> think Metacello will not update because it rely on the numbers to know when >>>>> to update. If it find a .15 and currently have a .15 I think Metacello will >>>>> think they are at the same version. >>>>> >>>>> Norbert >>>>> >>>>> -- >>>>> Cyril Ferlicot >>>>> https://ferlicot.fr >>>>> >>>> >>>> >>> >>> >> >> > >

July 22, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Norbert Hartl

Sven, thank you very much for quickly providing a new version. I changed the dependencies of my project for 2.9.4 and everything is fine now. So I can start using pharo7 on my laptop and keep 6.1 images for building the deployment artefact. Great! thanks, Norberr > Am 22.07.2018 um 12:59 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: > > Done: > > === > Name: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.117 > Author: SvenVanCaekenberghe > Time: 22 July 2018, 12:58:12.362 pm > UUID: 949d5a24-f62d-0d00-a12c-ead9078b1897 > Ancestors: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.116 > > stable v 2.9.4 > === > > Please tell us if this works for you > >> On 22 Jul 2018, at 12:10, Norbert Hartl <norbert(a)hartl.name> wrote: >> >> Thank you guys! It is really great to see that there are people like Max jumping onto problems like this and solving it. >> >> @sven Iâm eager to test when the new configuration is available. Please drop us a note when you are done! >> >> thanks again, >> >> Norbert >> >>> Am 22.07.2018 um 11:06 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >>> >>> Ah, that makes total sense. Great catch. Thank you. >>> >>> === >>> Name: Zinc-FileSystem-SvenVanCaekenberghe.17 >>> Author: SvenVanCaekenberghe >>> Time: 22 July 2018, 11:04:40.148339 am >>> UUID: e97a508e-f42d-0d00-a127-f016078b1897 >>> Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.16 >>> >>> Bugfix to ZnFileSystemUtils class>>#newBinaryFileNamed:do: we must ensure that the binaryStream opened here is closed as well (thx Max Leske for the bug hunting and fix) >>> === >>> >>> The method in question now look as follows: >>> >>> newBinaryFileNamed: fileName do: block >>> | fileReference | >>> fileReference := fileName asFileReference. >>> ^ (fileReference respondsTo: #binaryWriteStreamDo:ifPresent:) >>> ifTrue: [ >>> fileName asFileReference >>> binaryWriteStreamDo: block >>> ifPresent: [ FileExists signalWith: fileReference ] ] >>> ifFalse: [ >>> fileReference isFile >>> ifTrue: [ FileExists signalWith: fileReference ] >>> ifFalse: [ | binaryStream | >>> binaryStream := self binaryFileStreamFor: fileName. >>> [ block value: binaryStream ] ensure: [ binaryStream close ] ] ] >>> >>> I create a new 2.9.4 of the config later on. >>> >>>> On 21 Jul 2018, at 22:00, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>> Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: >>>> >>>> block value: (self binaryFileStreamFor: fileName) >>>> >>>> This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: >>>> >>>> | s | >>>> s := self binaryFileStreamFor: fileName. >>>> [ block value: s ] ensure: [ s close ] >>>> >>>> ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. >>>> >>>> Cheers, >>>> Max >>>> >>>> On 21 Jul 2018, at 17:21, Max Leske wrote: >>>> >>>> Just to be safe, I'll redo my experiment and get back to you. >>>> >>>> On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: >>>> I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), >>>> >>>> ZnClient new >>>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>> downloadTo: '/tmp/foobar.zip'. >>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>>> Transcript open; show: bytes size; cr. >>>> 5 seconds asDelay wait. >>>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>>> >>>> And it gives me, >>>> >>>> 318420 >>>> 318420 >>>> >>>> For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. >>>> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >>>> Max, >>>> >>>> what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. >>>> >>>> In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. >>>> I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. >>>> >>>> Cheers, >>>> Max >>>> Norbert >>>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>>> >>>> Hi Nicolas, >>>> >>>> The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. >>>> >>>> Thanks for the pointer though. >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>> >>>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>>> >>>> Isn't there a recent PR on opensmalltalk that address just this? >>>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : >>>> >>>> Hi Norbert, >>>> >>>> I was able to reproduce the problem and then identify the culprit, >>>> although what I don't yet understand is how this is related to the changes >>>> in Zinc. >>>> >>>> Anyway, the problem is that the file stream isn't properly flushed in >>>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last >>>> statement fixes your problem. Apparently, StandardFileStream / >>>> MultiByteFileStream perform a simple close() on the file which, according >>>> to the man page on close(), does *not* guarantee that the contents have >>>> all been written to file. In this case a flush is necessary because the >>>> entire file is immediately read again. >>>> >>>> Here's a smaller test case for you to play with Sven: >>>> >>>> ZnClient new >>>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>> downloadTo: '/tmp/foobar.zip'. >>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>>> Transcript open; show: bytes size; cr. >>>> 5 seconds asDelay wait. >>>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>>> >>>> The output in the Transcript should be: >>>> >>>> 315392 >>>> 318420 >>>> >>>> Cheers, >>>> Max >>>> >>>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>>> >>>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>>> >>>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>>> >>>> Hi Max, >>>> >>>> I constructed a case that fails exactly like I experience it. Could you >>>> try? Just unpack the attachment on a linux, set PHARO_VM env to your >>>> executable and execute build.sh >>>> >>>> I will. Might take a couple of days though. >>>> >>>> No problem. Iâm happy if you find time. >>>> >>>> Norbert >>>> >>>> Max >>>> >>>> Norbert >>>> >>>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>>> >>>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>>> >>>> Max, >>>> >>>> thanks for taking the effort. >>>> >>>> No worries. >>>> >>>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>>> >>>> I did my best to reproduce the issue but didn't have any luck. I really >>>> need access to a Linux VM to debug this. So I'm praying that Apple fixes >>>> the access restrictions to kernel extensions in their next beta... >>>> >>>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is >>>> indeed a chance that something goes wrong during download of the zip >>>> archive and that something may be tied to a difference in the versions of >>>> Zinc (although I still think that's unlikely). >>>> >>>> Yes there is potential but I donât see it. I take a fresh 6.1 image and >>>> load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather >>>> early in that process. So I wonder why it does not go wrong in the first >>>> phase. And also not if I load the test group within the first phase. >>>> It must be either the second Metacello invocation or the stopping, copying >>>> and starting of the image. >>>> I try to isolate this case more and provide a script that goes wrong on my >>>> machine. But it will take some time because I needed to stop trying to >>>> solve this as I wasted nearly two days on that already. >>>> >>>> Let me know once you have something and I'll try to help out. >>>> >>>> Norbert >>>> >>>> Max >>>> >>>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>>> >>>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>>> >>>> I checked the Parasol Archive and it does not appear to be in Zip64 format >>>> (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can >>>> read the Parasol zip). So my next guess is that there's either a problem >>>> with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness >>>> problem or non-binary stream data). It would therefore be helpful to find >>>> out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is >>>> passed / opened, is it binary, does the file exist and is it still the >>>> correct file. >>>> >>>> I couldnât see anything obvious. The file in the debug message exists, it >>>> is a readable zip file. The way Metacello uses it it is a >>>> StandardFileStream. It switches it to binary in the code. >>>> But the only difference between a working and non-working build is the >>>> upgrade to zinc 2.9.2. >>>> >>>> Norbert >>>> >>>> I'd debug it myself but I can't run VirtualBox at the moment because I'm >>>> on the macOS beta and it won't start... >>>> >>>> Max >>>> >>>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>>> >>>> Hi Max, >>>> >>>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>>> >>>> Hi Norbert, >>>> >>>> This is a bit of a guess, but it's possible that the archive that is >>>> downloaded from github is in Zip64 format and that the libraries for >>>> extracting Zip64 are missing on your Linux. That would of course contradict >>>> the experience that the same operation appears to work in 6.1. >>>> >>>> to be honest I donât know what Zip64 format is. I thought the Zip classes >>>> are pure smalltalk for unpacking. >>>> >>>> Try extracting the archive manually on your Linux machine with the same >>>> method that Metacello uses (I assume, Metacello uses the ZipPlugin, which >>>> will probably use the system libraries). >>>> >>>> I have no ZipPlugin as library in any of my vms. >>>> >>>> But there are zips downloaded and unpacked. I start the image the first >>>> time loading all the code of my project. Then it is saved, copied to a new >>>> name and reopened to load the tests and then it fails. I did try to load >>>> the tests in the first run and then it works. >>>> >>>> Iâm running out of ideas >>>> >>>> thanks, >>>> >>>> Norbert >>>> >>>> Cheers, >>>> Max >>>> >>>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>>> >>>> With the help of Esteban I got one step further. If I do >>>> >>>> MCWorkingCopy >>>> managersForClass: ZnFileSystemUtils >>>> do: [ :each | each ancestry initialize ] >>>> >>>> before loading my project it updates Zinc-FileSystem as well. Sadly it >>>> still does not work for me because I get >>>> >>>> Loading baseline of BaselineOfMobilityMap... >>>> ...RETRY->BaselineOfParasol >>>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>>> >>>> and I donât know why. zip file is downloaded from github and present but >>>> it fails and only on my jenkins on linux. On my Mac this works. >>>> >>>> I try a few things but then Iâm back on pharo6.1 for the time being :( >>>> >>>> Norbert >>>> >>>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>>> >>>> Really? I thought the same but then I didnât believe it works like that. >>>> >>>> Anyway this would be very easy to solve. We just need to ask Sven if he is >>>> fine with doing an empty .16 version for Zinc-FileSystem and does an >>>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced >>>> that will solve it but the cost wonât be very high. >>>> >>>> Norbert >>>> >>>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: >>>> >>>> Need to investigate more. There are two .15 versions but there is 1 year >>>> in between (if you didnât notice TheIntegratior.15 is from 2017). Just want >>>> to have more context information because I can only see that this is >>>> strange but lacking insight. >>>> >>>> Iâm trying to figure out why it does not update Zinc-FileSystem. No matter >>>> what I do I cannot get Metacello to load the newer package. That would fix >>>> the issue because Iâm loading 2.9.2 which should have >>>> Zinc-FileSystem-SVC.15 and not stay on the one included in the image. >>>> >>>> I think this is important for everyone that has a product based on 6.1 and >>>> that want to migrate someday to pharo7. This way it is impossible to do it >>>> step by step. >>>> >>>> If there is a package .15 in the image and a package .15 in the repo, I >>>> think Metacello will not update because it rely on the numbers to know when >>>> to update. If it find a .15 and currently have a .15 I think Metacello will >>>> think they are at the same version. >>>> >>>> Norbert >>>> >>>> -- >>>> Cyril Ferlicot >>>> https://ferlicot.fr >>>> >>> >>> >> >> > >

July 22, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Sven Van Caekenberghe

Done: === Name: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.117 Author: SvenVanCaekenberghe Time: 22 July 2018, 12:58:12.362 pm UUID: 949d5a24-f62d-0d00-a12c-ead9078b1897 Ancestors: ConfigurationOfZincHTTPComponents-SvenVanCaekenberghe.116 stable v 2.9.4 === Please tell us if this works for you > On 22 Jul 2018, at 12:10, Norbert Hartl <norbert(a)hartl.name> wrote: > > Thank you guys! It is really great to see that there are people like Max jumping onto problems like this and solving it. > > @sven Iâm eager to test when the new configuration is available. Please drop us a note when you are done! > > thanks again, > > Norbert > >> Am 22.07.2018 um 11:06 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: >> >> Ah, that makes total sense. Great catch. Thank you. >> >> === >> Name: Zinc-FileSystem-SvenVanCaekenberghe.17 >> Author: SvenVanCaekenberghe >> Time: 22 July 2018, 11:04:40.148339 am >> UUID: e97a508e-f42d-0d00-a127-f016078b1897 >> Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.16 >> >> Bugfix to ZnFileSystemUtils class>>#newBinaryFileNamed:do: we must ensure that the binaryStream opened here is closed as well (thx Max Leske for the bug hunting and fix) >> === >> >> The method in question now look as follows: >> >> newBinaryFileNamed: fileName do: block >> | fileReference | >> fileReference := fileName asFileReference. >> ^ (fileReference respondsTo: #binaryWriteStreamDo:ifPresent:) >> ifTrue: [ >> fileName asFileReference >> binaryWriteStreamDo: block >> ifPresent: [ FileExists signalWith: fileReference ] ] >> ifFalse: [ >> fileReference isFile >> ifTrue: [ FileExists signalWith: fileReference ] >> ifFalse: [ | binaryStream | >> binaryStream := self binaryFileStreamFor: fileName. >> [ block value: binaryStream ] ensure: [ binaryStream close ] ] ] >> >> I create a new 2.9.4 of the config later on. >> >>> On 21 Jul 2018, at 22:00, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: >>> >>> block value: (self binaryFileStreamFor: fileName) >>> >>> This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: >>> >>> | s | >>> s := self binaryFileStreamFor: fileName. >>> [ block value: s ] ensure: [ s close ] >>> >>> ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. >>> >>> Cheers, >>> Max >>> >>> On 21 Jul 2018, at 17:21, Max Leske wrote: >>> >>> Just to be safe, I'll redo my experiment and get back to you. >>> >>> On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: >>> I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), >>> >>> ZnClient new >>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>> downloadTo: '/tmp/foobar.zip'. >>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>> Transcript open; show: bytes size; cr. >>> 5 seconds asDelay wait. >>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>> >>> And it gives me, >>> >>> 318420 >>> 318420 >>> >>> For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. >>> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >>> Max, >>> >>> what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. >>> >>> In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. >>> I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. >>> >>> Cheers, >>> Max >>> Norbert >>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>> >>> Hi Nicolas, >>> >>> The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. >>> >>> Thanks for the pointer though. >>> >>> >>> Cheers, >>> Max >>> >>> >>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>> >>> Isn't there a recent PR on opensmalltalk that address just this? >>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : >>> >>> Hi Norbert, >>> >>> I was able to reproduce the problem and then identify the culprit, >>> although what I don't yet understand is how this is related to the changes >>> in Zinc. >>> >>> Anyway, the problem is that the file stream isn't properly flushed in >>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last >>> statement fixes your problem. Apparently, StandardFileStream / >>> MultiByteFileStream perform a simple close() on the file which, according >>> to the man page on close(), does *not* guarantee that the contents have >>> all been written to file. In this case a flush is necessary because the >>> entire file is immediately read again. >>> >>> Here's a smaller test case for you to play with Sven: >>> >>> ZnClient new >>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>> downloadTo: '/tmp/foobar.zip'. >>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>> Transcript open; show: bytes size; cr. >>> 5 seconds asDelay wait. >>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>> >>> The output in the Transcript should be: >>> >>> 315392 >>> 318420 >>> >>> Cheers, >>> Max >>> >>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>> >>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>> >>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>> >>> Hi Max, >>> >>> I constructed a case that fails exactly like I experience it. Could you >>> try? Just unpack the attachment on a linux, set PHARO_VM env to your >>> executable and execute build.sh >>> >>> I will. Might take a couple of days though. >>> >>> No problem. Iâm happy if you find time. >>> >>> Norbert >>> >>> Max >>> >>> Norbert >>> >>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>> >>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>> >>> Max, >>> >>> thanks for taking the effort. >>> >>> No worries. >>> >>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>> >>> I did my best to reproduce the issue but didn't have any luck. I really >>> need access to a Linux VM to debug this. So I'm praying that Apple fixes >>> the access restrictions to kernel extensions in their next beta... >>> >>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is >>> indeed a chance that something goes wrong during download of the zip >>> archive and that something may be tied to a difference in the versions of >>> Zinc (although I still think that's unlikely). >>> >>> Yes there is potential but I donât see it. I take a fresh 6.1 image and >>> load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather >>> early in that process. So I wonder why it does not go wrong in the first >>> phase. And also not if I load the test group within the first phase. >>> It must be either the second Metacello invocation or the stopping, copying >>> and starting of the image. >>> I try to isolate this case more and provide a script that goes wrong on my >>> machine. But it will take some time because I needed to stop trying to >>> solve this as I wasted nearly two days on that already. >>> >>> Let me know once you have something and I'll try to help out. >>> >>> Norbert >>> >>> Max >>> >>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>> >>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>> >>> I checked the Parasol Archive and it does not appear to be in Zip64 format >>> (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can >>> read the Parasol zip). So my next guess is that there's either a problem >>> with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness >>> problem or non-binary stream data). It would therefore be helpful to find >>> out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is >>> passed / opened, is it binary, does the file exist and is it still the >>> correct file. >>> >>> I couldnât see anything obvious. The file in the debug message exists, it >>> is a readable zip file. The way Metacello uses it it is a >>> StandardFileStream. It switches it to binary in the code. >>> But the only difference between a working and non-working build is the >>> upgrade to zinc 2.9.2. >>> >>> Norbert >>> >>> I'd debug it myself but I can't run VirtualBox at the moment because I'm >>> on the macOS beta and it won't start... >>> >>> Max >>> >>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>> >>> Hi Max, >>> >>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>> >>> Hi Norbert, >>> >>> This is a bit of a guess, but it's possible that the archive that is >>> downloaded from github is in Zip64 format and that the libraries for >>> extracting Zip64 are missing on your Linux. That would of course contradict >>> the experience that the same operation appears to work in 6.1. >>> >>> to be honest I donât know what Zip64 format is. I thought the Zip classes >>> are pure smalltalk for unpacking. >>> >>> Try extracting the archive manually on your Linux machine with the same >>> method that Metacello uses (I assume, Metacello uses the ZipPlugin, which >>> will probably use the system libraries). >>> >>> I have no ZipPlugin as library in any of my vms. >>> >>> But there are zips downloaded and unpacked. I start the image the first >>> time loading all the code of my project. Then it is saved, copied to a new >>> name and reopened to load the tests and then it fails. I did try to load >>> the tests in the first run and then it works. >>> >>> Iâm running out of ideas >>> >>> thanks, >>> >>> Norbert >>> >>> Cheers, >>> Max >>> >>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>> >>> With the help of Esteban I got one step further. If I do >>> >>> MCWorkingCopy >>> managersForClass: ZnFileSystemUtils >>> do: [ :each | each ancestry initialize ] >>> >>> before loading my project it updates Zinc-FileSystem as well. Sadly it >>> still does not work for me because I get >>> >>> Loading baseline of BaselineOfMobilityMap... >>> ...RETRY->BaselineOfParasol >>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>> >>> and I donât know why. zip file is downloaded from github and present but >>> it fails and only on my jenkins on linux. On my Mac this works. >>> >>> I try a few things but then Iâm back on pharo6.1 for the time being :( >>> >>> Norbert >>> >>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>> >>> Really? I thought the same but then I didnât believe it works like that. >>> >>> Anyway this would be very easy to solve. We just need to ask Sven if he is >>> fine with doing an empty .16 version for Zinc-FileSystem and does an >>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced >>> that will solve it but the cost wonât be very high. >>> >>> Norbert >>> >>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: >>> >>> Need to investigate more. There are two .15 versions but there is 1 year >>> in between (if you didnât notice TheIntegratior.15 is from 2017). Just want >>> to have more context information because I can only see that this is >>> strange but lacking insight. >>> >>> Iâm trying to figure out why it does not update Zinc-FileSystem. No matter >>> what I do I cannot get Metacello to load the newer package. That would fix >>> the issue because Iâm loading 2.9.2 which should have >>> Zinc-FileSystem-SVC.15 and not stay on the one included in the image. >>> >>> I think this is important for everyone that has a product based on 6.1 and >>> that want to migrate someday to pharo7. This way it is impossible to do it >>> step by step. >>> >>> If there is a package .15 in the image and a package .15 in the repo, I >>> think Metacello will not update because it rely on the numbers to know when >>> to update. If it find a .15 and currently have a .15 I think Metacello will >>> think they are at the same version. >>> >>> Norbert >>> >>> -- >>> Cyril Ferlicot >>> https://ferlicot.fr >>> >> >> > >

July 22, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Norbert Hartl

Thank you guys! It is really great to see that there are people like Max jumping onto problems like this and solving it. @sven Iâm eager to test when the new configuration is available. Please drop us a note when you are done! thanks again, Norbert > Am 22.07.2018 um 11:06 schrieb Sven Van Caekenberghe <sven(a)stfx.eu>: > > Ah, that makes total sense. Great catch. Thank you. > > === > Name: Zinc-FileSystem-SvenVanCaekenberghe.17 > Author: SvenVanCaekenberghe > Time: 22 July 2018, 11:04:40.148339 am > UUID: e97a508e-f42d-0d00-a127-f016078b1897 > Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.16 > > Bugfix to ZnFileSystemUtils class>>#newBinaryFileNamed:do: we must ensure that the binaryStream opened here is closed as well (thx Max Leske for the bug hunting and fix) > === > > The method in question now look as follows: > > newBinaryFileNamed: fileName do: block > | fileReference | > fileReference := fileName asFileReference. > ^ (fileReference respondsTo: #binaryWriteStreamDo:ifPresent:) > ifTrue: [ > fileName asFileReference > binaryWriteStreamDo: block > ifPresent: [ FileExists signalWith: fileReference ] ] > ifFalse: [ > fileReference isFile > ifTrue: [ FileExists signalWith: fileReference ] > ifFalse: [ | binaryStream | > binaryStream := self binaryFileStreamFor: fileName. > [ block value: binaryStream ] ensure: [ binaryStream close ] ] ] > > I create a new 2.9.4 of the config later on. > >> On 21 Jul 2018, at 22:00, Max Leske <maxleske(a)gmail.com> wrote: >> >> Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: >> >> block value: (self binaryFileStreamFor: fileName) >> >> This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: >> >> | s | >> s := self binaryFileStreamFor: fileName. >> [ block value: s ] ensure: [ s close ] >> >> ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. >> >> Cheers, >> Max >> >> On 21 Jul 2018, at 17:21, Max Leske wrote: >> >> Just to be safe, I'll redo my experiment and get back to you. >> >> On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: >> I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), >> >> ZnClient new >> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >> downloadTo: '/tmp/foobar.zip'. >> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >> Transcript open; show: bytes size; cr. >> 5 seconds asDelay wait. >> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >> >> And it gives me, >> >> 318420 >> 318420 >> >> For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. >> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >> >> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >> Max, >> >> what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. >> >> In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. >> I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. >> >> Cheers, >> Max >> Norbert >> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >> >> Hi Nicolas, >> >> The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. >> >> Thanks for the pointer though. >> >> >> Cheers, >> Max >> >> >> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >> >> Isn't there a recent PR on opensmalltalk that address just this? >> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : >> >> Hi Norbert, >> >> I was able to reproduce the problem and then identify the culprit, >> although what I don't yet understand is how this is related to the changes >> in Zinc. >> >> Anyway, the problem is that the file stream isn't properly flushed in >> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last >> statement fixes your problem. Apparently, StandardFileStream / >> MultiByteFileStream perform a simple close() on the file which, according >> to the man page on close(), does *not* guarantee that the contents have >> all been written to file. In this case a flush is necessary because the >> entire file is immediately read again. >> >> Here's a smaller test case for you to play with Sven: >> >> ZnClient new >> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >> downloadTo: '/tmp/foobar.zip'. >> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >> Transcript open; show: bytes size; cr. >> 5 seconds asDelay wait. >> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >> >> The output in the Transcript should be: >> >> 315392 >> 318420 >> >> Cheers, >> Max >> >> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >> >> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >> >> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >> >> Hi Max, >> >> I constructed a case that fails exactly like I experience it. Could you >> try? Just unpack the attachment on a linux, set PHARO_VM env to your >> executable and execute build.sh >> >> I will. Might take a couple of days though. >> >> No problem. Iâm happy if you find time. >> >> Norbert >> >> Max >> >> Norbert >> >> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >> >> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >> >> Max, >> >> thanks for taking the effort. >> >> No worries. >> >> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >> >> I did my best to reproduce the issue but didn't have any luck. I really >> need access to a Linux VM to debug this. So I'm praying that Apple fixes >> the access restrictions to kernel extensions in their next beta... >> >> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is >> indeed a chance that something goes wrong during download of the zip >> archive and that something may be tied to a difference in the versions of >> Zinc (although I still think that's unlikely). >> >> Yes there is potential but I donât see it. I take a fresh 6.1 image and >> load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather >> early in that process. So I wonder why it does not go wrong in the first >> phase. And also not if I load the test group within the first phase. >> It must be either the second Metacello invocation or the stopping, copying >> and starting of the image. >> I try to isolate this case more and provide a script that goes wrong on my >> machine. But it will take some time because I needed to stop trying to >> solve this as I wasted nearly two days on that already. >> >> Let me know once you have something and I'll try to help out. >> >> Norbert >> >> Max >> >> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >> >> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >> >> I checked the Parasol Archive and it does not appear to be in Zip64 format >> (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can >> read the Parasol zip). So my next guess is that there's either a problem >> with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness >> problem or non-binary stream data). It would therefore be helpful to find >> out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is >> passed / opened, is it binary, does the file exist and is it still the >> correct file. >> >> I couldnât see anything obvious. The file in the debug message exists, it >> is a readable zip file. The way Metacello uses it it is a >> StandardFileStream. It switches it to binary in the code. >> But the only difference between a working and non-working build is the >> upgrade to zinc 2.9.2. >> >> Norbert >> >> I'd debug it myself but I can't run VirtualBox at the moment because I'm >> on the macOS beta and it won't start... >> >> Max >> >> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >> >> Hi Max, >> >> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >> >> Hi Norbert, >> >> This is a bit of a guess, but it's possible that the archive that is >> downloaded from github is in Zip64 format and that the libraries for >> extracting Zip64 are missing on your Linux. That would of course contradict >> the experience that the same operation appears to work in 6.1. >> >> to be honest I donât know what Zip64 format is. I thought the Zip classes >> are pure smalltalk for unpacking. >> >> Try extracting the archive manually on your Linux machine with the same >> method that Metacello uses (I assume, Metacello uses the ZipPlugin, which >> will probably use the system libraries). >> >> I have no ZipPlugin as library in any of my vms. >> >> But there are zips downloaded and unpacked. I start the image the first >> time loading all the code of my project. Then it is saved, copied to a new >> name and reopened to load the tests and then it fails. I did try to load >> the tests in the first run and then it works. >> >> Iâm running out of ideas >> >> thanks, >> >> Norbert >> >> Cheers, >> Max >> >> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >> >> With the help of Esteban I got one step further. If I do >> >> MCWorkingCopy >> managersForClass: ZnFileSystemUtils >> do: [ :each | each ancestry initialize ] >> >> before loading my project it updates Zinc-FileSystem as well. Sadly it >> still does not work for me because I get >> >> Loading baseline of BaselineOfMobilityMap... >> ...RETRY->BaselineOfParasol >> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >> >> and I donât know why. zip file is downloaded from github and present but >> it fails and only on my jenkins on linux. On my Mac this works. >> >> I try a few things but then Iâm back on pharo6.1 for the time being :( >> >> Norbert >> >> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >> >> Really? I thought the same but then I didnât believe it works like that. >> >> Anyway this would be very easy to solve. We just need to ask Sven if he is >> fine with doing an empty .16 version for Zinc-FileSystem and does an >> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced >> that will solve it but the cost wonât be very high. >> >> Norbert >> >> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: >> >> Need to investigate more. There are two .15 versions but there is 1 year >> in between (if you didnât notice TheIntegratior.15 is from 2017). Just want >> to have more context information because I can only see that this is >> strange but lacking insight. >> >> Iâm trying to figure out why it does not update Zinc-FileSystem. No matter >> what I do I cannot get Metacello to load the newer package. That would fix >> the issue because Iâm loading 2.9.2 which should have >> Zinc-FileSystem-SVC.15 and not stay on the one included in the image. >> >> I think this is important for everyone that has a product based on 6.1 and >> that want to migrate someday to pharo7. This way it is impossible to do it >> step by step. >> >> If there is a package .15 in the image and a package .15 in the repo, I >> think Metacello will not update because it rely on the numbers to know when >> to update. If it find a .15 and currently have a .15 I think Metacello will >> think they are at the same version. >> >> Norbert >> >> -- >> Cyril Ferlicot >> https://ferlicot.fr >> > >

July 22, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Sven Van Caekenberghe

Ah, that makes total sense. Great catch. Thank you. === Name: Zinc-FileSystem-SvenVanCaekenberghe.17 Author: SvenVanCaekenberghe Time: 22 July 2018, 11:04:40.148339 am UUID: e97a508e-f42d-0d00-a127-f016078b1897 Ancestors: Zinc-FileSystem-SvenVanCaekenberghe.16 Bugfix to ZnFileSystemUtils class>>#newBinaryFileNamed:do: we must ensure that the binaryStream opened here is closed as well (thx Max Leske for the bug hunting and fix) === The method in question now look as follows: newBinaryFileNamed: fileName do: block | fileReference | fileReference := fileName asFileReference. ^ (fileReference respondsTo: #binaryWriteStreamDo:ifPresent:) ifTrue: [ fileName asFileReference binaryWriteStreamDo: block ifPresent: [ FileExists signalWith: fileReference ] ] ifFalse: [ fileReference isFile ifTrue: [ FileExists signalWith: fileReference ] ifFalse: [ | binaryStream | binaryStream := self binaryFileStreamFor: fileName. [ block value: binaryStream ] ensure: [ binaryStream close ] ] ] I create a new 2.9.4 of the config later on. > On 21 Jul 2018, at 22:00, Max Leske <maxleske(a)gmail.com> wrote: > > Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: > > block value: (self binaryFileStreamFor: fileName) > > This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: > > | s | > s := self binaryFileStreamFor: fileName. > [ block value: s ] ensure: [ s close ] > > ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. > > Cheers, > Max > > On 21 Jul 2018, at 17:21, Max Leske wrote: > > Just to be safe, I'll redo my experiment and get back to you. > > On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: > I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), > > ZnClient new > url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; > downloadTo: '/tmp/foobar.zip'. > bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. > Transcript open; show: bytes size; cr. > 5 seconds asDelay wait. > Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. > > And it gives me, > > 318420 > 318420 > > For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. > On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: > > On 21 Jul 2018, at 12:07, Norbert Hartl wrote: > Max, > > what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. > > In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. > I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. > > Cheers, > Max > Norbert > Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: > > Hi Nicolas, > > The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. > > Thanks for the pointer though. > > > Cheers, > Max > > > [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 > On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: > > Isn't there a recent PR on opensmalltalk that address just this? > Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : > > Hi Norbert, > > I was able to reproduce the problem and then identify the culprit, > although what I don't yet understand is how this is related to the changes > in Zinc. > > Anyway, the problem is that the file stream isn't properly flushed in > ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last > statement fixes your problem. Apparently, StandardFileStream / > MultiByteFileStream perform a simple close() on the file which, according > to the man page on close(), does *not* guarantee that the contents have > all been written to file. In this case a flush is necessary because the > entire file is immediately read again. > > Here's a smaller test case for you to play with Sven: > > ZnClient new > url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; > downloadTo: '/tmp/foobar.zip'. > bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. > Transcript open; show: bytes size; cr. > 5 seconds asDelay wait. > Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. > > The output in the Transcript should be: > > 315392 > 318420 > > Cheers, > Max > > On 12 Jul 2018, at 8:17, Norbert Hartl wrote: > > Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: > > On 11 Jul 2018, at 22:44, Norbert Hartl wrote: > > Hi Max, > > I constructed a case that fails exactly like I experience it. Could you > try? Just unpack the attachment on a linux, set PHARO_VM env to your > executable and execute build.sh > > I will. Might take a couple of days though. > > No problem. Iâm happy if you find time. > > Norbert > > Max > > Norbert > > Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: > > On 10 Jul 2018, at 8:48, Norbert Hartl wrote: > > Max, > > thanks for taking the effort. > > No worries. > > Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: > > I did my best to reproduce the issue but didn't have any luck. I really > need access to a Linux VM to debug this. So I'm praying that Apple fixes > the access restrictions to kernel extensions in their next beta... > > BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is > indeed a chance that something goes wrong during download of the zip > archive and that something may be tied to a difference in the versions of > Zinc (although I still think that's unlikely). > > Yes there is potential but I donât see it. I take a fresh 6.1 image and > load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather > early in that process. So I wonder why it does not go wrong in the first > phase. And also not if I load the test group within the first phase. > It must be either the second Metacello invocation or the stopping, copying > and starting of the image. > I try to isolate this case more and provide a script that goes wrong on my > machine. But it will take some time because I needed to stop trying to > solve this as I wasted nearly two days on that already. > > Let me know once you have something and I'll try to help out. > > Norbert > > Max > > On 9 Jul 2018, at 19:43, Norbert Hartl wrote: > > Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: > > I checked the Parasol Archive and it does not appear to be in Zip64 format > (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can > read the Parasol zip). So my next guess is that there's either a problem > with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness > problem or non-binary stream data). It would therefore be helpful to find > out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is > passed / opened, is it binary, does the file exist and is it still the > correct file. > > I couldnât see anything obvious. The file in the debug message exists, it > is a readable zip file. The way Metacello uses it it is a > StandardFileStream. It switches it to binary in the code. > But the only difference between a working and non-working build is the > upgrade to zinc 2.9.2. > > Norbert > > I'd debug it myself but I can't run VirtualBox at the moment because I'm > on the macOS beta and it won't start... > > Max > > On 9 Jul 2018, at 18:31, Norbert Hartl wrote: > > Hi Max, > > Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: > > Hi Norbert, > > This is a bit of a guess, but it's possible that the archive that is > downloaded from github is in Zip64 format and that the libraries for > extracting Zip64 are missing on your Linux. That would of course contradict > the experience that the same operation appears to work in 6.1. > > to be honest I donât know what Zip64 format is. I thought the Zip classes > are pure smalltalk for unpacking. > > Try extracting the archive manually on your Linux machine with the same > method that Metacello uses (I assume, Metacello uses the ZipPlugin, which > will probably use the system libraries). > > I have no ZipPlugin as library in any of my vms. > > But there are zips downloaded and unpacked. I start the image the first > time loading all the code of my project. Then it is saved, copied to a new > name and reopened to load the tests and then it fails. I did try to load > the tests in the first run and then it works. > > Iâm running out of ideas > > thanks, > > Norbert > > Cheers, > Max > > On 9 Jul 2018, at 17:14, Norbert Hartl wrote: > > With the help of Esteban I got one step further. If I do > > MCWorkingCopy > managersForClass: ZnFileSystemUtils > do: [ :each | each ancestry initialize ] > > before loading my project it updates Zinc-FileSystem as well. Sadly it > still does not work for me because I get > > Loading baseline of BaselineOfMobilityMap... > ...RETRY->BaselineOfParasol > ...RETRY->BaselineOfParasol[31mError: can't find EOCD position > > and I donât know why. zip file is downloaded from github and present but > it fails and only on my jenkins on linux. On my Mac this works. > > I try a few things but then Iâm back on pharo6.1 for the time being :( > > Norbert > > Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: > > Really? I thought the same but then I didnât believe it works like that. > > Anyway this would be very easy to solve. We just need to ask Sven if he is > fine with doing an empty .16 version for Zinc-FileSystem and does an > in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced > that will solve it but the cost wonât be very high. > > Norbert > > Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: > > Need to investigate more. There are two .15 versions but there is 1 year > in between (if you didnât notice TheIntegratior.15 is from 2017). Just want > to have more context information because I can only see that this is > strange but lacking insight. > > Iâm trying to figure out why it does not update Zinc-FileSystem. No matter > what I do I cannot get Metacello to load the newer package. That would fix > the issue because Iâm loading 2.9.2 which should have > Zinc-FileSystem-SVC.15 and not stay on the one included in the image. > > I think this is important for everyone that has a product based on 6.1 and > that want to migrate someday to pharo7. This way it is impossible to do it > step by step. > > If there is a package .15 in the image and a package .15 in the repo, I > think Metacello will not update because it rely on the numbers to know when > to update. If it find a .15 and currently have a .15 I think Metacello will > think they are at the same version. > > Norbert > > -- > Cyril Ferlicot > https://ferlicot.fr >

July 22, 2018

Re: [Pharo-dev] [PROPOSAL] Put the WorldMenu in a Menu bar

by Eric Gade

This should definitely be a theme option On Sat, Jul 21, 2018 at 11:28 AM, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > Sean P. DeNigris wrote > > If there's a "preserve the status quo" option (e.g. World Menu behavior > > unchanged and no toolbar), I'm happy! > > Is there still no way to get the old World Menu back in Pharo 7? Am I the > only one that wants this? On the positive side, I had to finally learn and > use the tool opening shortcuts. > > My understanding of general experimental-new-feature etiquette is that > whoever broke something should fix itâ¦ > > > > ----- > Cheers, > Sean > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html > > -- Eric

July 22, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Max Leske

Sven, you're right. What I found last time works as a side-effect. The real problem is in ZnFileSystemUtils class>>newBinaryFileNamed:do:. In Pharo 6.1 the logic there leads to the following evaluation: `block value: (self binaryFileStreamFor: fileName)` This is the only place where the stream is created without a surrounding safety block (e.g. #writeStream:do:), which means, no one closes the stream. The line should be: `| s | s := self binaryFileStreamFor: fileName. [ block value: s ] ensure: [ s close ]` ZnFileSystemUtils class>>newBinaryFileNamed:do: is only used by ZnClient after loading Zinc-HTTP-SvenVanCaekenberghe.472, which is being loaded by version 2.9.1 of ConfigurationOfZincHTTPComponents. Cheers, Max On 21 Jul 2018, at 17:21, Max Leske wrote: > Just to be safe, I'll redo my experiment and get back to you. > > On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: > >> I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), >> >> ZnClient new >> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >> downloadTo: '/tmp/foobar.zip'. >> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | >> s contents ]. >> Transcript open; show: bytes size; cr. >> 5 seconds asDelay wait. >> Transcript show: ('/tmp/foobar.zip' asFileReference >> binaryReadStreamDo: [ :s | s contents ]) size. >> >> And it gives me, >> >> 318420 >> 318420 >> >> For me there is nothing nothing to see here ... and I have the >> impression we are all talking about different things. >> >>> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >>> >>>> Max, >>>> >>>> what do you think will be a proper fix for this? I cannot see if >>>> this is a vm problem or not. >>> >>> In my opinion it should be fixed in Zinc as there's no way for the >>> user of Zinc to flush the stream. The alternative would be for Zinc >>> to expose a method or setting that allows the user to specify the >>> expected behaviour. >>> I do not think that this should be handled by the VM plugin or the >>> stream class, as there are indeed different behaviours that make >>> sense. >>> >>> Cheers, >>> Max >>> >>>> >>>> Norbert >>>> >>>>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>>>> >>>>> Hi Nicolas, >>>>> >>>>> The PR you are referring [1] to concerns a missing fflush() when >>>>> truncating a file. I believe that is a different case to this one, >>>>> as a) the file is not truncated and b) in the case of truncation >>>>> it is an explicit requirement that the file be empty before the >>>>> first write happens. In our case, and in general, it depends on >>>>> the situation whether flushing must occur immediately, e.g. >>>>> because the file will be read again immediately, or not, e.g. when >>>>> writing a data to a backup file. >>>>> >>>>> Thanks for the pointer though. >>>>> >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> >>>>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>>>> >>>>>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>>>>> >>>>>> Isn't there a recent PR on opensmalltalk that address just this? >>>>>> >>>>>>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a >>>>>>> Ã©crit : >>>>>>> >>>>>>> Hi Norbert, >>>>>>> >>>>>>> I was able to reproduce the problem and then identify the >>>>>>> culprit, >>>>>>> although what I don't yet understand is how this is related to >>>>>>> the changes >>>>>>> in Zinc. >>>>>>> >>>>>>> Anyway, the problem is that the file stream isn't properly >>>>>>> flushed in >>>>>>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as >>>>>>> the last >>>>>>> statement fixes your problem. Apparently, StandardFileStream / >>>>>>> MultiByteFileStream perform a simple close() on the file which, >>>>>>> according >>>>>>> to the man page on close(), does *not* guarantee that the >>>>>>> contents have >>>>>>> all been written to file. In this case a flush is necessary >>>>>>> because the >>>>>>> entire file is immediately read again. >>>>>>> >>>>>>> Here's a smaller test case for you to play with Sven: >>>>>>> >>>>>>> ZnClient new >>>>>>> url: >>>>>>> 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>>>>> downloadTo: '/tmp/foobar.zip'. >>>>>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ >>>>>>> :s | s contents ]. >>>>>>> Transcript open; show: bytes size; cr. >>>>>>> 5 seconds asDelay wait. >>>>>>> Transcript show: ('/tmp/foobar.zip' asFileReference >>>>>>> binaryReadStreamDo: [ :s | s contents ]) size. >>>>>>> >>>>>>> The output in the Transcript should be: >>>>>>> >>>>>>> 315392 >>>>>>> 318420 >>>>>>> >>>>>>> Cheers, >>>>>>> Max >>>>>>> >>>>>>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>>>>>> >>>>>>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>>>>>> >>>>>>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>>>>>> >>>>>>> Hi Max, >>>>>>> >>>>>>> I constructed a case that fails exactly like I experience it. >>>>>>> Could you >>>>>>> try? Just unpack the attachment on a linux, set PHARO_VM env to >>>>>>> your >>>>>>> executable and execute build.sh >>>>>>> >>>>>>> I will. Might take a couple of days though. >>>>>>> >>>>>>> No problem. Iâm happy if you find time. >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Max >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>>>>>> >>>>>>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>>>>>> >>>>>>> Max, >>>>>>> >>>>>>> thanks for taking the effort. >>>>>>> >>>>>>> No worries. >>>>>>> >>>>>>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>>>>>> >>>>>>> I did my best to reproduce the issue but didn't have any luck. I >>>>>>> really >>>>>>> need access to a Linux VM to debug this. So I'm praying that >>>>>>> Apple fixes >>>>>>> the access restrictions to kernel extensions in their next >>>>>>> beta... >>>>>>> >>>>>>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so >>>>>>> there is >>>>>>> indeed a chance that something goes wrong during download of the >>>>>>> zip >>>>>>> archive and that something may be tied to a difference in the >>>>>>> versions of >>>>>>> Zinc (although I still think that's unlikely). >>>>>>> >>>>>>> Yes there is potential but I donât see it. I take a fresh 6.1 >>>>>>> image and >>>>>>> load my project into. Iâm not sure but I think zinc 2.9.2 is >>>>>>> loaded rather >>>>>>> early in that process. So I wonder why it does not go wrong in >>>>>>> the first >>>>>>> phase. And also not if I load the test group within the first >>>>>>> phase. >>>>>>> It must be either the second Metacello invocation or the >>>>>>> stopping, copying >>>>>>> and starting of the image. >>>>>>> I try to isolate this case more and provide a script that goes >>>>>>> wrong on my >>>>>>> machine. But it will take some time because I needed to stop >>>>>>> trying to >>>>>>> solve this as I wasted nearly two days on that already. >>>>>>> >>>>>>> Let me know once you have something and I'll try to help out. >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Max >>>>>>> >>>>>>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>>>>>> >>>>>>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>>>>>> >>>>>>> I checked the Parasol Archive and it does not appear to be in >>>>>>> Zip64 format >>>>>>> (Metacello uses ZipArchive which can't cope with Zip64 but >>>>>>> ZipArchive can >>>>>>> read the Parasol zip). So my next guess is that there's either a >>>>>>> problem >>>>>>> with Metacello or Pharo in the way that ZipArchive is used (e.g. >>>>>>> endianness >>>>>>> problem or non-binary stream data). It would therefore be >>>>>>> helpful to find >>>>>>> out what happens in ZipArchive>>readFrom:, i.e. what kind of >>>>>>> stream is >>>>>>> passed / opened, is it binary, does the file exist and is it >>>>>>> still the >>>>>>> correct file. >>>>>>> >>>>>>> I couldnât see anything obvious. The file in the debug message >>>>>>> exists, it >>>>>>> is a readable zip file. The way Metacello uses it it is a >>>>>>> StandardFileStream. It switches it to binary in the code. >>>>>>> But the only difference between a working and non-working build >>>>>>> is the >>>>>>> upgrade to zinc 2.9.2. >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> I'd debug it myself but I can't run VirtualBox at the moment >>>>>>> because I'm >>>>>>> on the macOS beta and it won't start... >>>>>>> >>>>>>> Max >>>>>>> >>>>>>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>>>>>> >>>>>>> Hi Max, >>>>>>> >>>>>>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>>>>>> >>>>>>> Hi Norbert, >>>>>>> >>>>>>> This is a bit of a guess, but it's possible that the archive >>>>>>> that is >>>>>>> downloaded from github is in Zip64 format and that the libraries >>>>>>> for >>>>>>> extracting Zip64 are missing on your Linux. That would of course >>>>>>> contradict >>>>>>> the experience that the same operation appears to work in 6.1. >>>>>>> >>>>>>> to be honest I donât know what Zip64 format is. I thought the >>>>>>> Zip classes >>>>>>> are pure smalltalk for unpacking. >>>>>>> >>>>>>> Try extracting the archive manually on your Linux machine with >>>>>>> the same >>>>>>> method that Metacello uses (I assume, Metacello uses the >>>>>>> ZipPlugin, which >>>>>>> will probably use the system libraries). >>>>>>> >>>>>>> I have no ZipPlugin as library in any of my vms. >>>>>>> >>>>>>> But there are zips downloaded and unpacked. I start the image >>>>>>> the first >>>>>>> time loading all the code of my project. Then it is saved, >>>>>>> copied to a new >>>>>>> name and reopened to load the tests and then it fails. I did try >>>>>>> to load >>>>>>> the tests in the first run and then it works. >>>>>>> >>>>>>> Iâm running out of ideas >>>>>>> >>>>>>> thanks, >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Cheers, >>>>>>> Max >>>>>>> >>>>>>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>>>>>> >>>>>>> With the help of Esteban I got one step further. If I do >>>>>>> >>>>>>> MCWorkingCopy >>>>>>> managersForClass: ZnFileSystemUtils >>>>>>> do: [ :each | each ancestry initialize ] >>>>>>> >>>>>>> before loading my project it updates Zinc-FileSystem as well. >>>>>>> Sadly it >>>>>>> still does not work for me because I get >>>>>>> >>>>>>> Loading baseline of BaselineOfMobilityMap... >>>>>>> ...RETRY->BaselineOfParasol >>>>>>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>>>>>> >>>>>>> and I donât know why. zip file is downloaded from github and >>>>>>> present but >>>>>>> it fails and only on my jenkins on linux. On my Mac this works. >>>>>>> >>>>>>> I try a few things but then Iâm back on pharo6.1 for the time >>>>>>> being :( >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>>>>>> >>>>>>> Really? I thought the same but then I didnât believe it works >>>>>>> like that. >>>>>>> >>>>>>> Anyway this would be very easy to solve. We just need to ask >>>>>>> Sven if he is >>>>>>> fine with doing an empty .16 version for Zinc-FileSystem and >>>>>>> does an >>>>>>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully >>>>>>> convinced >>>>>>> that will solve it but the cost wonât be very high. >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot >>>>>>> cyril.ferlicot(a)gmail.com: >>>>>>> >>>>>>> Need to investigate more. There are two .15 versions but there >>>>>>> is 1 year >>>>>>> in between (if you didnât notice TheIntegratior.15 is from >>>>>>> 2017). Just want >>>>>>> to have more context information because I can only see that >>>>>>> this is >>>>>>> strange but lacking insight. >>>>>>> >>>>>>> Iâm trying to figure out why it does not update >>>>>>> Zinc-FileSystem. No matter >>>>>>> what I do I cannot get Metacello to load the newer package. That >>>>>>> would fix >>>>>>> the issue because Iâm loading 2.9.2 which should have >>>>>>> Zinc-FileSystem-SVC.15 and not stay on the one included in the >>>>>>> image. >>>>>>> >>>>>>> I think this is important for everyone that has a product based >>>>>>> on 6.1 and >>>>>>> that want to migrate someday to pharo7. This way it is >>>>>>> impossible to do it >>>>>>> step by step. >>>>>>> >>>>>>> If there is a package .15 in the image and a package .15 in the >>>>>>> repo, I >>>>>>> think Metacello will not update because it rely on the numbers >>>>>>> to know when >>>>>>> to update. If it find a .15 and currently have a .15 I think >>>>>>> Metacello will >>>>>>> think they are at the same version. >>>>>>> >>>>>>> Norbert >>>>>>> >>>>>>> -- >>>>>>> Cyril Ferlicot >>>>>>> https://ferlicot.fr >>>>>>> >>>>>>> >>>>> >>>>> >>>>> >>>

July 21, 2018

Re: [Pharo-dev] [PROPOSAL] Put the WorldMenu in a Menu bar

by Sean P. DeNigris

Sean P. DeNigris wrote > If there's a "preserve the status quo" option (e.g. World Menu behavior > unchanged and no toolbar), I'm happy! Is there still no way to get the old World Menu back in Pharo 7? Am I the only one that wants this? On the positive side, I had to finally learn and use the tool opening shortcuts. My understanding of general experimental-new-feature etiquette is that whoever broke something should fix itâ¦ ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

July 21, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Max Leske

Just to be safe, I'll redo my experiment and get back to you. On 21 Jul 2018, at 16:59, Sven Van Caekenberghe wrote: > I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), > > ZnClient new > url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; > downloadTo: '/tmp/foobar.zip'. > bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | > s contents ]. > Transcript open; show: bytes size; cr. > 5 seconds asDelay wait. > Transcript show: ('/tmp/foobar.zip' asFileReference > binaryReadStreamDo: [ :s | s contents ]) size. > > And it gives me, > > 318420 > 318420 > > For me there is nothing nothing to see here ... and I have the > impression we are all talking about different things. > >> On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: >> >> On 21 Jul 2018, at 12:07, Norbert Hartl wrote: >> >>> Max, >>> >>> what do you think will be a proper fix for this? I cannot see if >>> this is a vm problem or not. >> >> In my opinion it should be fixed in Zinc as there's no way for the >> user of Zinc to flush the stream. The alternative would be for Zinc >> to expose a method or setting that allows the user to specify the >> expected behaviour. >> I do not think that this should be handled by the VM plugin or the >> stream class, as there are indeed different behaviours that make >> sense. >> >> Cheers, >> Max >> >>> >>> Norbert >>> >>>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>>> >>>> Hi Nicolas, >>>> >>>> The PR you are referring [1] to concerns a missing fflush() when >>>> truncating a file. I believe that is a different case to this one, >>>> as a) the file is not truncated and b) in the case of truncation it >>>> is an explicit requirement that the file be empty before the first >>>> write happens. In our case, and in general, it depends on the >>>> situation whether flushing must occur immediately, e.g. because the >>>> file will be read again immediately, or not, e.g. when writing a >>>> data to a backup file. >>>> >>>> Thanks for the pointer though. >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>> >>>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>>> >>>>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>>>> >>>>> Isn't there a recent PR on opensmalltalk that address just this? >>>>> >>>>>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a >>>>>> Ã©crit : >>>>>> >>>>>> Hi Norbert, >>>>>> >>>>>> I was able to reproduce the problem and then identify the >>>>>> culprit, >>>>>> although what I don't yet understand is how this is related to >>>>>> the changes >>>>>> in Zinc. >>>>>> >>>>>> Anyway, the problem is that the file stream isn't properly >>>>>> flushed in >>>>>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as >>>>>> the last >>>>>> statement fixes your problem. Apparently, StandardFileStream / >>>>>> MultiByteFileStream perform a simple close() on the file which, >>>>>> according >>>>>> to the man page on close(), does *not* guarantee that the >>>>>> contents have >>>>>> all been written to file. In this case a flush is necessary >>>>>> because the >>>>>> entire file is immediately read again. >>>>>> >>>>>> Here's a smaller test case for you to play with Sven: >>>>>> >>>>>> ZnClient new >>>>>> url: >>>>>> 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>>>> downloadTo: '/tmp/foobar.zip'. >>>>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ >>>>>> :s | s contents ]. >>>>>> Transcript open; show: bytes size; cr. >>>>>> 5 seconds asDelay wait. >>>>>> Transcript show: ('/tmp/foobar.zip' asFileReference >>>>>> binaryReadStreamDo: [ :s | s contents ]) size. >>>>>> >>>>>> The output in the Transcript should be: >>>>>> >>>>>> 315392 >>>>>> 318420 >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>>>>> >>>>>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>>>>> >>>>>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>>>>> >>>>>> Hi Max, >>>>>> >>>>>> I constructed a case that fails exactly like I experience it. >>>>>> Could you >>>>>> try? Just unpack the attachment on a linux, set PHARO_VM env to >>>>>> your >>>>>> executable and execute build.sh >>>>>> >>>>>> I will. Might take a couple of days though. >>>>>> >>>>>> No problem. Iâm happy if you find time. >>>>>> >>>>>> Norbert >>>>>> >>>>>> Max >>>>>> >>>>>> Norbert >>>>>> >>>>>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>>>>> >>>>>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>>>>> >>>>>> Max, >>>>>> >>>>>> thanks for taking the effort. >>>>>> >>>>>> No worries. >>>>>> >>>>>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>>>>> >>>>>> I did my best to reproduce the issue but didn't have any luck. I >>>>>> really >>>>>> need access to a Linux VM to debug this. So I'm praying that >>>>>> Apple fixes >>>>>> the access restrictions to kernel extensions in their next >>>>>> beta... >>>>>> >>>>>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so >>>>>> there is >>>>>> indeed a chance that something goes wrong during download of the >>>>>> zip >>>>>> archive and that something may be tied to a difference in the >>>>>> versions of >>>>>> Zinc (although I still think that's unlikely). >>>>>> >>>>>> Yes there is potential but I donât see it. I take a fresh 6.1 >>>>>> image and >>>>>> load my project into. Iâm not sure but I think zinc 2.9.2 is >>>>>> loaded rather >>>>>> early in that process. So I wonder why it does not go wrong in >>>>>> the first >>>>>> phase. And also not if I load the test group within the first >>>>>> phase. >>>>>> It must be either the second Metacello invocation or the >>>>>> stopping, copying >>>>>> and starting of the image. >>>>>> I try to isolate this case more and provide a script that goes >>>>>> wrong on my >>>>>> machine. But it will take some time because I needed to stop >>>>>> trying to >>>>>> solve this as I wasted nearly two days on that already. >>>>>> >>>>>> Let me know once you have something and I'll try to help out. >>>>>> >>>>>> Norbert >>>>>> >>>>>> Max >>>>>> >>>>>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>>>>> >>>>>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>>>>> >>>>>> I checked the Parasol Archive and it does not appear to be in >>>>>> Zip64 format >>>>>> (Metacello uses ZipArchive which can't cope with Zip64 but >>>>>> ZipArchive can >>>>>> read the Parasol zip). So my next guess is that there's either a >>>>>> problem >>>>>> with Metacello or Pharo in the way that ZipArchive is used (e.g. >>>>>> endianness >>>>>> problem or non-binary stream data). It would therefore be helpful >>>>>> to find >>>>>> out what happens in ZipArchive>>readFrom:, i.e. what kind of >>>>>> stream is >>>>>> passed / opened, is it binary, does the file exist and is it >>>>>> still the >>>>>> correct file. >>>>>> >>>>>> I couldnât see anything obvious. The file in the debug message >>>>>> exists, it >>>>>> is a readable zip file. The way Metacello uses it it is a >>>>>> StandardFileStream. It switches it to binary in the code. >>>>>> But the only difference between a working and non-working build >>>>>> is the >>>>>> upgrade to zinc 2.9.2. >>>>>> >>>>>> Norbert >>>>>> >>>>>> I'd debug it myself but I can't run VirtualBox at the moment >>>>>> because I'm >>>>>> on the macOS beta and it won't start... >>>>>> >>>>>> Max >>>>>> >>>>>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>>>>> >>>>>> Hi Max, >>>>>> >>>>>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>>>>> >>>>>> Hi Norbert, >>>>>> >>>>>> This is a bit of a guess, but it's possible that the archive that >>>>>> is >>>>>> downloaded from github is in Zip64 format and that the libraries >>>>>> for >>>>>> extracting Zip64 are missing on your Linux. That would of course >>>>>> contradict >>>>>> the experience that the same operation appears to work in 6.1. >>>>>> >>>>>> to be honest I donât know what Zip64 format is. I thought the >>>>>> Zip classes >>>>>> are pure smalltalk for unpacking. >>>>>> >>>>>> Try extracting the archive manually on your Linux machine with >>>>>> the same >>>>>> method that Metacello uses (I assume, Metacello uses the >>>>>> ZipPlugin, which >>>>>> will probably use the system libraries). >>>>>> >>>>>> I have no ZipPlugin as library in any of my vms. >>>>>> >>>>>> But there are zips downloaded and unpacked. I start the image the >>>>>> first >>>>>> time loading all the code of my project. Then it is saved, copied >>>>>> to a new >>>>>> name and reopened to load the tests and then it fails. I did try >>>>>> to load >>>>>> the tests in the first run and then it works. >>>>>> >>>>>> Iâm running out of ideas >>>>>> >>>>>> thanks, >>>>>> >>>>>> Norbert >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>>>>> >>>>>> With the help of Esteban I got one step further. If I do >>>>>> >>>>>> MCWorkingCopy >>>>>> managersForClass: ZnFileSystemUtils >>>>>> do: [ :each | each ancestry initialize ] >>>>>> >>>>>> before loading my project it updates Zinc-FileSystem as well. >>>>>> Sadly it >>>>>> still does not work for me because I get >>>>>> >>>>>> Loading baseline of BaselineOfMobilityMap... >>>>>> ...RETRY->BaselineOfParasol >>>>>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>>>>> >>>>>> and I donât know why. zip file is downloaded from github and >>>>>> present but >>>>>> it fails and only on my jenkins on linux. On my Mac this works. >>>>>> >>>>>> I try a few things but then Iâm back on pharo6.1 for the time >>>>>> being :( >>>>>> >>>>>> Norbert >>>>>> >>>>>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>>>>> >>>>>> Really? I thought the same but then I didnât believe it works >>>>>> like that. >>>>>> >>>>>> Anyway this would be very easy to solve. We just need to ask Sven >>>>>> if he is >>>>>> fine with doing an empty .16 version for Zinc-FileSystem and does >>>>>> an >>>>>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully >>>>>> convinced >>>>>> that will solve it but the cost wonât be very high. >>>>>> >>>>>> Norbert >>>>>> >>>>>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot >>>>>> cyril.ferlicot(a)gmail.com: >>>>>> >>>>>> Need to investigate more. There are two .15 versions but there is >>>>>> 1 year >>>>>> in between (if you didnât notice TheIntegratior.15 is from >>>>>> 2017). Just want >>>>>> to have more context information because I can only see that this >>>>>> is >>>>>> strange but lacking insight. >>>>>> >>>>>> Iâm trying to figure out why it does not update >>>>>> Zinc-FileSystem. No matter >>>>>> what I do I cannot get Metacello to load the newer package. That >>>>>> would fix >>>>>> the issue because Iâm loading 2.9.2 which should have >>>>>> Zinc-FileSystem-SVC.15 and not stay on the one included in the >>>>>> image. >>>>>> >>>>>> I think this is important for everyone that has a product based >>>>>> on 6.1 and >>>>>> that want to migrate someday to pharo7. This way it is impossible >>>>>> to do it >>>>>> step by step. >>>>>> >>>>>> If there is a package .15 in the image and a package .15 in the >>>>>> repo, I >>>>>> think Metacello will not update because it rely on the numbers to >>>>>> know when >>>>>> to update. If it find a .15 and currently have a .15 I think >>>>>> Metacello will >>>>>> think they are at the same version. >>>>>> >>>>>> Norbert >>>>>> >>>>>> -- >>>>>> Cyril Ferlicot >>>>>> https://ferlicot.fr >>>>>> >>>>>> >>>> >>>> >>>> >>

July 21, 2018

Re: [Pharo-dev] Problem with zinc 2.9.2

by Sven Van Caekenberghe

I tried running Max's snippet (Pharo 6.1 on Ubuntu 16.04 LTS), ZnClient new url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; downloadTo: '/tmp/foobar.zip'. bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. Transcript open; show: bytes size; cr. 5 seconds asDelay wait. Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. And it gives me, 318420 318420 For me there is nothing nothing to see here ... and I have the impression we are all talking about different things. > On 21 Jul 2018, at 14:18, Max Leske <maxleske(a)gmail.com> wrote: > > On 21 Jul 2018, at 12:07, Norbert Hartl wrote: > >> Max, >> >> what do you think will be a proper fix for this? I cannot see if this is a vm problem or not. > > In my opinion it should be fixed in Zinc as there's no way for the user of Zinc to flush the stream. The alternative would be for Zinc to expose a method or setting that allows the user to specify the expected behaviour. > I do not think that this should be handled by the VM plugin or the stream class, as there are indeed different behaviours that make sense. > > Cheers, > Max > >> >> Norbert >> >>> Am 14.07.2018 um 08:25 schrieb Max Leske <maxleske(a)gmail.com>: >>> >>> Hi Nicolas, >>> >>> The PR you are referring [1] to concerns a missing fflush() when truncating a file. I believe that is a different case to this one, as a) the file is not truncated and b) in the case of truncation it is an explicit requirement that the file be empty before the first write happens. In our case, and in general, it depends on the situation whether flushing must occur immediately, e.g. because the file will be read again immediately, or not, e.g. when writing a data to a backup file. >>> >>> Thanks for the pointer though. >>> >>> >>> Cheers, >>> Max >>> >>> >>> [1] https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/271 >>> >>>> On 12 Jul 2018, at 23:19, Nicolas Cellier wrote: >>>> >>>> Isn't there a recent PR on opensmalltalk that address just this? >>>> >>>>> Le jeu. 12 juil. 2018 Ã 23:16, Max Leske <maxleske(a)gmail.com> a Ã©crit : >>>>> >>>>> Hi Norbert, >>>>> >>>>> I was able to reproduce the problem and then identify the culprit, >>>>> although what I don't yet understand is how this is related to the changes >>>>> in Zinc. >>>>> >>>>> Anyway, the problem is that the file stream isn't properly flushed in >>>>> ZnUtils class>>streamFrom:to:size:. Adding outputStream flush as the last >>>>> statement fixes your problem. Apparently, StandardFileStream / >>>>> MultiByteFileStream perform a simple close() on the file which, according >>>>> to the man page on close(), does *not* guarantee that the contents have >>>>> all been written to file. In this case a flush is necessary because the >>>>> entire file is immediately read again. >>>>> >>>>> Here's a smaller test case for you to play with Sven: >>>>> >>>>> ZnClient new >>>>> url: 'https://github.com/zweidenker/Parasol/archive/master.zip'; >>>>> downloadTo: '/tmp/foobar.zip'. >>>>> bytes := '/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]. >>>>> Transcript open; show: bytes size; cr. >>>>> 5 seconds asDelay wait. >>>>> Transcript show: ('/tmp/foobar.zip' asFileReference binaryReadStreamDo: [ :s | s contents ]) size. >>>>> >>>>> The output in the Transcript should be: >>>>> >>>>> 315392 >>>>> 318420 >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On 12 Jul 2018, at 8:17, Norbert Hartl wrote: >>>>> >>>>> Am 12.07.2018 um 08:05 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> On 11 Jul 2018, at 22:44, Norbert Hartl wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> I constructed a case that fails exactly like I experience it. Could you >>>>> try? Just unpack the attachment on a linux, set PHARO_VM env to your >>>>> executable and execute build.sh >>>>> >>>>> I will. Might take a couple of days though. >>>>> >>>>> No problem. Iâm happy if you find time. >>>>> >>>>> Norbert >>>>> >>>>> Max >>>>> >>>>> Norbert >>>>> >>>>> Am 10.07.2018 um 09:17 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> On 10 Jul 2018, at 8:48, Norbert Hartl wrote: >>>>> >>>>> Max, >>>>> >>>>> thanks for taking the effort. >>>>> >>>>> No worries. >>>>> >>>>> Am 10.07.2018 um 08:37 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> I did my best to reproduce the issue but didn't have any luck. I really >>>>> need access to a Linux VM to debug this. So I'm praying that Apple fixes >>>>> the access restrictions to kernel extensions in their next beta... >>>>> >>>>> BTW, Metacello uses ZnClient, which uses ZnFileSystemUtils, so there is >>>>> indeed a chance that something goes wrong during download of the zip >>>>> archive and that something may be tied to a difference in the versions of >>>>> Zinc (although I still think that's unlikely). >>>>> >>>>> Yes there is potential but I donât see it. I take a fresh 6.1 image and >>>>> load my project into. Iâm not sure but I think zinc 2.9.2 is loaded rather >>>>> early in that process. So I wonder why it does not go wrong in the first >>>>> phase. And also not if I load the test group within the first phase. >>>>> It must be either the second Metacello invocation or the stopping, copying >>>>> and starting of the image. >>>>> I try to isolate this case more and provide a script that goes wrong on my >>>>> machine. But it will take some time because I needed to stop trying to >>>>> solve this as I wasted nearly two days on that already. >>>>> >>>>> Let me know once you have something and I'll try to help out. >>>>> >>>>> Norbert >>>>> >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 19:43, Norbert Hartl wrote: >>>>> >>>>> Am 09.07.2018 um 19:10 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> I checked the Parasol Archive and it does not appear to be in Zip64 format >>>>> (Metacello uses ZipArchive which can't cope with Zip64 but ZipArchive can >>>>> read the Parasol zip). So my next guess is that there's either a problem >>>>> with Metacello or Pharo in the way that ZipArchive is used (e.g. endianness >>>>> problem or non-binary stream data). It would therefore be helpful to find >>>>> out what happens in ZipArchive>>readFrom:, i.e. what kind of stream is >>>>> passed / opened, is it binary, does the file exist and is it still the >>>>> correct file. >>>>> >>>>> I couldnât see anything obvious. The file in the debug message exists, it >>>>> is a readable zip file. The way Metacello uses it it is a >>>>> StandardFileStream. It switches it to binary in the code. >>>>> But the only difference between a working and non-working build is the >>>>> upgrade to zinc 2.9.2. >>>>> >>>>> Norbert >>>>> >>>>> I'd debug it myself but I can't run VirtualBox at the moment because I'm >>>>> on the macOS beta and it won't start... >>>>> >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 18:31, Norbert Hartl wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> Am 09.07.2018 um 18:18 schrieb Max Leske maxleske(a)gmail.com: >>>>> >>>>> Hi Norbert, >>>>> >>>>> This is a bit of a guess, but it's possible that the archive that is >>>>> downloaded from github is in Zip64 format and that the libraries for >>>>> extracting Zip64 are missing on your Linux. That would of course contradict >>>>> the experience that the same operation appears to work in 6.1. >>>>> >>>>> to be honest I donât know what Zip64 format is. I thought the Zip classes >>>>> are pure smalltalk for unpacking. >>>>> >>>>> Try extracting the archive manually on your Linux machine with the same >>>>> method that Metacello uses (I assume, Metacello uses the ZipPlugin, which >>>>> will probably use the system libraries). >>>>> >>>>> I have no ZipPlugin as library in any of my vms. >>>>> >>>>> But there are zips downloaded and unpacked. I start the image the first >>>>> time loading all the code of my project. Then it is saved, copied to a new >>>>> name and reopened to load the tests and then it fails. I did try to load >>>>> the tests in the first run and then it works. >>>>> >>>>> Iâm running out of ideas >>>>> >>>>> thanks, >>>>> >>>>> Norbert >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>> On 9 Jul 2018, at 17:14, Norbert Hartl wrote: >>>>> >>>>> With the help of Esteban I got one step further. If I do >>>>> >>>>> MCWorkingCopy >>>>> managersForClass: ZnFileSystemUtils >>>>> do: [ :each | each ancestry initialize ] >>>>> >>>>> before loading my project it updates Zinc-FileSystem as well. Sadly it >>>>> still does not work for me because I get >>>>> >>>>> Loading baseline of BaselineOfMobilityMap... >>>>> ...RETRY->BaselineOfParasol >>>>> ...RETRY->BaselineOfParasol[31mError: can't find EOCD position >>>>> >>>>> and I donât know why. zip file is downloaded from github and present but >>>>> it fails and only on my jenkins on linux. On my Mac this works. >>>>> >>>>> I try a few things but then Iâm back on pharo6.1 for the time being :( >>>>> >>>>> Norbert >>>>> >>>>> Am 07.07.2018 um 13:28 schrieb Norbert Hartl norbert(a)hartl.name: >>>>> >>>>> Really? I thought the same but then I didnât believe it works like that. >>>>> >>>>> Anyway this would be very easy to solve. We just need to ask Sven if he is >>>>> fine with doing an empty .16 version for Zinc-FileSystem and does an >>>>> in-place version reset of 2.9.2 or a new 2.9.3. Iâm not fully convinced >>>>> that will solve it but the cost wonât be very high. >>>>> >>>>> Norbert >>>>> >>>>> Am 07.07.2018 um 13:22 schrieb Cyril Ferlicot cyril.ferlicot(a)gmail.com: >>>>> >>>>> Need to investigate more. There are two .15 versions but there is 1 year >>>>> in between (if you didnât notice TheIntegratior.15 is from 2017). Just want >>>>> to have more context information because I can only see that this is >>>>> strange but lacking insight. >>>>> >>>>> Iâm trying to figure out why it does not update Zinc-FileSystem. No matter >>>>> what I do I cannot get Metacello to load the newer package. That would fix >>>>> the issue because Iâm loading 2.9.2 which should have >>>>> Zinc-FileSystem-SVC.15 and not stay on the one included in the image. >>>>> >>>>> I think this is important for everyone that has a product based on 6.1 and >>>>> that want to migrate someday to pharo7. This way it is impossible to do it >>>>> step by step. >>>>> >>>>> If there is a package .15 in the image and a package .15 in the repo, I >>>>> think Metacello will not update because it rely on the numbers to know when >>>>> to update. If it find a .15 and currently have a .15 I think Metacello will >>>>> think they are at the same version. >>>>> >>>>> Norbert >>>>> >>>>> -- >>>>> Cyril Ferlicot >>>>> https://ferlicot.fr >>>>> >>>>> >>> >>> >>> >

July 21, 2018

Jump to page: