- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] FileHandle creates a buffered stream on a buffered stream

by Sven Van Caekenberghe

> On 16 Mar 2018, at 11:46, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > Nice catch. The write stream is well defined in comparison. > > https://pharo.fogbugz.com/f/cases/21604/FileHandle-creates-a-buffered-strea… > > https://github.com/pharo-project/pharo/pull/1118 > > Can somebody review please? OK by me > Tx, > Guille > > On Thu, Mar 15, 2018 at 4:00 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Hi (Guille), > > FileHandle creates a buffered stream on a buffered stream when sent #readStream. > > Note how #binaryReadStream adds a bufferer stream as well as #readStreamEncoded: > > Probably the latter should not do this > > Sven > > > > -- > > Guille Polito > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr > > Web: http://guillep.github.io > Phone: +33 06 52 70 66 13

March 16, 2018

Windows PharoLauncher cannot create "C:\Program Files (x86)\Pharo\Pharo.image"

by Ben Coman

I popped over to Windows for a moment to try to make my CloudflareUn crossplatform, and installed PharoLauncher from the downloads page... http://files.pharo.org/pharo-launcher/1.1/pharo-launcher-installer-1.1-x86.… I logged in as an Admin to do this, but when I open it I get an error... "PrimitiveFailed: primitive #createDirectory: in WindowsStore failed" where... path ==> "Path / 'C:' / 'Program Files (x86)' / 'Pharo' / 'pharo-local'" encodedPath ==> C:\Program Files (x86)\Pharo\pharo-local (self isDirectory: parent) ==> true in... WindowsStore(DiskStore) >> createDirectory: path | parent encodedPathString pathString result | pathString := self stringFromPath: path. encodedPathString := Primitives encode: pathString. result := Primitives createDirectory: encodedPathString. result ifNil: [ parent := path parent. (self exists: path) ifTrue: [ (self isFile: path) ifTrue: [ self signalFileExists: path ] ifFalse: [ self signalDirectoryExists: path ] ]. (self isDirectory: parent) ifFalse: [ ^ self signalDirectoryDoesNotExist: parent ]. self primitiveFailed ]. ^ self A few calls down the stack is OmSessioNStore>>resetWithStoreNamed: Does this happen for anyone else? Version Info... OS ----- Windows 10 Image ----- C:\Program Files (x86)\Pharo\Pharo.image Pharo6.0 Latest update: #60528 Unnamed Virtual Machine --------------- C:\Program Files (x86)\Pharo\Pharo.exe CoInterpreter VMMaker.oscog-eem.2254 uuid: 4f2c2cce-f4a2-469a-93f1-97ed941df0ad Jul 20 2017 StackToRegisterMappingCogit VMMaker.oscog-eem.2252 uuid: 2f3e9b0e-ecd3-4adf-b092-cce2e2587a5c Jul 20 2017 VM: 201707201942 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Thu Jul 20 12:42:21 2017 -0700 $ Plugins: 201707201942 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Win32 built on Jul 20 2017 20:21:26 GMT Compiler: 5.4.0 VMMaker versionString VM: 201707201942 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ Date: Thu Jul 20 12:42:21 2017 -0700 $ Plugins: 201707201942 https://github.com/OpenSmalltalk/opensmalltalk-vm.git $ CoInterpreter VMMaker.oscog-eem.2254 uuid: 4f2c2cce-f4a2-469a-93f1-97ed941df0ad Jul 20 2017 StackToRegisterMappingCogit VMMaker.oscog-eem.2252 uuid: 2f3e9b0e-ecd3-4adf-b092-cce2e2587a5c Jul 20 2017 Also, I would expect trouble for Non-Admin users trying to use the Admin installed Pharo Launcher.trying to write to "pharo-local" under "Program Files". Does PharoLauncher have its own CI testing to ensure it can open okay and run an image? I'm glad to see it promoted in importance but with this it probably needs such CI testing of its own separate to the usual Pharo CI testing. On Windows such CI testing *should* run as a Non-Admin user. btw, requiring Administrative privileges to install PharoLauncher is going to lock out people who don't have administrative control over their PCs e.g. University student labs. Wanting to run a quick demo for someone in a corporate office. Why don't we make it the same as OSX and Linux and be usable without Admin privileges. That would lower a barrier to entry. cheers -ben

March 16, 2018

Re: [Pharo-dev] Pharo 7 new file stream encoders

by Sven Van Caekenberghe

> On 16 Mar 2018, at 11:29, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > Hi, > > Yes, I'll provide a PR with that functionality soon. Probably in that case *ZnCrPortableWriteStream* is not the right name for that class. Sven, do you think renaming it now (before it has lots of users) would be acceptable towards backwards compatibility? Otherwise I would name it ZnNewLineConverterStream... Renaming is OK/better, if it works (critical class in IO) Note #forPlatformLineEnding should do: OSPlatform current lineEnding > Keep you updated, > Guille > > On Thu, Mar 15, 2018 at 3:11 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > I think the idea is to use ZnCrPortableWriteStream for that. > > > On 15 Mar 2018, at 14:58, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > > > Hi all, > > > > how does one create a file stream in Pharo 7 that forces a specific > > line end convention ? For example, for forcing all end of lines to be > > Character lf. > > > > The old way, > > > > fileStream lineEndConvention: #'lf'. > > > > does not work anymore. > > > > Thanks, > > > > Thierry > > > > > > > > -- > > Guille Polito > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > CRIStAL - UMR 9189 > French National Center for Scientific Research - http://www.cnrs.fr > > Web: http://guillep.github.io > Phone: +33 06 52 70 66 13

March 16, 2018

Re: [Pharo-dev] [Vm-dev] Reproducible VM crash from commit 4749358 onwards

by Alistair Grant

Hi Eliot, On 15 March 2018 at 23:53, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Alistair, > > I can reproduce this on Mac OS too. Thanks very much for the report. Looking at it now. Thanks! No more crashes with the latest VM (Thu Mar 15 19:15:05 2018 -0700) (and the correct FileAttributesPlugin version :-)). Cheers, Alistair > On Thu, Mar 15, 2018 at 2:43 PM, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> >> Hi Everyone, >> >> If I download a recent Pharo 64 bit image, e.g.: >> >> http://files.pharo.org/image/70/latest-64.zip >> >> And use a Pharo VM from commit: >> >> commit 4749358b5717b9a2f7e1d44f9baac48ed5d80431 >> Author: Eliot Miranda <eliot.miranda(a)gmail.com> >> Date: Wed Mar 7 15:11:21 2018 -0800 >> >> or later, the following will reliably crash the VM: >> >> >> ./pharo Pharo7.0-64bit-a26248b.image eval "OCBytecodeGeneratorTest >> run: #testExample2" >> >> >> Earlier VMs don't crash. >> >> Stack dump below my signature. >> >> I'm a bit out of my depth beyond that... >> >> OS: Ubuntu 16.04 64 bit >> >> ...

March 16, 2018

Re: [Pharo-dev] FileHandle creates a buffered stream on a buffered stream

by Guillermo Polito

Nice catch. The write stream is well defined in comparison. https://pharo.fogbugz.com/f/cases/21604/FileHandle-creates-a-buffered-strea… https://github.com/pharo-project/pharo/pull/1118 Can somebody review please? Tx, Guille On Thu, Mar 15, 2018 at 4:00 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Hi (Guille), > > FileHandle creates a buffered stream on a buffered stream when sent > #readStream. > > Note how #binaryReadStream adds a bufferer stream as well as > #readStreamEncoded: > > Probably the latter should not do this > > Sven > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

March 16, 2018

Re: [Pharo-dev] Executive Summary of the recent FileStream Changes

by Sven Van Caekenberghe

> On 16 Mar 2018, at 07:05, monty <monty2(a)programmer.net> wrote: > >> Sent: Thursday, March 15, 2018 at 4:01 PM >> From: "Sven Van Caekenberghe" <sven(a)stfx.eu> >> To: "Pharo Development List" <pharo-dev(a)lists.pharo.org> >> Subject: [Pharo-dev] Executive Summary of the recent FileStream Changes >> >> Executive Summary of the recent FileStream Changes >> >> In Pharo 7 Guille Polito recently committed a heroic set of changes that we were planning to do for a long time but were afraid to take on. >> >> The idea is to replace a couple of fat, overly complex, multi-functional, do-all classes with a set of simpler single purpose classes that can be combined as needed. >> >> The classes that we want to get rid of can be found in the package DeprecatedFileSystem, in particular FileStream, StandardFileStream, MultiByteFileStream, MultiByteBinaryOrTextStream and RWBinaryOrTextStream. > > StandardFileStream, at least, should remain for backwards compatibility and cross-platform compatibility with Squeak. It's a no-frills, non-decoding, non-LE normalizing stream that is heavily depended on. Hmm, maybe. The standard (no pun intended) interface to the file system in Pharo has been FileSystem (FileReference) for quite a while. Many packages dealing with either different Pharo versions or different Smalltalk implementations have constructed their own portability facade (heck, I even did it in ZnFileSystemUtils myself). Note however that some aspects (API, behaviour) about the streams themselves changed as well (no longer being bivalent, separating reading/writing, smaller/simpler API, sometimes no positioning). >> The replacements are can be found in packages Files and Zinc-Character-Encoding-Core. >> >> Encoding and decoding characters to and from bytes is done using classes that you wrap around a more primitive binary stream. The same goes for buffering or translating line endings. >> >> For example, >> >> '/Users/sven/Desktop/foo.txt' asFileReference binaryReadStream. >> >> gives you a ZnBufferedWriteStream wrapping a BinaryWriteStream. >> >> While, >> >> '/Users/sven/Desktop/foo.txt' asFileReference readStream. > > What do you think about this algorithm for encoding detection: http://www.yaml.org/spec/1.2/spec.html#id2771184 > > I have an implementation (with tests), if you're interested. (I was waiting to propose it until the FileSystem API switched over to using Zn streams and encoders. The TextConverter API doesn't support UTF-32.) I did a primitive one in ZnCharacterEncoding class>>#detectEncoding: but I am not happy with it. I will read your reference and I am certainly interested in seeing your code ! >> gives a ZnCharacterReadStream wrapping a ZnBufferedWriteStream wrapping a BinaryWriteStream. >> >> To translate line endings, we would wrap a ZnCharacterWriteStream using a ZnCrPortableWriteStream. >> >> There are a couple of more specialised streams to cover special cases (like read and writing at the same time). >> >> SocketStream remains another fat, overly complex, multi-functional, do-all class, for which usable replacements exist in the form of ZdcSocketStream and ZdcSecureSocketStream, which are simpler, cleaner and binary only. >> >> Of course, switching is more than replacing one class with a 100% compatible alternative, that would give us the same complex result. The challenge is to use a simpler API as well, to rethink how the streams are used. You know, KISS. >> >> Of course, we are far from done and need more testing, debugging and help from as many people as possible. >> >> Sven >> >> >> -- >> Sven Van Caekenberghe >> Proudly supporting Pharo >> http://pharo.org >> http://association.pharo.org >> http://consortium.pharo.org

March 16, 2018

Re: [Pharo-dev] Pharo 7 new file stream encoders

by Guillermo Polito

Thierry, Sven, Can you review https://github.com/pharo-project/pharo/pull/1116 ? Tx, Guille On Fri, Mar 16, 2018 at 11:29 AM, Guillermo Polito < guillermopolito(a)gmail.com> wrote: > Hi, > > Yes, I'll provide a PR with that functionality soon. Probably in that case > *ZnCrPortableWriteStream* is not the right name for that class. Sven, do > you think renaming it now (before it has lots of users) would be acceptable > towards backwards compatibility? Otherwise I would name it > ZnNewLineConverterStream... > > Keep you updated, > Guille > > On Thu, Mar 15, 2018 at 3:11 PM, Sven Van Caekenberghe <sven(a)stfx.eu> > wrote: > >> I think the idea is to use ZnCrPortableWriteStream for that. >> >> > On 15 Mar 2018, at 14:58, Thierry Goubier <thierry.goubier(a)gmail.com> >> wrote: >> > >> > Hi all, >> > >> > how does one create a file stream in Pharo 7 that forces a specific >> > line end convention ? For example, for forcing all end of lines to be >> > Character lf. >> > >> > The old way, >> > >> > fileStream lineEndConvention: #'lf'. >> > >> > does not work anymore. >> > >> > Thanks, >> > >> > Thierry >> > >> >> >> > > > -- > > > > Guille Polito > > Research Engineer > > Centre de Recherche en Informatique, Signal et Automatique de Lille > > CRIStAL - UMR 9189 > > French National Center for Scientific Research - *http://www.cnrs.fr > <http://www.cnrs.fr>* > > > *Web:* *http://guillep.github.io* <http://guillep.github.io> > > *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

March 16, 2018

Re: [Pharo-dev] Executive Summary of the recent FileStream Changes

by Sven Van Caekenberghe

> On 15 Mar 2018, at 23:36, Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com> wrote: > > Very good initiative! > It's worth a few hick-ups. Thanks, the really hard work is in the all the using code, writing against simpler APIs. It will be a while before all the dust settles and all performance issues are covered, but we are already quite far. > I've tried to reduce RWBinaryOrTextStream usage in Squeak maybe 10 years ago, but you know it very well, the last places which are resisting are the more intricated and convoluted. > I call it the SwiisKnifeStream and allways wandered why we would have to carry so many subclasses of Stream, since we have one capable of almost everything! > Managing the states in such a hierarchy was an art, if ever torture is an art... and we are loosing a great can of worms for fishing the bugs. > So, R.I.P. and no regret! > > 2018-03-15 21:01 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Executive Summary of the recent FileStream Changes > > In Pharo 7 Guille Polito recently committed a heroic set of changes that we were planning to do for a long time but were afraid to take on. > > The idea is to replace a couple of fat, overly complex, multi-functional, do-all classes with a set of simpler single purpose classes that can be combined as needed. > > The classes that we want to get rid of can be found in the package DeprecatedFileSystem, in particular FileStream, StandardFileStream, MultiByteFileStream, MultiByteBinaryOrTextStream and RWBinaryOrTextStream. > > The replacements are can be found in packages Files and Zinc-Character-Encoding-Core. > > Encoding and decoding characters to and from bytes is done using classes that you wrap around a more primitive binary stream. The same goes for buffering or translating line endings. > > For example, > > '/Users/sven/Desktop/foo.txt' asFileReference binaryReadStream. > > gives you a ZnBufferedWriteStream wrapping a BinaryWriteStream. > > While, > > '/Users/sven/Desktop/foo.txt' asFileReference readStream. > > gives a ZnCharacterReadStream wrapping a ZnBufferedWriteStream wrapping a BinaryWriteStream. > > To translate line endings, we would wrap a ZnCharacterWriteStream using a ZnCrPortableWriteStream. > > There are a couple of more specialised streams to cover special cases (like read and writing at the same time). > > SocketStream remains another fat, overly complex, multi-functional, do-all class, for which usable replacements exist in the form of ZdcSocketStream and ZdcSecureSocketStream, which are simpler, cleaner and binary only. > > Of course, switching is more than replacing one class with a 100% compatible alternative, that would give us the same complex result. The challenge is to use a simpler API as well, to rethink how the streams are used. You know, KISS. > > Of course, we are far from done and need more testing, debugging and help from as many people as possible. > > Sven > > > -- > Sven Van Caekenberghe > Proudly supporting Pharo > http://pharo.org > http://association.pharo.org > http://consortium.pharo.org > > > > > >

March 16, 2018

Re: [Pharo-dev] Pharo 7 new file stream encoders

by Guillermo Polito

Hi, Yes, I'll provide a PR with that functionality soon. Probably in that case *ZnCrPortableWriteStream* is not the right name for that class. Sven, do you think renaming it now (before it has lots of users) would be acceptable towards backwards compatibility? Otherwise I would name it ZnNewLineConverterStream... Keep you updated, Guille On Thu, Mar 15, 2018 at 3:11 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > I think the idea is to use ZnCrPortableWriteStream for that. > > > On 15 Mar 2018, at 14:58, Thierry Goubier <thierry.goubier(a)gmail.com> > wrote: > > > > Hi all, > > > > how does one create a file stream in Pharo 7 that forces a specific > > line end convention ? For example, for forcing all end of lines to be > > Character lf. > > > > The old way, > > > > fileStream lineEndConvention: #'lf'. > > > > does not work anymore. > > > > Thanks, > > > > Thierry > > > > > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

March 16, 2018

Re: [Pharo-dev] Executive Summary of the recent FileStream Changes

by Stephane Ducasse

Hi Monty We will keep the old stuff around in a kind of deprecated packages. Now in the long term we do not intend to maintain them forever because we will collapse under the load. You see it was either that clean up or modules.... so at the end ***Pharo*** should move on because this is our future. In this vein, we are planning to migrate a selection of the old deprecated methods (of Pharo 30, 40, 50, 60) into a MigrationPackage with deprecatedWithTransform rule to help the migration of the old code to new one. Now of course nobody really stood up to give a hand. Normal this is easier to give us order :) Stef On Fri, Mar 16, 2018 at 7:05 AM, monty <monty2(a)programmer.net> wrote: > > >> Sent: Thursday, March 15, 2018 at 4:01 PM >> From: "Sven Van Caekenberghe" <sven(a)stfx.eu> >> To: "Pharo Development List" <pharo-dev(a)lists.pharo.org> >> Subject: [Pharo-dev] Executive Summary of the recent FileStream Changes >> >> Executive Summary of the recent FileStream Changes >> >> In Pharo 7 Guille Polito recently committed a heroic set of changes that we were planning to do for a long time but were afraid to take on. >> >> The idea is to replace a couple of fat, overly complex, multi-functional, do-all classes with a set of simpler single purpose classes that can be combined as needed. >> >> The classes that we want to get rid of can be found in the package DeprecatedFileSystem, in particular FileStream, StandardFileStream, MultiByteFileStream, MultiByteBinaryOrTextStream and RWBinaryOrTextStream. > > StandardFileStream, at least, should remain for backwards compatibility and cross-platform compatibility with Squeak. It's a no-frills, non-decoding, non-LE normalizing stream that is heavily depended on. > >> The replacements are can be found in packages Files and Zinc-Character-Encoding-Core. >> >> Encoding and decoding characters to and from bytes is done using classes that you wrap around a more primitive binary stream. The same goes for buffering or translating line endings. >> >> For example, >> >> '/Users/sven/Desktop/foo.txt' asFileReference binaryReadStream. >> >> gives you a ZnBufferedWriteStream wrapping a BinaryWriteStream. >> >> While, >> >> '/Users/sven/Desktop/foo.txt' asFileReference readStream. > > What do you think about this algorithm for encoding detection: http://www.yaml.org/spec/1.2/spec.html#id2771184 > > I have an implementation (with tests), if you're interested. (I was waiting to propose it until the FileSystem API switched over to using Zn streams and encoders. The TextConverter API doesn't support UTF-32.) > >> gives a ZnCharacterReadStream wrapping a ZnBufferedWriteStream wrapping a BinaryWriteStream. >> >> To translate line endings, we would wrap a ZnCharacterWriteStream using a ZnCrPortableWriteStream. >> >> There are a couple of more specialised streams to cover special cases (like read and writing at the same time). >> >> SocketStream remains another fat, overly complex, multi-functional, do-all class, for which usable replacements exist in the form of ZdcSocketStream and ZdcSecureSocketStream, which are simpler, cleaner and binary only. >> >> Of course, switching is more than replacing one class with a 100% compatible alternative, that would give us the same complex result. The challenge is to use a simpler API as well, to rethink how the streams are used. You know, KISS. >> >> Of course, we are far from done and need more testing, debugging and help from as many people as possible. >> >> Sven >> >> >> -- >> Sven Van Caekenberghe >> Proudly supporting Pharo >> http://pharo.org >> http://association.pharo.org >> http://consortium.pharo.org >> >> >> >> >> >> >

March 16, 2018