- Pharo-dev - lists.pharo.org

Windows FileSystem '\test\bar' not an absolute path

by Ben Coman

I have some strong concerns about the semantics of FileSystem on MS Windows treating '\directory' as a relative path rather than an absolute path. For example currently... '/tmp/test.txt' asFileReference fullName --> 'C:\Users\Ben\AppData\Roaming\Pharo\images\30790\tmp\test.txt' when it _should_ go... '/tmp/test.txt' asFileReference fullName --> 'C:\tmp\x.txt' This seems to be by design per '\test\bar' below... WindowsStoreTest>>testAbsolutePath #('c:\' 'C:\temp' 'A:\temp\test') do: [:each | self assert: (WindowsStore current pathFromString: each) isAbsolute ] WindowsStoreTest>>testRelativePath #('a' 'bin\foo' 'temp\test' '\test\bar') do: [:each | self assert: (WindowsStore current pathFromString: each) isRelative ] However taking as reference the Windows Command Shell (cmd.exe)... (1) C:\> mkdir \a\b\a\c (2) C:\> cd a (3) C:\a> cd b (4) C:\a\b> cd \a (5) C:\a> observe that after 'cd \a' (4) the working directory becomes absolute 'C:\a' rather than relative 'C:\a\b\a' . Microsoft defines [1] a file name is RELATIVE to the current directory if it DOES NOT begin with: * A single backslash, for example, "\directory" or "\file.txt". This is also referred to as an ABSOLUTE path. Indeed all Windows applications I know operate this way. Using different semantics is a bug. [1] http://msdn.microsoft.com/en-us/library/aa365247.aspx#fully_qualified_vs._r… What are your thoughts? cheers -ben

March 9, 2014

Re: [Pharo-dev] [Moose-dev] [Pharo-business] [ANN] DBPedia: Query Wikipedia from Pharo

by Pharo4Stef

Just avoiding to only manipulate strings would be already good. On 09 Mar 2014, at 11:22, Norbert Hartl <norbert(a)hartl.name> wrote: > > Am 03.03.2014 um 14:22 schrieb Johan Fabry <jfabry(a)dcc.uchile.cl>: > >> +1 to that! It would be good to have some kind of uniform syntax for constructing queries. Especially now that the use of JSON objects is quite common. >> > I would like to have that, too. I need to have query support for elasticsearch and I would do something similar like MongoQueries. The idea of having a common query syntax is nice but Iâm not convinced it will be easy to do. The parser needs to be easily extendable because most applications will have their custom additions to the base syntax. > > Norbert > >> The OP was actually referring to MongoQueries. To get some examples of the syntax see section 4 of https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/lastSucces… >> >> On Mar 3, 2014, at 10:13 AM, Alexandre Bergel <alexandre.bergel(a)me.com> wrote: >> >>>> What would be nice is to have an abstraction like mongoTalk on top to avoid to manipulate strings but to manipulate query elements. >>> >>> That is a nice idea. >>> >>> Alexandre >> >> >> >> ---> Save our in-boxes! http://emailcharter.org <--- >> >> Johan Fabry - http://pleiad.cl/~jfabry >> PLEIAD lab - Computer Science Department (DCC) - University of Chile >> >> > >

March 9, 2014

Re: [Pharo-dev] base64.b64encode(data) equivalent

by Pharo4Stef

thanks sven Iâm reading Hbase connection and Iâm trying to understand it :) On 09 Mar 2014, at 09:50, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > ZnBase64Encoder new encode: 42 asByteArray. > => 'Kg==' > > (ZnBase64Encoder new decode: 'Kg==') asInteger. > => 42 > > There is also #asByteArrayOfSize: and signed/unsigned might come into play as well. > > Note that strictly speaking (as implemented by ZnBase64Encoder), Base64 is a binary to string encoding. But is it often used string to string (as in #base64Encoded and #base64Decoded, which are still implemented using Base64MimeConverter) but the explicit correct way is as in ZnUtils #encodeBase64: and #decodeBase64: using a character encoding. > > On 09 Mar 2014, at 09:09, Pharo4Stef <pharo4Stef(a)free.fr> wrote: > >> around Zinc-Character-Encoding-Core >> :) >> >> Stef >> On 09 Mar 2014, at 09:07, Pharo4Stef <pharo4Stef(a)free.fr> wrote: >> >>> Iâm sure that we have that. I remember svenâs code using encoder >>> but I do not know where. >>> >>> # Method for encoding ints with base64 encoding >>> def encode(n): >>> data = struct.pack("i", n) >>> s = base64.b64encode(data) >>> return s >>> >> > >

March 9, 2014

Re: [Pharo-dev] Git & MS Windows path length restriction

by GOUBIER Thierry

Thanks Ben for that info about long file names in Windows, of importance when discussing file formats for smalltalk packages and git. Just that your link only describe how to get a significant diff display when dealing with zip files stored as-is inside a Git repository, not automagically zipping and unzipping in and out of git storage. Too bad :(. However, searching around that info points out that git will do a binary delta on a versionned zip file [1], which means storing zip files inside git is not that bad. [1] http://stackoverflow.com/questions/9973151/does-git-smartly-handle-a-zip-ar… Thierry ________________________________________ De : Pharo-dev [pharo-dev-bounces(a)lists.pharo.org] de la part de btc(a)openinworld.com [btc(a)openinworld.com] EnvoyÃ© : dimanche 9 mars 2014 16:43 Ã : metacello(a)googlegroups.com; Discusses Development of Pharo; Squeak Virtual Machine Development Discussion Objet : [Pharo-dev] Git & MS Windows path length restriction I started looking into Pharo Case 13030 "Many tests failing in MetacelloValidation Job on Jenkins" and even before getting into it, I've hit a stumbling block on Windows 7 with its pathName length restriction of 259 characters. I managed to isolate the problem as follows... MetacelloPlatform current downloadFile: 'https://github.com/dalehenrich/metacello-work/zipball/master' to: 'C:\tmp\github-dalehenrichmetacelloworkmaster.zip'. zip := ZipArchive new readFrom: 'C:\tmp\github-dalehenrichmetacelloworkmaster.zip'. zip extractAllTo: 'C:\tmp\unzippedByPharo' asFileReference. which produces the error "FileDoesNotExist: File @ C:\tmp\unzippedByPharo\dalehenrich-metacello-work-96e07b1\repository\Metacello-TestsMCB.package\MetacelloScriptingStandardTestHarness.class\instance\validateExpectedConfigurationClassName.expectedConfigurationVersion.expectedConfigurationRepository.expectedBaselineClassName.expectedBaselineVersion.expectedBaselineRepository..st" where the #size of that filename is 354 characters. Trying to drill into github-dalehenrichmetacelloworkmaster.zip from Windows Explorer produces error "The Compresses (zipped) file is invalid." However using 7Zip [1] I can extract the file so that _all_ files appear in the hierarchy, so it seems 259 is not a hard limit, and indeed that limit is imposed by the Windows Shell, since NTFS can have a path length of ~32K [2]. Operation of 7Zip was verified by: * From Pharo: zip members size --> 6007 * For cmd.exe: dir /b /s > dir.txt Open dir.txt into Notepad++ --> 6007 (after dir.txt line removed) Also Windows Explorer <Properties> reports (5277 files + 730 folders) = 6007 [1] www.7-zip.org [2] http://stackoverflow.com/questions/265769/maximum-filename-length-in-ntfs-w… Now presuming its reasonable for the working directory to have 30 characters, using... histogram := (zip members collect: [ :member | | mySize | mySize := member fileName size. (mySize >=230) ifTrue: [Transcript crShow: mySize printString , '-->' , member fileName printString ]. mySize. ]) asBag. produces... 306-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-TestsMCB.package/MetacelloScriptingStandardTestHarness.class/instance/validateExpectedConfigurationClassName.expectedConfigurationVersion.expectedConfigurationRepository.expectedBaselineClassName.expectedBaselineVersion.expectedBaselineRepository..st' 252-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/createBaseline.for.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 257-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/modifyBaselineVersionIn.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 259-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/modifyVersion.section.for.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 262-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/addSection.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' 265-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/modifySection.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' 278-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/modifySection.sectionIndex.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' So... * Are there many users of Smalltalk/git on Windows * What is the design plan for dealing with the long path name? I guess the change to using the unicode functions to get the 32K pathName length would need to happen in the VM. I wonder if instead of Smalltalk-git working with individual files in the file system, it might work directly from the a zip file. It seems git can be configured to accept zip files and unpack them before pushing into its repository [3]. Some benefits of this: * Avoid this problem on Windows * Accessing one stream when loading a from git repository might be faster than accessing many individual files * Git sees one file per method, but users see one file per package. * Git-zip-files might act more like familiar mcz files - that could be copied around the same - or even provide some convergence if an mcz held a file per method rather than a single source.st file. [3] http://tante.cc/2010/06/23/managing-zip-based-file-formats-in-git/ Interested in you thoughts. cheers -ben

March 9, 2014

Git & MS Windows path length restriction

by btc＠openinworld.com

I started looking into Pharo Case 13030 "Many tests failing in MetacelloValidation Job on Jenkins" and even before getting into it, I've hit a stumbling block on Windows 7 with its pathName length restriction of 259 characters. I managed to isolate the problem as follows... MetacelloPlatform current downloadFile: 'https://github.com/dalehenrich/metacello-work/zipball/master' to: 'C:\tmp\github-dalehenrichmetacelloworkmaster.zip'. zip := ZipArchive new readFrom: 'C:\tmp\github-dalehenrichmetacelloworkmaster.zip'. zip extractAllTo: 'C:\tmp\unzippedByPharo' asFileReference. which produces the error "FileDoesNotExist: File @ C:\tmp\unzippedByPharo\dalehenrich-metacello-work-96e07b1\repository\Metacello-TestsMCB.package\MetacelloScriptingStandardTestHarness.class\instance\validateExpectedConfigurationClassName.expectedConfigurationVersion.expectedConfigurationRepository.expectedBaselineClassName.expectedBaselineVersion.expectedBaselineRepository..st" where the #size of that filename is 354 characters. Trying to drill into github-dalehenrichmetacelloworkmaster.zip from Windows Explorer produces error "The Compresses (zipped) file is invalid." However using 7Zip [1] I can extract the file so that _all_ files appear in the hierarchy, so it seems 259 is not a hard limit, and indeed that limit is imposed by the Windows Shell, since NTFS can have a path length of ~32K [2]. Operation of 7Zip was verified by: * From Pharo: zip members size --> 6007 * For cmd.exe: dir /b /s > dir.txt Open dir.txt into Notepad++ --> 6007 (after dir.txt line removed) Also Windows Explorer <Properties> reports (5277 files + 730 folders) = 6007 [1] www.7-zip.org [2] http://stackoverflow.com/questions/265769/maximum-filename-length-in-ntfs-w… Now presuming its reasonable for the working directory to have 30 characters, using... histogram := (zip members collect: [ :member | | mySize | mySize := member fileName size. (mySize >=230) ifTrue: [Transcript crShow: mySize printString , '-->' , member fileName printString ]. mySize. ]) asBag. produces... 306-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-TestsMCB.package/MetacelloScriptingStandardTestHarness.class/instance/validateExpectedConfigurationClassName.expectedConfigurationVersion.expectedConfigurationRepository.expectedBaselineClassName.expectedBaselineVersion.expectedBaselineRepository..st' 252-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/createBaseline.for.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 257-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/modifyBaselineVersionIn.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 259-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/class/modifyVersion.section.for.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups..st' 262-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/addSection.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' 265-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/modifySection.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' 278-->'dalehenrich-metacello-work-96e07b1/repository/Metacello-ToolBox.package/MetacelloToolBox.class/instance/modifySection.sectionIndex.repository.requiredProjects.packages.dependencies.includes.files.repositories.preLoadDoIts.postLoadDoIts.supplyingAnswers.groups.versionSpecsDo..st' So... * Are there many users of Smalltalk/git on Windows * What is the design plan for dealing with the long path name? I guess the change to using the unicode functions to get the 32K pathName length would need to happen in the VM. I wonder if instead of Smalltalk-git working with individual files in the file system, it might work directly from the a zip file. It seems git can be configured to accept zip files and unpack them before pushing into its repository [3]. Some benefits of this: * Avoid this problem on Windows * Accessing one stream when loading a from git repository might be faster than accessing many individual files * Git sees one file per method, but users see one file per package. * Git-zip-files might act more like familiar mcz files - that could be copied around the same - or even provide some convergence if an mcz held a file per method rather than a single source.st file. [3] http://tante.cc/2010/06/23/managing-zip-based-file-formats-in-git/ Interested in you thoughts. cheers -ben

March 9, 2014

Re: [Pharo-dev] [Moose-dev] [Pharo-business] [ANN] DBPedia: Query Wikipedia from Pharo

by Norbert Hartl

Am 03.03.2014 um 14:22 schrieb Johan Fabry <jfabry(a)dcc.uchile.cl>: > +1 to that! It would be good to have some kind of uniform syntax for constructing queries. Especially now that the use of JSON objects is quite common. > I would like to have that, too. I need to have query support for elasticsearch and I would do something similar like MongoQueries. The idea of having a common query syntax is nice but Iâm not convinced it will be easy to do. The parser needs to be easily extendable because most applications will have their custom additions to the base syntax. Norbert > The OP was actually referring to MongoQueries. To get some examples of the syntax see section 4 of https://ci.inria.fr/pharo-contribution/job/PharoForTheEnterprise/lastSucces… > > On Mar 3, 2014, at 10:13 AM, Alexandre Bergel <alexandre.bergel(a)me.com> wrote: > >>> What would be nice is to have an abstraction like mongoTalk on top to avoid to manipulate strings but to manipulate query elements. >> >> That is a nice idea. >> >> Alexandre > > > > ---> Save our in-boxes! http://emailcharter.org <--- > > Johan Fabry - http://pleiad.cl/~jfabry > PLEIAD lab - Computer Science Department (DCC) - University of Chile > >

March 9, 2014

Re: [Pharo-dev] base64.b64encode(data) equivalent

by Sven Van Caekenberghe

ZnBase64Encoder new encode: 42 asByteArray. => 'Kg==' (ZnBase64Encoder new decode: 'Kg==') asInteger. => 42 There is also #asByteArrayOfSize: and signed/unsigned might come into play as well. Note that strictly speaking (as implemented by ZnBase64Encoder), Base64 is a binary to string encoding. But is it often used string to string (as in #base64Encoded and #base64Decoded, which are still implemented using Base64MimeConverter) but the explicit correct way is as in ZnUtils #encodeBase64: and #decodeBase64: using a character encoding. On 09 Mar 2014, at 09:09, Pharo4Stef <pharo4Stef(a)free.fr> wrote: > around Zinc-Character-Encoding-Core > :) > > Stef > On 09 Mar 2014, at 09:07, Pharo4Stef <pharo4Stef(a)free.fr> wrote: > >> Iâm sure that we have that. I remember svenâs code using encoder >> but I do not know where. >> >> # Method for encoding ints with base64 encoding >> def encode(n): >> data = struct.pack("i", n) >> s = base64.b64encode(data) >> return s >> >

March 9, 2014

Re: [Pharo-dev] base64.b64encode(data) equivalent

by Pharo4Stef

around Zinc-Character-Encoding-Core :) Stef On 09 Mar 2014, at 09:07, Pharo4Stef <pharo4Stef(a)free.fr> wrote: > Iâm sure that we have that. I remember svenâs code using encoder > but I do not know where. > > # Method for encoding ints with base64 encoding > def encode(n): > data = struct.pack("i", n) > s = base64.b64encode(data) > return s >

March 9, 2014

base64.b64encode(data) equivalent

by Pharo4Stef

Iâm sure that we have that. I remember svenâs code using encoder but I do not know where. # Method for encoding ints with base64 encoding def encode(n): data = struct.pack("i", n) s = base64.b64encode(data) return s

March 9, 2014

Re: [Pharo-dev] Smalltalk tools: explore failing in 3.0

by Pharo4Stef

I do not know the code of the inspector (we fixed the processes use friday with clement) and I got not resolve the problem in less than 10min > > On 08 Mar 2014, at 08:33, phil(a)highoctane.be wrote: > >> If I do inspect >> >> Smalltalk tools >> >> it works. >> >> But explore crashes. >> >> How is that? > > not tested :) > We will fix that. Probably nothing >> >> Phil > >

March 9, 2014