For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths: https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin... For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) : DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ]. please test and give feedback. This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations. nicolai
This is great news! I had tons of problems last year with that and I did not know what to do about this. Doru On Sun, Aug 23, 2015 at 1:44 PM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- www.tudorgirba.com "Every thing has its own flow"
Thanks for trying to fix this issue! Le 23/8/15 13:44, Nicolai Hess a écrit :
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
And If you want to review the changes: https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-lon... 2015-08-23 13:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
On 23 Aug 2015, at 6:09 , Nicolai Hess <nicolaihess@web.de> wrote:
And If you want to review the changes:
https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-lon... <https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-lon...>
2015-08-23 13:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de <mailto:nicolaihess@web.de>>: For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin... <https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...>
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
+ #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength) { \ + buffer = (WCHAR*)alloca((size+4+1)*sizeof(WCHAR));\ + buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\ + MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\ + buffer[size + 4] = 0;\ + size += 4;} - Is an alloca version really worth it for the potential problems? - Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines? - The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into. I'm curious what the "\\?\ <smb://?/>" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering int caseSensitiveFileMode = 0; if(!caseSensitiveFileMode) return 0; I guess it doesn't really matter. Otherwise, looks good to me! Cheers, Henry
2015-08-24 13:34 GMT+02:00 Henrik Johansen <henrik.s.johansen@veloxit.no>:
On 23 Aug 2015, at 6:09 , Nicolai Hess <nicolaihess@web.de> wrote:
And If you want to review the changes:
https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-lon...
2015-08-23 13:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
Thank you Henrik, for looking at this
+ #define CONVERT_MULTIBYTE_TO_WIDECHAR_PATH(buffer, size, fileNameString, fileNameLength) { \+ buffer = (WCHAR*)alloca((size+4+1)* sizeof(WCHAR));\+ buffer[0] = L'\\';buffer[1] = L'\\'; buffer[2] = L'?'; buffer[3] = L'\\';\+ MultiByteToWideChar(CP_UTF8, 0, fileNameString, fileNameLength, buffer + 4, size);\+ buffer[size + 4] = 0;\+ size += 4;}
- Is an alloca version really worth it for the potential problems?
It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).
- Also, LONG_PATH_PREFIX_SIZE is defined above in the header, should probably use this instead of 4 in the last two lines?
Yes
- The comment on lines 111/164/265/381, while not modified, are somewhat incorrect, null-terminated C-string indicates UTF8, which we are converting from, not into.
Right, I ll chang that
I'm curious what the "\\?\" prefix does to hasCaseSensitiveDuplicate (adding a new ? segment to the path), but I cba to understand that code, and considering int caseSensitiveFileMode = 0; if(!caseSensitiveFileMode) return 0;
I guess it doesn't really matter.
Yes, I tried to make this change in a way that does not break the case sensitive duplicate check, but even without my change, enabling the caseSenstiveFileMode makes the windows vm unusable. So, at least, it is no worse.
Otherwise, looks good to me!
Cheers, Henry
On Mon, Aug 24, 2015 at 8:44 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-08-24 13:34 GMT+02:00 Henrik Johansen <henrik.s.johansen@veloxit.no>:
On 23 Aug 2015, at 6:09 , Nicolai Hess <nicolaihess@web.de> wrote:
And If you want to review the changes:
https://github.com/nicolaihess/pharo-vm/compare/master...nicolaihess:win-lon...
2015-08-23 13:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
- Is an alloca version really worth it for the potential problems?
It is what Marcel used for the long path support in the squeak vm (I would have copy and paste the whole change, but I couldn't find the appropriate change for directory functions).
"Potential problems" sounded like something interesting I could learn. Here is a summary... http://stackoverflow.com/questions/5807612/on-the-use-and-abuse-of-alloca On the use and abuse of alloca * alloca five times faster than malloc (but is that necessary given file operations likely take longer anyway?) * most concerns with alloca no problem here e.g. its a very short lifespan; the pointer is not returned from the function. http://www.strchr.com/alloca The Perils of alloca() * remaining concern is potential stack exhaustion... * is MAX_LONG_FILE_PATH=32kB significant today (but also consider embedded application) ? * counter argument, primitive functions are not likely to entered recursively ? Microsoft advises alloca [1] deprecated in favour of malloca [2] which allocates either stack or heap depending on requested size [1] https://msdn.microsoft.com/en-us/library/wb1s57t5(v=vs.80).aspx [2] https://msdn.microsoft.com/en-us/library/5471dc8s(VS.80).aspx https://sourceware.org/ml/libc-alpha/2012-06/msg00246.html An example of a DIY malloca... * might have the advantage of tuning MAX_ALLOCA_CUTOFF post-compilation * minGW gcc might not have malloca * if large allocations go on the heap, maybe MAX_LONG_FILE_PATH test not required, just MAX_ALLOCA_CUTOFF ---------- btw, perhaps rename CONVERT_MULTIBYTE_TO_WIDECHAR_PATH to ALLOC_WIDECHAR_PATH_FROM_MULTIBYTE or similar so its more obviously symmetrical with FREE_WIDECHAR_PATH. cheers -ben
Nicolai 2015-08-23 8:44 GMT-03:00 Nicolai Hess <nicolaihess@web.de>:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore. However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :( Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limi... such Win Unicode API would let VM to use the 32767 bricks This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
Thank you Nicolai for your effort, I know it's hard to dive into depths with C. Hernán
nicolai
Hi Hernán Thanks for your feedback 2015-08-24 8:13 GMT+02:00 Hernán Morales Durand <hernan.morales@gmail.com>:
Nicolai
2015-08-23 8:44 GMT-03:00 Nicolai Hess <nicolaihess@web.de>:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
Using Win 8.1, I confirm FileList tree now is reading directories beyond 260 limit, without need to #initialize DiskStore.
However there should be another concern with GitFileTree uncompression because loading Aconcagua/Chalten, etc with your new VM in a directory name with more characters, still signals File not found :(
I tried Aconcagua from Catalogbrowser in pharo 5.0 and it worked for me. Can you show what path is the base path, the working directory of your pharo package-cache/github directory. Is there something special in your path name or the disk?
Is the VM using the "Unicode-aware API"? According to this guys http://serverfault.com/questions/163419/window-256-characters-path-name-limi... such Win Unicode API would let VM to use the 32767 bricks
I am not sure, we are using all file/directory operations that should use the unicode version with multibyte char (CreateFileW / FindFirstFileW / ...)
This wasn't as easy as I thought, and I had to make some more changes
for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
Thank you Nicolai for your effort, I know it's hard to dive into depths with C.
Hernán
nicolai
Hi Nicolai, I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized? On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)). But I got no response, so I tried to adopt Marcels change. And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted? I would love if someone else got this working. nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
Hi Nicolai, On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)).
Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out #define PharoVM 0 ... #if PharoVM instead of #if !defined(PharoVM) # define PharoVM 0 #endif ... #if PharoVM but I'll fix it v soon. But I got no response, so I tried to adopt Marcels change.
And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted?
Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Di... I'm sorry I didn't see your message. Did you ask on vm-dev?
I would love if someone else got this working.
:-) I hear you :-)
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-10 23:07 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)).
Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out
#define PharoVM 0 ... #if PharoVM
instead of
#if !defined(PharoVM) # define PharoVM 0 #endif ... #if PharoVM
but I'll fix it v soon.
OK, I'll try to copy marcels solution
But I got no response, so I tried to adopt Marcels change.
And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted?
Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Di...
Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there be a "plugins" folder in Cog/platforms/win32/? Where are the changes to sqWin32FilePrims.c now?
I'm sorry I didn't see your message. Did you ask on vm-dev?
I am not sure, maybe I only send it in the squeak-dev list
I would love if someone else got this working.
:-) I hear you :-)
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-11 9:43 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-10 23:07 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)).
Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out
#define PharoVM 0 ... #if PharoVM
instead of
#if !defined(PharoVM) # define PharoVM 0 #endif ... #if PharoVM
but I'll fix it v soon.
OK, I'll try to copy marcels solution
But I got no response, so I tried to adopt Marcels change.
And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted?
Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Di...
Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there be a "plugins" folder in Cog/platforms/win32/? Where are the changes to sqWin32FilePrims.c now?
Ok, found it. It is in the trunk branch :)
I'm sorry I didn't see your message. Did you ask on vm-dev?
I am not sure, maybe I only send it in the squeak-dev list
I would love if someone else got this working.
:-) I hear you :-)
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-11 9:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-11 9:43 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-10 23:07 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)).
Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out
#define PharoVM 0 ... #if PharoVM
instead of
#if !defined(PharoVM) # define PharoVM 0 #endif ... #if PharoVM
but I'll fix it v soon.
OK, I'll try to copy marcels solution
But I got no response, so I tried to adopt Marcels change.
And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted?
Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Di...
Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there be a "plugins" folder in Cog/platforms/win32/? Where are the changes to sqWin32FilePrims.c now?
Ok, found it. It is in the trunk branch :)
No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be used for a merge into pharos vm repository?
I'm sorry I didn't see your message. Did you ask on vm-dev?
I am not sure, maybe I only send it in the squeak-dev list
I would love if someone else got this working.
:-) I hear you :-)
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
Hi Nicolai, On Fri, Sep 11, 2015 at 12:48 AM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-11 9:44 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-11 9:43 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-10 23:07 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
On Thu, Sep 10, 2015 at 1:17 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Yes, I know, and I already asked on the ML if we can take that change or if it is not possible because we have already other changes (pharos file plugin provides addition attributes (the file permissions)).
Esteban and I just harmonized this. I'm (almost correctly) generating the plugin source such that one can define either PharoVM or not to chose between the different versions. I say "almost" because I'm currently spitting out
#define PharoVM 0 ... #if PharoVM
instead of
#if !defined(PharoVM) # define PharoVM 0 #endif ... #if PharoVM
but I'll fix it v soon.
OK, I'll try to copy marcels solution
But I got no response, so I tried to adopt Marcels change.
And I only found the change for sqWin32FilePrims.c, but missed the other change that is necessary in sqWin32Directory.c (and asked on the ML). In the meantime Marcel added this but I could not find the changed source, only an updated vm. Is this change already comitted?
Yes, see http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/vm/sqWin32Di...
Ah, good. Now (I must be blind, but) I can not see the other change, shouldn't there be a "plugins" folder in Cog/platforms/win32/? Where are the changes to sqWin32FilePrims.c now?
Ok, found it. It is in the trunk branch :)
No, wait, now I am confused. The change to sqWin32FilePrims.c is only in the trunk branch and the change to sqWin32Directory.c is only in the Cog branch. What branch includes the solution for both (sqWin32Directory.c and sqWin32FilePrims.c) and should be used for a merge into pharos vm repository?
There are two "externals" in the Cog branch: X platforms/win32/plugins X platforms/Cross/plugins So http://www.squeakvm.org/svn/squeak/branches/Cog is the Cog repository, but http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/win32/plugins is in fact http://squeakvm.org/svn/squeak/trunk/platforms/win32/plugins and http://www.squeakvm.org/svn/squeak/branches/Cog/platforms/Cross/plugins is in fact http://squeakvm.org/svn/squeak/trunk/platforms/Cross/plugins OK?
I'm sorry I didn't see your message. Did you ask on vm-dev?
I am not sure, maybe I only send it in the squeak-dev list
I would love if someone else got this working.
:-) I hear you :-)
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source. I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the squeak vm main branch (wrapped with pharo vm ifdefs). github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin I'll create a push request. nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at? On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source. I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the squeak vm main branch (wrapped with pharo vm ifdefs).
github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin I'll create a push request.
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-14 23:10 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?
I changed (removed my code, added Marcels) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c I added (should be the same as in squeak vm repository) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c But my changes only about that code that already is in squeaks repository. If we want to merge some of the pharo-vm code into squeaks branch, we first need to wrap that code with some #ifdefs (For the win32 file/directory stuff, this would be code in platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c and platforms/Cross/plugins/FilePlugin/FilePlugin.h) Maybe Estaban should look at this, and tell which path he would follow to merge those changes.
On Sat, Sep 12, 2015 at 8:00 AM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-10 21:31 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Hi Nicolai,
I'm a bit concerned that this is creating drift with the "official" Cog source base at http://www.squeakvm.org/svn/squeak/branches/Cog. Long filename support for win32 was recently added by Marcel Taumel in May of this year. Are you tracking that? How can we keep the sources harmonized?
Ok, I removed all of my code again and merged with sqWin32Directory.c and sqWin32FilePrims.c from the official cog source. I left only pharos additions for the fileattributes (posixpermissions) this should make it easier to merge which the squeak vm main branch (wrapped with pharo vm ifdefs).
github branch: https://github.com/nicolaihess/pharo-vm/tree/harmonize_win_file_plugin I'll create a push request.
nicolai
On Sun, Aug 23, 2015 at 4:44 AM, Nicolai Hess <nicolaihess@web.de> wrote:
For those who had problems with pharo on windows and github based repositories, I built a windows vm with support for long paths:
https://drive.google.com/file/d/0B8yEahnuIem2bmxwdzJuUXFxVGM/view?usp=sharin...
For browsing directories with large paths (FileList or Inspect), you may need one additional change in the image (But I am not really sure about that) :
DiskStore>>initialize super initialize. maxFileNameLength := Smalltalk vm maxFilenameLength ifNil: [ 32767 ].
please test and give feedback.
This wasn't as easy as I thought, and I had to make some more changes for the file permissions (the stat-functions don't work for files with long paths). Please test other file/folder operations.
nicolai
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
2015-09-15 9:19 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-14 23:10 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?
I changed (removed my code, added Marcels) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c
I added (should be the same as in squeak vm repository) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
But my changes only about that code that already is in squeaks repository.
If we want to merge some of the pharo-vm code into squeaks branch, we first need to wrap that code with some #ifdefs
(For the win32 file/directory stuff, this would be code in platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c and platforms/Cross/plugins/FilePlugin/FilePlugin.h)
Maybe Estaban should look at this, and tell which path he would follow to merge those changes.
OK, these changes are now in pharos vm repository, .... but I just found another issue with long paths, squeak and pharo: This is a path from one github repository based package u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st Yes, pathlength is 282! But I think the problem is not the overall path name, but the length of the filename. On squeak, you can open and browse the parent directory in a FileList and even show the contents of the contained files. On Pharo, this does not work, because Pharos FileList uses a different method for accessing the file attributes. (the difference is, we call dir_EntryLookup, whereas (most) of Squeaks FileList methods just call dir_Lookup). But there are some methods in Squeak that call dir_EntryLookup and those are failing too. For example: "Creating a Stream and reading the contents works" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') contents. "Creating a Stream and accessing the directory entry does not work" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') directoryEntry " -> Error: file not in directory: initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st" nicolai
Hi Nicolai, I have to say that the problem lies with the git integration. The length of Smalltalk selectors is effectively unlimited and the language encourages Kong descriptive names. Many (most?) unixes have a limitation of 256 characters in any path element and a PATH_MAX that may be as little as 1024. So the current architecture is, for me, way too close to implementation limits. IMO a far better design would be to use protocol names, all methods per protocol, eg Object.accessing or two files per class (instance & class side) with methods ordered by selector. Either of those, but especially the last, should give good diff results in git but be well away from the implementation limit. The one method per file, selector as file name approach is asking for trouble. _,,,^..^,,,_ (phone)
On Oct 12, 2015, at 3:04 PM, Nicolai Hess <nicolaihess@web.de> wrote:
2015-09-15 9:19 GMT+02:00 Nicolai Hess <nicolaihess@web.de>:
2015-09-14 23:10 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com>:
Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?
I changed (removed my code, added Marcels) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c
I added (should be the same as in squeak vm repository) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
But my changes only about that code that already is in squeaks repository.
If we want to merge some of the pharo-vm code into squeaks branch, we first need to wrap that code with some #ifdefs
(For the win32 file/directory stuff, this would be code in platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c and platforms/Cross/plugins/FilePlugin/FilePlugin.h)
Maybe Estaban should look at this, and tell which path he would follow to merge those changes.
OK, these changes are now in pharos vm repository, ....
but I just found another issue with long paths, squeak and pharo:
This is a path from one github repository based package
u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st
Yes, pathlength is 282! But I think the problem is not the overall path name, but the length of the filename.
On squeak, you can open and browse the parent directory in a FileList and even show the contents of the contained files. On Pharo, this does not work, because Pharos FileList uses a different method for accessing the file attributes. (the difference is, we call dir_EntryLookup, whereas (most) of Squeaks FileList methods just call dir_Lookup).
But there are some methods in Squeak that call dir_EntryLookup and those are failing too.
For example:
"Creating a Stream and reading the contents works" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') contents.
"Creating a Stream and accessing the directory entry does not work" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') directoryEntry " -> Error: file not in directory: initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st"
nicolai
Hi Eliot, I believe everytime this comes up, it is not an issue of filetree making files too long for the filesystem, it is an issue of Pharo not being able to read or write a valid file for the filesystem. It is easy to "fix" filetree to workaround bugs in FilePlugin on Windows, but I'm not sure it's the right way to solve the problem ;) Simply because someone will hit it when reading a random, valid file one day. Thierry Le 13/10/2015 01:01, Eliot Miranda a écrit :
Hi Nicolai,
I have to say that the problem lies with the git integration. The length of Smalltalk selectors is effectively unlimited and the language encourages Kong descriptive names. Many (most?) unixes have a limitation of 256 characters in any path element and a PATH_MAX that may be as little as 1024. So the current architecture is, for me, way too close to implementation limits. IMO a far better design would be to use protocol names, all methods per protocol, eg Object.accessing or two files per class (instance & class side) with methods ordered by selector. Either of those, but especially the last, should give good diff results in git but be well away from the implementation limit. The one method per file, selector as file name approach is asking for trouble.
_,,,^..^,,,_ (phone)
On Oct 12, 2015, at 3:04 PM, Nicolai Hess <nicolaihess@web.de <mailto:nicolaihess@web.de>> wrote:
2015-09-15 9:19 GMT+02:00 Nicolai Hess <nicolaihess@web.de <mailto:nicolaihess@web.de>>:
2015-09-14 23:10 GMT+02:00 Eliot Miranda <eliot.miranda@gmail.com <mailto:eliot.miranda@gmail.com>>:
Thanks Nicolai, that's great! I'll merge in your changes. Which files do I need to look at?
I changed (removed my code, added Marcels) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c
I added (should be the same as in squeak vm repository) platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c
But my changes only about that code that already is in squeaks repository.
If we want to merge some of the pharo-vm code into squeaks branch, we first need to wrap that code with some #ifdefs
(For the win32 file/directory stuff, this would be code in platforms/win32/plugins/FilePlugin/sqWin32FilePrims.c platforms/win32/vm/sqWin32.h platforms/win32/vm/sqWin32Directory.c and platforms/Cross/plugins/FilePlugin/FilePlugin.h)
Maybe Estaban should look at this, and tell which path he would follow to merge those changes.
OK, these changes are now in pharos vm repository, ....
but I just found another issue with long paths, squeak and pharo:
This is a path from one github repository based package
u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st
Yes, pathlength is 282! But I think the problem is not the overall path name, but the length of the filename.
On squeak, you can open and browse the parent directory in a FileList and even show the contents of the contained files. On Pharo, this does not work, because Pharos FileList uses a different method for accessing the file attributes. (the difference is, we call dir_EntryLookup, whereas (most) of Squeaks FileList methods just call dir_Lookup).
But there are some methods in Squeak that call dir_EntryLookup and those are failing too.
For example:
"Creating a Stream and reading the contents works" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') contents.
"Creating a Stream and accessing the directory entry does not work" (FileStream fileNamed:'u:\pharo_images\images\Pharo 5.0 (beta)-50376\github-cache\mtaborda\aconcagua\master\mtaborda-aconcagua-4d0a14f\repository\Aconcagua-Core.package\NotProportionalDerivedUnit.class\instance\initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st') directoryEntry " -> Error: file not in directory: initializeBaseUnit.conversionBlock.reciprocalConversionBlock.nameForOne.nameForMany.sign..st"
nicolai
participants (8)
-
Ben Coman -
Eliot Miranda -
Henrik Johansen -
Hernán Morales Durand -
Nicolai Hess -
stepharo -
Thierry Goubier -
Tudor Girba