August 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] R.I.P. FileDirectory

by Esteban Lorenzano

On Aug 2, 2012, at 9:29 PM, Chris Muller <asqueaker(a)gmail.com> wrote: >> - FileDirectory uses Strings as path descriptions, FileSystem uses real Objects for that. > > Nope, it uses DirectoryEntry. > >> - The main entry point for FileSystem is the FileReference, the old way is String + FileDirectory > > No, DirectoryEntry is like a FileReference. But some folks may prefer > just to use simple Strings in some cases. Strings are a *real* part > of any modern filesystem domain, so you've got to deal with them at > SOME point. And, since there is shorthand syntax for Strings > understood by the Compiler, it can actually make code *cleaner* in > some cases than having to instantiate an object model every time just > to, for example, get a list of directory contents (not to mention > allocation expense). > It is a matter of taste, or maybe not, which design is better. Maybe FD uses DE, but that is internally. In your programs, you do: FileDirectory on: 'a/path/to/a/file', which gives you a FileDirectory instance, on which you can do operations, like a C library. For instance, FileDirectory default readOnlyFileNamed: 'Blah'. Unless you uses specifically #entries, you never see the DE instances... With FileSystem, you have the reference concept reified into something that represents a file entry, and that object can understand operations related to a file reference. In the same example, with FileSystem you do: 'blah' asFileReference readStream. That of course, gives you the easy of using strings, in a much better design, IMO. Also, DirectoryEntry is *not* like FileReference... maybe in their intentions, but for real DE works more like a data structure with no much behavior remaining (for instance, a DirectoryEntryDirectory cannot give you the directory entries, you have to go back to a FileDirectory to do that). Notice that even FileDirectory creates internally DirectoryEntry instances (as you said), so... the optimization you claim does not occur there either, unless I'm missing something (?) I do believe FileSystem design can be improved, but I think is a huge step in the right direction. Cheers, Esteban

Aug. 2, 2012

Re: [Pharo-project] FileTree sources

by Dale Henrichs

Cami, I just saw your email ... are there any particular versions that you need? We were conversing over on GitHub around that same time, so perhaps you no longer need the versions? Dale ----- Original Message ----- | From: "Camillo Bruni" <camillobruni(a)gmail.com> | To: "Pharo-project(a)lists.gforge.inria.fr Development" <pharo-project(a)lists.gforge.inria.fr> | Sent: Thursday, August 2, 2012 8:06:14 AM | Subject: [Pharo-project] FileTree sources | | I am trying to fix FileTree for 2.0 without FileDirectory. | Can anyone give the mcz versions (they usually get cached) to fill | the | gap between the gemstone and the github repos? | | Cause currently I cannot merge, and I realized that only after | applying all the | changes... | | best | cami |

Aug. 2, 2012

Re: [Pharo-project] Memory usage

by StÃ©phane Ducasse

On Aug 2, 2012, at 10:17 PM, jannik.laval wrote: > Hi, > > I tried the VM included in Moose (moosetechnology.org) > If I remember well, it is a cog vm. > > The maximal heap size is (precisely): 2138046463 bytes. > Why this number ? I don't know. But I will need more than 2Gb. HI jannik was the system usable? Because the problem with more memory is that you need specific GCes. Stef > > I will try the 64bits vm, and see. > > Cheers, > Jannik > > On Aug 1, 2012, at 7:51 PM, David T. Lewis <lewis(a)mail.msen.com> wrote: > >> Hi John, >> >> There is no regression in the interpreter VM ... well actually there was >> about 6 months back, but I keep an eye on it and it's fixed again now :) >> >> The 32-bit interpreter VM will not fail on any 2GB or 4BG boundaries, >> and an interpreter VM compiled for the 64-bit object format can handle >> images greater than 7GB (probably much more, but my 8GB PC is too small >> to do anything larger). >> >> I'm not sure if all of the necessary fixes are in place for the StackVM >> and Cog. If not, I'm sure it will be addressed over time (it's just not >> something that I have ever checked). >> >> I believe that Jannik is interested in running very large images, at >> least on an experimental basis. For anything over a few GB, this requires >> an interpreter VM and a 64-bit image. As you know, this is sure to run >> into problems for the garbage collector as the number of objects >> increases, but it would certainly be interesting to see how far the >> current garbage collector can go in real world conditions before it >> turns to mollasses. >> >> Dave >> >> >> On Wed, Aug 01, 2012 at 08:57:02AM -0400, John McIntosh wrote: >>> A few years back the interpreted virtual machine was fixed to allow an >>> image to grow to the 4 GB limit. >>> It is unclear to me if someone regressed the software to impose a 2GB limit >>> again, or if the 2GB number >>> mentioned is based on how things worked10 years ago? >>> >>> On Wed, Aug 1, 2012 at 5:01 AM, St?phane Ducasse >>> <stephane.ducasse(a)inria.fr>wrote: >>> >>>> >>>> On Jul 31, 2012, at 11:46 PM, johnmci wrote: >>>> >>>>> David Lewis and I spent a far amount of time a few years back to make >>>> the 32 >>>>> vm 4gb clean. So are you running on stale knowledge here, or does the vm >>>>> crash when to goes over 2gb? >>>> >>>> sorry my english limit does not let me know understanding what you mean >>>> exactly. >>>> Jannik in the context of moose would like to see if we can have image >>>> larger than 500 mb (on mac it should be possible). >>>> >>>> Stef >>>> >> > > --- > Jannik Laval > >

Aug. 2, 2012

Re: [Pharo-project] Memory usage

by jannik.laval

Hi, I tried the VM included in Moose (moosetechnology.org) If I remember well, it is a cog vm. The maximal heap size is (precisely): 2138046463 bytes. Why this number ? I don't know. But I will need more than 2Gb. I will try the 64bits vm, and see. Cheers, Jannik On Aug 1, 2012, at 7:51 PM, David T. Lewis <lewis(a)mail.msen.com> wrote: > Hi John, > > There is no regression in the interpreter VM ... well actually there was > about 6 months back, but I keep an eye on it and it's fixed again now :) > > The 32-bit interpreter VM will not fail on any 2GB or 4BG boundaries, > and an interpreter VM compiled for the 64-bit object format can handle > images greater than 7GB (probably much more, but my 8GB PC is too small > to do anything larger). > > I'm not sure if all of the necessary fixes are in place for the StackVM > and Cog. If not, I'm sure it will be addressed over time (it's just not > something that I have ever checked). > > I believe that Jannik is interested in running very large images, at > least on an experimental basis. For anything over a few GB, this requires > an interpreter VM and a 64-bit image. As you know, this is sure to run > into problems for the garbage collector as the number of objects > increases, but it would certainly be interesting to see how far the > current garbage collector can go in real world conditions before it > turns to mollasses. > > Dave > > > On Wed, Aug 01, 2012 at 08:57:02AM -0400, John McIntosh wrote: >> A few years back the interpreted virtual machine was fixed to allow an >> image to grow to the 4 GB limit. >> It is unclear to me if someone regressed the software to impose a 2GB limit >> again, or if the 2GB number >> mentioned is based on how things worked10 years ago? >> >> On Wed, Aug 1, 2012 at 5:01 AM, St?phane Ducasse >> <stephane.ducasse(a)inria.fr>wrote: >> >>> >>> On Jul 31, 2012, at 11:46 PM, johnmci wrote: >>> >>>> David Lewis and I spent a far amount of time a few years back to make >>> the 32 >>>> vm 4gb clean. So are you running on stale knowledge here, or does the vm >>>> crash when to goes over 2gb? >>> >>> sorry my english limit does not let me know understanding what you mean >>> exactly. >>> Jannik in the context of moose would like to see if we can have image >>> larger than 500 mb (on mac it should be possible). >>> >>> Stef >>> > --- Jannik Laval

Aug. 2, 2012

Re: [Pharo-project] Memory usage

by jannik.laval

Thank you ! I will try it in few days. Cheers, Jannik On Aug 1, 2012, at 8:35 PM, David T. Lewis <lewis(a)mail.msen.com> wrote: > On Wed, Aug 01, 2012 at 09:04:20AM -0400, John McIntosh wrote: >> I'm sure I built a vm for 64 bits a few years back, maybe not a stack one, >> can't remember >> ya >> http://lists.squeakfoundation.org/pipermail/vm-dev/2010-January/003732.html > > John: > > Thanks, I had forgotten about your 64*64 VM, I think it is probably exactly > what Jannik needs :) > > Jannik: > > Please try the "64bitImage*64bitVM" image from John's site here: > ftp://ftp.smalltalkconsulting.com/experimental/64bit/ > > If you can run that VM on your Mac, it should do exactly what you need. > Try running that VM with one of your images. If it runs and gives you some > sort of error message about not understanding the image format, that's good! > That means all you need to do is trace your image to 64-bit object format, > which should be fairly easy to do. > > Dave > --- Jannik Laval

Aug. 2, 2012

Re: [Pharo-project] R.I.P. FileDirectory

by Chris Muller

> - FileDirectory uses Strings as path descriptions, FileSystem uses real Objects for that. Nope, it uses DirectoryEntry. > - The main entry point for FileSystem is the FileReference, the old way is String + FileDirectory No, DirectoryEntry is like a FileReference. But some folks may prefer just to use simple Strings in some cases. Strings are a *real* part of any modern filesystem domain, so you've got to deal with them at SOME point. And, since there is shorthand syntax for Strings understood by the Compiler, it can actually make code *cleaner* in some cases than having to instantiate an object model every time just to, for example, get a list of directory contents (not to mention allocation expense).

Aug. 2, 2012

Re: [Pharo-project] R.I.P. FileDirectory

by StÃ©phane Ducasse

camillo it would be good to get a chapter (or update a bit the one we have) on filesystem. Because already having a cool set of examples can help a lot. Stef On Aug 2, 2012, at 8:00 PM, Camillo Bruni wrote: > Finally we did it, that were some hours of hard work :P. > Now we have to provide some help to those people who port their code... > > Certainly some of you might request a compat package for FileDirectory. > There are certain issues with that: > > - FileDirectory is implemented like a C library > - FileSystem is implemented in a OO way with lots of delegation > > - FileDirectory uses Strings as path descriptions, FileSystem uses real Objects for that. > - The main entry point for FileSystem is the FileReference, the old way is String + FileDirectory > > so we have a total inversion of control compared to the old FileDirectory which > will make it hard to maintain a real compatibility package. > > Furthermore I have seen how much ugly and bad code the old FileDirectory has > caused, believe me it is worth the effort to spend a little time to rewrite your > library! You will avoid so much useless code! > > We will sit together these days to see what we can do to limit the efforts > of porting old code. > > best > cami > > > On 2012-08-02, at 19:53, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > >> >> I always wanted to get rid of it. Now there is a difference between wanting and doing :) >> and I should say that we should thank two persons: colin for releasing his code (once we have some kind of metadata for package - relay soon we should not forget to mention that he built the first version and camillo for pushing like a crazy to get Filesystem used everywhere. >> Of course we should thank also guillermo and esteban for the steady, positively aggressive attitude against old code. >> >> So this is cool to see that happing. >> >> PS: I have other dreams :). >> >> Stef > >

Aug. 2, 2012

Re: [Pharo-project] R.I.P. FileDirectory

by StÃ©phane Ducasse

may be a nice parseTree write bunch of rules may help Stef On Aug 2, 2012, at 8:00 PM, Camillo Bruni wrote: > Finally we did it, that were some hours of hard work :P. > Now we have to provide some help to those people who port their code... > > Certainly some of you might request a compat package for FileDirectory. > There are certain issues with that: > > - FileDirectory is implemented like a C library > - FileSystem is implemented in a OO way with lots of delegation > > - FileDirectory uses Strings as path descriptions, FileSystem uses real Objects for that. > - The main entry point for FileSystem is the FileReference, the old way is String + FileDirectory > > so we have a total inversion of control compared to the old FileDirectory which > will make it hard to maintain a real compatibility package. > > Furthermore I have seen how much ugly and bad code the old FileDirectory has > caused, believe me it is worth the effort to spend a little time to rewrite your > library! You will avoid so much useless code! > > We will sit together these days to see what we can do to limit the efforts > of porting old code. > > best > cami > > > On 2012-08-02, at 19:53, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > >> >> I always wanted to get rid of it. Now there is a difference between wanting and doing :) >> and I should say that we should thank two persons: colin for releasing his code (once we have some kind of metadata for package - relay soon we should not forget to mention that he built the first version and camillo for pushing like a crazy to get Filesystem used everywhere. >> Of course we should thank also guillermo and esteban for the steady, positively aggressive attitude against old code. >> >> So this is cool to see that happing. >> >> PS: I have other dreams :). >> >> Stef > >

Aug. 2, 2012

Re: [Pharo-project] R.I.P. FileDirectory

by Camillo Bruni

Finally we did it, that were some hours of hard work :P. Now we have to provide some help to those people who port their code... Certainly some of you might request a compat package for FileDirectory. There are certain issues with that: - FileDirectory is implemented like a C library - FileSystem is implemented in a OO way with lots of delegation - FileDirectory uses Strings as path descriptions, FileSystem uses real Objects for that. - The main entry point for FileSystem is the FileReference, the old way is String + FileDirectory so we have a total inversion of control compared to the old FileDirectory which will make it hard to maintain a real compatibility package. Furthermore I have seen how much ugly and bad code the old FileDirectory has caused, believe me it is worth the effort to spend a little time to rewrite your library! You will avoid so much useless code! We will sit together these days to see what we can do to limit the efforts of porting old code. best cami On 2012-08-02, at 19:53, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > > I always wanted to get rid of it. Now there is a difference between wanting and doing :) > and I should say that we should thank two persons: colin for releasing his code (once we have some kind of metadata for package - relay soon we should not forget to mention that he built the first version and camillo for pushing like a crazy to get Filesystem used everywhere. > Of course we should thank also guillermo and esteban for the steady, positively aggressive attitude against old code. > > So this is cool to see that happing. > > PS: I have other dreams :). > > Stef

Aug. 2, 2012

Re: [Pharo-project] R.I.P. FileDirectory

by StÃ©phane Ducasse

I always wanted to get rid of it. Now there is a difference between wanting and doing :) and I should say that we should thank two persons: colin for releasing his code (once we have some kind of metadata for package - relay soon we should not forget to mention that he built the first version and camillo for pushing like a crazy to get Filesystem used everywhere. Of course we should thank also guillermo and esteban for the steady, positively aggressive attitude against old code. So this is cool to see that happing. PS: I have other dreams :). Stef

Aug. 2, 2012