- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Jan Vrany

On Wed, 2017-05-24 at 11:46 +0530, K K Subbu wrote: > On Tuesday 23 May 2017 08:37 PM, Jan Vrany wrote: > > Hi, > > > > > > Is it possible to build a file responder right into Pharo and > > > > expose packages through WebDAV or FUSE or sshfs service? Then > > > > the > > > > contents can go directly from RAM (pharo) to RAM (repo server > > > > or > > > > git) without going through slow disk filesystem. > > > > > > Yes it's possible and really interesting for lots of > > > reasons.Â Â Not > > > sure why no one is exploring this avenue. > > > > I did explored this path with FUSE ages age (~10 yeas, IIRC) and, > > if my memory is not failing me, was possible but turned out to be > > useless in practice. The problem was that the call to fuse entered > > a kernel and sit there until there was a FS request at which point > > it called back my smalltalk. This means smalltalk was unresponsive > > most of the time. Besides, one needed superuser priviledges and all > > that stuff. Things may be different these days. > > Yes, FUSE has evolved. Here is a recent paper on it: It did, indeed. Thanks!Â I had a wee look at the newest source and it looks one would still have to implement fuse event loop [1] herself.Â It seems that functions needed are not part of public fuse API so one may need to enter a thin ice of using internal API which mayÂ not beÂ visible from outside. Still, seems doable. Jan [1]: https://github.com/libfuse/libfuse/blob/master/lib/fuse_loop.c#L19

May 24, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by K K Subbu

On Tuesday 23 May 2017 08:37 PM, Jan Vrany wrote: > Hi, > >>> Is it possible to build a file responder right into Pharo and >>> expose packages through WebDAV or FUSE or sshfs service? Then the >>> contents can go directly from RAM (pharo) to RAM (repo server or >>> git) without going through slow disk filesystem. >> Yes it's possible and really interesting for lots of reasons. Not >> sure why no one is exploring this avenue. > I did explored this path with FUSE ages age (~10 yeas, IIRC) and, > if my memory is not failing me, was possible but turned out to be > useless in practice. The problem was that the call to fuse entered > a kernel and sit there until there was a FS request at which point > it called back my smalltalk. This means smalltalk was unresponsive > most of the time. Besides, one needed superuser priviledges and all > that stuff. Things may be different these days. Yes, FUSE has evolved. Here is a recent paper on it: https://www.usenix.org/system/files/conference/fast17/fast17-vangoor.pdf The main advantage of FUSE is that responders don't need root permissions to mount archives, unlike NFS. There are over 100 of them like iso, zip, vfat, cram, smb etc. I use fuse-zip regularly to skip extracting lots of small files into disk/SSDs. WebDAV or sshfs is a little more complex but is multi-platform as they work over network sockets instead of device i/o. The important point here is not about the protocol but about live mapping of Pharo packages/sources into the host fs namespace. Live mapping will let us leverage existing tools for devops instead of having to build separate clients for each version control system right into Pharo. The perf boost we get when objects pass from RAM to RAM without going through disk block is a bonus. Regards .. Subbu

May 24, 2017

Re: [Pharo-dev] Pharo booklet collection

by Tim Mackinnon

Thanks Damien - installing docker now, and Iâll see how I get on (Iâm still working through the reddit booklet - which is proving quite fun) Tim > On 23 May 2017, at 22:11, Damien Pollet <damien.pollet(a)gmail.com> wrote: > > Check the travis configuration file. It has the docker invocation that will get the correct image and build the booklet. > https://github.com/SquareBracketAssociates/Booklet-Reddit/blob/master/.trav… <https://github.com/SquareBracketAssociates/Booklet-Reddit/blob/master/.trav…> > > You can also build without docker but that requires an up-to-date TeXlive 2016 (unfortunately we rely on some fixes that are not in the installation iso, so you'd have to either use the net installer or do `tlmgr update --self --all` after installing the iso snapshot) > > On 23 May 2017 at 15:25, Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>> wrote: > Hi - so I forked a repo (the Reddit one), and am reading through it to check for typos (Iâve see a few). Obviously I can correct those, but how do I actually preview my changes to ensure I havenât screwed it all up? > > Iâm guessing I need to install Docker and pull down a specific image - which I can then use to generate the output to test it? Or, is there an ST image that I can run on this directory? > > As a quick scan of whats in a book repo - it all looks a bit more complicated than I expected - with make files and what look like âinplace templatesâ (as in format1.format2.format3 etc). Having played around with Metalsmith.io <http://metalsmith.io/> I think its sort of similar, but It seems like there is more to learn somehow? > > Tim > >> On 6 May 2017, at 11:36, Stephane Ducasse <stepharo.self(a)gmail.com <mailto:stepharo.self@gmail.com>> wrote: >> >> Hi phil >> >> yes I loved this idea. We got burned by that too. >> >> About pillar. The departure of Damien was not expected and it slows us quite a bit. >> >> We (mainly me helped with maxime) are working on >> - making the pillar core nicer >> - less dependencies >> - and with a small in image renderer :) We had fun with guille >> >> >> - In the pharo 60 version I started to clean the command-line. >> - merged many unmerged changes. >> >> But even with all the tests it is not easy.... >> I'm testing in addition with reall >> - book production >> - slides >> - html >> >> But it takes time and energy. I'm trying to remove the dependencies to magritte for example. >> >> Stef >> >> >> <Screen Shot 2017-05-06 at 12.33.18.png> >> >> >> On Sat, May 6, 2017 at 9:30 AM, phil(a)highoctane.be <mailto:phil@highoctane.be> <phil(a)highoctane.be <mailto:phil@highoctane.be>> wrote: >> For books, there is the docker-texlive thing from Damien that we >> looked into yesterday because TexLive was giving some headaches. >> >> https://github.com/philippeback/docker-texlive <https://github.com/philippeback/docker-texlive> >> >> has my current thing for one to build a container with all >> prerequisites for TexLive and Pharo installed. >> >> Idea is to clone the book repo on the host and get into an container >> to build the book. >> >> There is a /work volume that can be mapped onto the host with the >> docker run's -v command. >> >> Then just use ./download.sh in there to install Pharo and pillar in >> the container. Do not do that in the host or you'll end up with wrong >> versions (like my host is CentOS and the container is Ubuntu, even >> worse with host is macOS or Windows). >> >> Then "make book" should give a book in build/ >> There is a "make wipeout" command that obliterates the build folder >> for good so you can start clean. No harm done on your files. >> >> Will document more of this but woul dbe nice if someone could check >> this out too. >> >> There is a script in base/ to build the docker image. >> >> Phil >> >> On Sat, May 6, 2017 at 9:20 AM, Stephane Ducasse >> <stepharo.self(a)gmail.com <mailto:stepharo.self@gmail.com>> wrote: >> > I fixed it. >> > The code for the web site in on github. >> > >> > Stef >> > >> > On Tue, May 2, 2017 at 10:32 PM, Esteban A. Maringolo <emaringolo(a)gmail.com <mailto:emaringolo@gmail.com>> >> > wrote: >> >> >> >> Correct link seems to be >> >> >> >> >> >> http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-Glorp.spiral.pdf <http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-Glorp.spiral.pdf> >> >> Esteban A. Maringolo >> >> >> >> >> >> 2017-05-02 17:28 GMT-03:00 phil(a)highoctane.be <mailto:phil@highoctane.be> <phil(a)highoctane.be <mailto:phil@highoctane.be>>: >> >> > link for glorp gives smacc book error >> >> > >> >> > On Tue, May 2, 2017 at 10:25 PM, Stephane Ducasse >> >> > <stepharo.self(a)gmail.com <mailto:stepharo.self@gmail.com>> >> >> > wrote: >> >> >> >> >> >> Hi Pharoers >> >> >> >> >> >> You are lucky. The "Pharo booklet collection" edited by S. Ducasse is >> >> >> arriving... >> >> >> >> >> >> Are you ready to read nice and focused booklets? >> >> >> >> >> >> You can find at http://files.pharo.org/books/ <http://files.pharo.org/books/> in beta version: >> >> >> >> >> >> Smacc: the Smalltalk Compiler Compiler by J. Brant, T. Goubier, J. >> >> >> Lecerf >> >> >> and S. Ducasse. >> >> >> Glorp: the Object Relational Mapper framework by E. Maringolo, N. Pratt >> >> >> and R. Withney >> >> >> >> >> >> All the book material is hosted on >> >> >> https://github.com/SquareBracketAssociates <https://github.com/SquareBracketAssociates> so you can contribute! Fix >> >> >> typos >> >> >> and propose new material. >> >> >> >> >> >> We are planning a booklet on Magritte, Reddit in 10 cool classes, >> >> >> Voyage, >> >> >> and Mocking with BabyMock and Mocketry, XML (but we need a writer) and >> >> >> potentially PetitParser. >> >> >> >> >> >> If you have material for a tutorial and you would like to be part of >> >> >> the >> >> >> Pharo Booklet Collection contact S. Ducasse. >> >> >> >> >> >> S. Ducasse >> >> > >> >> > >> >> >> > >> >> > > > > > -- > Damien Pollet > type less, do more [ | ] http://people.untyped.org/damien.pollet <http://people.untyped.org/damien.pollet>

May 23, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Dale Henrichs

On 05/23/2017 03:38 PM, Stephan Eggermont wrote: > On 23/05/17 15:50, Dale Henrichs wrote: >> Out of curiosity. Why are you copying the git repo? > > We were following a tutorial. The first participant > downloaded the environment and I assumed copying it > would be faster than downloading through a shared wifi > network. Ah, okay ... now that we know what we know, I guess a zip file of the directory structure would have been a better solution ... Dale

May 23, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Stephan Eggermont

On 23/05/17 15:50, Dale Henrichs wrote: > Out of curiosity. Why are you copying the git repo? We were following a tutorial. The first participant downloaded the environment and I assumed copying it would be faster than downloading through a shared wifi network. Stephan

May 23, 2017

Re: [Pharo-dev] Pharo booklet collection

by Damien Pollet

Check the travis configuration file. It has the docker invocation that will get the correct image and build the booklet. https://github.com/SquareBracketAssociates/Booklet-Reddit/blob/master/.trav… You can also build without docker but that requires an up-to-date TeXlive 2016 (unfortunately we rely on some fixes that are not in the installation iso, so you'd have to either use the net installer or do `tlmgr update --self --all` after installing the iso snapshot) On 23 May 2017 at 15:25, Tim Mackinnon <tim(a)testit.works> wrote: > Hi - so I forked a repo (the Reddit one), and am reading through it to > check for typos (Iâve see a few). Obviously I can correct those, but how do > I actually preview my changes to ensure I havenât screwed it all up? > > Iâm guessing I need to install Docker and pull down a specific image - > which I can then use to generate the output to test it? Or, is there an ST > image that I can run on this directory? > > As a quick scan of whats in a book repo - it all looks a bit more > complicated than I expected - with make files and what look like âinplace > templatesâ (as in format1.format2.format3 etc). Having played around with > Metalsmith.io I think its sort of similar, but It seems like there is > more to learn somehow? > > Tim > > On 6 May 2017, at 11:36, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > > Hi phil > > yes I loved this idea. We got burned by that too. > > About pillar. The departure of Damien was not expected and it slows us > quite a bit. > > We (mainly me helped with maxime) are working on > - making the pillar core nicer > - less dependencies > - and with a small in image renderer :) We had fun with guille > > > - In the pharo 60 version I started to clean the command-line. > - merged many unmerged changes. > > But even with all the tests it is not easy.... > I'm testing in addition with reall > - book production > - slides > - html > > But it takes time and energy. I'm trying to remove the dependencies to > magritte for example. > > Stef > > > <Screen Shot 2017-05-06 at 12.33.18.png> > > > On Sat, May 6, 2017 at 9:30 AM, phil(a)highoctane.be <phil(a)highoctane.be> > wrote: > >> For books, there is the docker-texlive thing from Damien that we >> looked into yesterday because TexLive was giving some headaches. >> >> https://github.com/philippeback/docker-texlive >> >> has my current thing for one to build a container with all >> prerequisites for TexLive and Pharo installed. >> >> Idea is to clone the book repo on the host and get into an container >> to build the book. >> >> There is a /work volume that can be mapped onto the host with the >> docker run's -v command. >> >> Then just use ./download.sh in there to install Pharo and pillar in >> the container. Do not do that in the host or you'll end up with wrong >> versions (like my host is CentOS and the container is Ubuntu, even >> worse with host is macOS or Windows). >> >> Then "make book" should give a book in build/ >> There is a "make wipeout" command that obliterates the build folder >> for good so you can start clean. No harm done on your files. >> >> Will document more of this but woul dbe nice if someone could check >> this out too. >> >> There is a script in base/ to build the docker image. >> >> Phil >> >> On Sat, May 6, 2017 at 9:20 AM, Stephane Ducasse >> <stepharo.self(a)gmail.com> wrote: >> > I fixed it. >> > The code for the web site in on github. >> > >> > Stef >> > >> > On Tue, May 2, 2017 at 10:32 PM, Esteban A. Maringolo < >> emaringolo(a)gmail.com> >> > wrote: >> >> >> >> Correct link seems to be >> >> >> >> >> >> http://files.pharo.org/books-pdfs/booklet-Glorp/2017-05-02-G >> lorp.spiral.pdf >> >> Esteban A. Maringolo >> >> >> >> >> >> 2017-05-02 17:28 GMT-03:00 phil(a)highoctane.be <phil(a)highoctane.be>: >> >> > link for glorp gives smacc book error >> >> > >> >> > On Tue, May 2, 2017 at 10:25 PM, Stephane Ducasse >> >> > <stepharo.self(a)gmail.com> >> >> > wrote: >> >> >> >> >> >> Hi Pharoers >> >> >> >> >> >> You are lucky. The "Pharo booklet collection" edited by S. Ducasse >> is >> >> >> arriving... >> >> >> >> >> >> Are you ready to read nice and focused booklets? >> >> >> >> >> >> You can find at http://files.pharo.org/books/ in beta version: >> >> >> >> >> >> Smacc: the Smalltalk Compiler Compiler by J. Brant, T. Goubier, J. >> >> >> Lecerf >> >> >> and S. Ducasse. >> >> >> Glorp: the Object Relational Mapper framework by E. Maringolo, N. >> Pratt >> >> >> and R. Withney >> >> >> >> >> >> All the book material is hosted on >> >> >> https://github.com/SquareBracketAssociates so you can contribute! >> Fix >> >> >> typos >> >> >> and propose new material. >> >> >> >> >> >> We are planning a booklet on Magritte, Reddit in 10 cool classes, >> >> >> Voyage, >> >> >> and Mocking with BabyMock and Mocketry, XML (but we need a writer) >> and >> >> >> potentially PetitParser. >> >> >> >> >> >> If you have material for a tutorial and you would like to be part of >> >> >> the >> >> >> Pharo Booklet Collection contact S. Ducasse. >> >> >> >> >> >> S. Ducasse >> >> > >> >> > >> >> >> > >> >> > > -- Damien Pollet type less, do more [ | ] http://people.untyped.org/damien.pollet

May 23, 2017

Re: [Pharo-dev] 60494 Red-pane-of-death when sources file missing

by Eliot Miranda

Hi Ben, On May 21, 2017, at 9:01 AM, Ben Coman <btc(a)openinworld.com> wrote: We must be getting close to Pharo 6 Release. Due to missing PharoV60.soruces file, image 60494 get a red-pane-of-death in class pane. Even though the release would bypass this by including the file, we should be more robust to the missing sources file. Hopefully this fix can make it last minute into the release... https://pharo.fogbugz.com/f/cases/20074/Red-pane-of-death- when-sources-file-missing cheers -ben I don't know if this is related but an image Max Leske provided me to look at ImageSegments I noticed that start up without a sources file took of the order of a minute, whereas startup with a source file was the usual snappy affair. Max's image is Pharo6.0 [Latest update: #60482]. I've attached the PharoDebug.log

May 23, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Jan Vrany

Hi, > > Is it possible to build a file responder right into Pharo and > > expose packages through WebDAV or FUSE or sshfs service? Then the > > contents can go directly from RAM (pharo) to RAM (repo server or > > git) without going through slow disk filesystem. > > Yes it's possible and really interesting for lots of reasons.Â Â Not > sure why no one is exploring this avenue. I did explored this path with FUSE ages age (~10 yeas, IIRC) and, if my memory is not failing me, was possible but turned out to be useless in practice. The problem was that the call to fuse entered a kernel and sit there until there was a FS request at which point it called back my smalltalk. This means smalltalk was unresponsive most of the time. Besides, one needed superuser priviledges and all that stuff. Things may be different these days. Some another 10 years before me Claus did exactly the same thing using NFSv3, being able to "mount" running Smalltalk image. Last time I tried it still worked. I was never used for anything practical, as far as I can recall. Jan > > > > Regards .. Subbu > > > >

May 23, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Eliot Miranda

Hi Subbu, > On May 21, 2017, at 11:26 PM, K K Subbu <kksubbu.ml(a)gmail.com> wrote: > >> On Sunday 21 May 2017 11:34 PM, Dale Henrichs wrote: >> As part of his work on Cypress 2.0 Martin McClure is planning on >> supporting a file per class disk format in addition to the file per >> method format and possibly a file per package format. >> >> I'm not sure whether Martin is at the point where he is ready to share >> his plans, but this is a problem that is being worked on and when Martin >> is ready for feedback he'll publish his spec. > > Is it possible to build a file responder right into Pharo and expose packages through WebDAV or FUSE or sshfs service? Then the contents can go directly from RAM (pharo) to RAM (repo server or git) without going through slow disk filesystem. I should say that there's a poor implementation ration for Squeak; poor because it uses the C binding. The right way is to go directly over sockets. This provides efficiency, flexibility and some degree of concurrency; all of which are lacking when programming against the C binding. One could do so much with this other than merely an elegant interface to a vcs. As my friend David Simmons points out, the major benefit of Docker is being able to configure (and hence version) an os virtual machine through the host file system, instead of having to start up the os virtual machine and configure within it and its own file system. A FUSE interface for Pharo/Squeak/Cuis (I'm going to start calling this triad Cusp, Cuis union Squeak & Pharo) would allow non-Smalltalker dev ops people to configure (or monitor, or...) a Cusp image as if it were done familiar native tool. > > Regards .. Subbu >

May 23, 2017

Re: [Pharo-dev] FileTree/Iceberg and SSDs, change the file format

by Eliot Miranda

Hi Subbu, > On May 21, 2017, at 11:26 PM, K K Subbu <kksubbu.ml(a)gmail.com> wrote: > >> On Sunday 21 May 2017 11:34 PM, Dale Henrichs wrote: >> As part of his work on Cypress 2.0 Martin McClure is planning on >> supporting a file per class disk format in addition to the file per >> method format and possibly a file per package format. >> >> I'm not sure whether Martin is at the point where he is ready to share >> his plans, but this is a problem that is being worked on and when Martin >> is ready for feedback he'll publish his spec. > > Is it possible to build a file responder right into Pharo and expose packages through WebDAV or FUSE or sshfs service? Then the contents can go directly from RAM (pharo) to RAM (repo server or git) without going through slow disk filesystem. Yes it's possible and really interesting for lots of reasons. Not sure why no one is exploring this avenue. > > Regards .. Subbu >

May 23, 2017