- Pharo-dev - lists.pharo.org

Re: [Pharo-project] NB FileSystem Primitives Speedup

by Igor Stasenko

You know, Sean, what i like about C that every time i entering C domain i need to solve yet another hard to crack puzzle which takes unpredictably big amount of time. And you can never be sure that there is no hidden surprises behind next turn.. something which conceptually trivial to implement and even can fit on one blank of paper in C code, could take days before you can make it working. Because C programming practices strictly following WYSIWYDG principle (What You See Is What You DONT Get). So, every time you feel like you lack of confusing moments during your day, just go to /usr/include directory, open any header and try to read it ... :) Or if you prefer C++, best is to read STL library code, or Boost code. On 2 June 2012 17:51, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > > StÃ©phane Ducasse wrote >> >> To me it just shows that there is no concerted momentum around VMs... >> Probably Pythoners and ruby people are more concerned. >> > > Yes. Probably they are concerned because they are so much slower than us ;) > http://shootout.alioth.debian.org/u32/performance.php?test=meteor > > Don't get discouraged. The VM is frightening and mysterious to many, but > with the help of Igor, Mariano, and others around to write blogs, answer > questions, improve the build process, and lead the charge to improve, it > seems that working on the VM is becoming more and more accessible/common... > Even last year at ESUG, with 1 year using Smalltalk, I found building and > working on the VM frustrating and confusing. But now, esp. with > Jenkins/git/etc, it's a snap, for example, to build debug VMs, step through > execution, and even submit VM patches. > > Let's keep our great energy and progress multiplying... > > Sean > > -- > View this message in context: http://forum.world.st/NB-FileSystem-Primitives-Speedup-tp4632820p4632911.ht… > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > -- Best regards, Igor Stasenko.

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by Igor Stasenko

On 2 June 2012 17:26, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > Don't you think that this is what we are trying to do since ages: improving the infrastructure > but not by piling junk on junk. hehe stef.. I can say this happens everywhere. Piles of junk on top of each other.. Like figuring out, why output of readdir() function having strange results.. To figure it out, you have to read this code, and understand what's going on: this is part of standard C library on mac (/usr/include/dirent.h on your machine, you can check it) __BEGIN_DECLS #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int alphasort(const void *, const void *) __DARWIN_INODE64(alphasort); #endif /* not POSIX */ int closedir(DIR *) __DARWIN_ALIAS(closedir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int getdirentries(int, char *, int, long *) #if __DARWIN_64_BIT_INO_T /* * getdirentries() doesn't work when 64-bit inodes is in effect, so we * generate a link error. */ __asm("_getdirentries_is_not_available_when_64_bit_inodes_are_in_effect") #else /* !__DARWIN_64_BIT_INO_T */ __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0,__MAC_10_6, __IPHONE_2_0,__IPHONE_2_0) #endif /* __DARWIN_64_BIT_INO_T */ ; #endif /* not POSIX */ DIR *opendir(const char *) __DARWIN_ALIAS_I(opendir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) DIR *__opendir2(const char *, int) __DARWIN_ALIAS_I(__opendir2); #endif /* not POSIX */ struct dirent *readdir(DIR *) __DARWIN_INODE64(readdir); int readdir_r(DIR *, struct dirent *, struct dirent **) __DARWIN_INODE64(readdir_r); void rewinddir(DIR *) __DARWIN_ALIAS_I(rewinddir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int scandir(const char *, struct dirent ***, int (*)(struct dirent *), int (*)(const void *, const void *)) __DARWIN_INODE64(scandir); #ifdef __BLOCKS__ int scandir_b(const char *, struct dirent ***, int (^)(struct dirent *), int (^)(const void *, const void *)) __DARWIN_INODE64(scandir_b) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2); #endif /* __BLOCKS__ */ #endif /* not POSIX */ void seekdir(DIR *, long) __DARWIN_ALIAS_I(seekdir); long telldir(DIR *) __DARWIN_ALIAS_I(telldir); __END_DECLS #endif /* !KERNEL */ -- Best regards, Igor Stasenko.

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by Sean P. DeNigris

StÃ©phane Ducasse wrote > > To me it just shows that there is no concerted momentum around VMs... > Probably Pythoners and ruby people are more concerned. > Yes. Probably they are concerned because they are so much slower than us ;) http://shootout.alioth.debian.org/u32/performance.php?test=meteor Don't get discouraged. The VM is frightening and mysterious to many, but with the help of Igor, Mariano, and others around to write blogs, answer questions, improve the build process, and lead the charge to improve, it seems that working on the VM is becoming more and more accessible/common... Even last year at ESUG, with 1 year using Smalltalk, I found building and working on the VM frustrating and confusing. But now, esp. with Jenkins/git/etc, it's a snap, for example, to build debug VMs, step through execution, and even submit VM patches. Let's keep our great energy and progress multiplying... Sean -- View this message in context: http://forum.world.st/NB-FileSystem-Primitives-Speedup-tp4632820p4632911.ht… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by Igor Stasenko

On 2 June 2012 17:01, David T. Lewis <lewis(a)mail.msen.com> wrote: > > <OT> > IMHO more people need to learn how to write small plugins. It is important > for modularity and reliability, and for trying new VM experiments without > waiting for a "VM guru" to help. Maybe you can also provide a way for > people to add their experimental plugins to the Jenkins builds too, so > that they can be made quickly available to anyone with an interest. > </OT> > > Dave yes, i would wellcome that. And if you know, for instance we got this lately: https://ci.lille.inria.fr/pharo/view/Cog/job/CogDroid/ but sometimes i feel like crying it the desert. 1. There's not many people who can be attracted in hacking VM(s) 2. And even among those who can, not many feel excited about idea of improving and reimplementing it The principle "don't fix it if it not broken" works pretty well in VM domain , because of factor of 10 in having fun & time spent when hacking smalltalk code vs hacking VM code. -- Best regards, Igor Stasenko.

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by StÃ©phane Ducasse

> > That is easy to explain ... I was lazy and did not bother to maintain it ;) I do not buy such easy excuse. This is not your fault. It means that nobody cares. > As for process, it is simple. Just build the plugin (though it probably > will need maintenance for bit rot), and add it to your VM builds. On > the image side, add the support to use the plugin if present, else fall > back on old methods. So like that bugs will just be more difficult to track? I like the idea everybody doing its own little hack in his corner. Sounds like software engineering but not with the right definition to me. > I like small plugins because they are modular and reliable. They can > be added and removed without a lot of coordination with VM builders, > platform maintainers, etc. On the image side, you can easily detect > if the plugin is present, so it is not necessary to worry so much > about what version of VM or FilePlugin is running. This was the same shitty story as having a slow socketStream in the image and that everybody had to load the FastSocketStream. Hopefully an idiot like me or marcus renamed FastSocketStream into SocketStream and did not say it and suddenly we got a decent socket stream. Else we would all have again a bad one. > So the answer is ... Just Do It :) Don't you think that this is what we are trying to do since ages: improving the infrastructure but not by piling junk on junk. David we are talking about File operations not funky experimental stuff. To me it just shows that there is no concerted momentum around VMs. Now people will shoot at me but I do not care just immensely sad. When I imagine that I'm spending my time trying to make sure that companies can built software around Smalltalk. Sometimes I think that I'm in the wrong community. Probably Pythoners and ruby people are more concerned. > <OT> > IMHO more people need to learn how to write small plugins. It is important > for modularity and reliability, and for trying new VM experiments without > waiting for a "VM guru" to help. Maybe you can also provide a way for > people to add their experimental plugins to the Jenkins builds too, so > that they can be made quickly available to anyone with an interest. > </OT> You mean with an svn structure where guys like igor do not have commit rights? Probably but this is difficult and not really welcoming. Stef > > Dave > >

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by David T. Lewis

On Sat, Jun 02, 2012 at 04:11:16PM +0200, St?phane Ducasse wrote: > >> That will definitely make a huge performance improvement. Here is something > >> similar (probably bit rotted but maybe still of interest): > >> > >> http://wiki.squeak.org/squeak/2274 > >> > > > > it is a pity that this stuff was never integrated into VMs.. > > or actually never added to file plugin replacing it's naive implementation. > > I do not understand the process of VM enh. > David what prevent it to get used? > May be we should add it the pharo version and suddenly it will kick the system. > But I'm rather puzzled and sad. > > Stef That is easy to explain ... I was lazy and did not bother to maintain it ;) As for process, it is simple. Just build the plugin (though it probably will need maintenance for bit rot), and add it to your VM builds. On the image side, add the support to use the plugin if present, else fall back on old methods. I like small plugins because they are modular and reliable. They can be added and removed without a lot of coordination with VM builders, platform maintainers, etc. On the image side, you can easily detect if the plugin is present, so it is not necessary to worry so much about what version of VM or FilePlugin is running. So the answer is ... Just Do It :) <OT> IMHO more people need to learn how to write small plugins. It is important for modularity and reliability, and for trying new VM experiments without waiting for a "VM guru" to help. Maybe you can also provide a way for people to add their experimental plugins to the Jenkins builds too, so that they can be made quickly available to anyone with an interest. </OT> Dave

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by StÃ©phane Ducasse

>> That will definitely make a huge performance improvement. Here is something >> similar (probably bit rotted but maybe still of interest): >> >> http://wiki.squeak.org/squeak/2274 >> > > it is a pity that this stuff was never integrated into VMs.. > or actually never added to file plugin replacing it's naive implementation. I do not understand the process of VM enh. David what prevent it to get used? May be we should add it the pharo version and suddenly it will kick the system. But I'm rather puzzled and sad. Stef

June 2, 2012

Re: [Pharo-project] NB FileSystem Primitives Speedup

by Igor Stasenko

On 2 June 2012 15:23, David T. Lewis <lewis(a)mail.msen.com> wrote: > On Fri, Jun 01, 2012 at 11:01:29PM +0200, Igor Stasenko wrote: >> On 1 June 2012 22:47, Chris Cunningham <cunningham.cb(a)gmail.com> wrote: >> > On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni(a)gmail.com> wrote: >> >> For those complaining lately about the new FS being a bit slow ;) >> >> Igor and I hacked together a test version for accessing files using NativeBoost. >> >> >> >> From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster >> >> than the file primitives in the current image (5ms instead of 200ms). >> >> >> >> With the current FS infrastructure such a migration can be done absolutely painless by >> >> simply replacing one back-end a time. >> >> >> >> best >> >> cami >> > >> > Can you share this work? ??I've been interested in this particular use >> > for NB for a while now, but haven't gotten around to trying it (or, >> > unfortunately, NB itself) yet. ??This would probably push me into a >> > user role - and is definitely something I'd like to see. >> > >> >> hehe.. there's almost nothing to show about.. >> we're just making calls to two C functions: >> opendir() >> and >> readdir() >> from standard C library. > > That will definitely make a huge performance improvement. Here is something > similar (probably bit rotted but maybe still of interest): > > http://wiki.squeak.org/squeak/2274 > it is a pity that this stuff was never integrated into VMs.. or actually never added to file plugin replacing it's naive implementation. > >> >> a very basic stuff, which you would do if your programming language is C :) >> >> anyways, see attached. >> >> If you on mac, it shall work.. try something like: >> >> (UnixDir on: '/usr/bin') entries >> >> >> > -Chris >> >> -- >> Best regards, >> Igor Stasenko. > > > -- Best regards, Igor Stasenko.

June 2, 2012

Re: [Pharo-project] What i would like to have instead of progress bar

by StÃ©phane Ducasse

so if somebody would like to help - we should spot showWhile: useless senders - progress bar useless calls Stef On Jun 1, 2012, at 1:58 AM, Igor Stasenko wrote: > If you would ask me (yeah nobody asking me, by i will tell anyways ;) ) > how the indication of a progress of some process should look like, > here's my potential answer: > > > - a tiny one line in a corner of desktop (instead of huge banner > centered in window, > sticking on top of everything and obscuring the view) > - not blocking user input > - no stupid nesting.. i don't care what happens inside of something > else happens.. > if it is really necessary, i would just replace the title of a task > with it's subtask title while it performing. > this is just about first rule: it has to be a single line, not a bunch > of lines stocking one over another.. > i think this is abuse. > > most of the times, i just need a notion of progress of a whole thing, > not progress of it's subthings > (if user really wants, he can hover the mouse over progress line to > see the details, but by default > it should take minimal space on the screen). > > > > -- > Best regards, > Igor Stasenko. >

June 2, 2012

[Pharo-project] [ANN] NativeBoost v 1.2 -> release

by Igor Stasenko

Hi, as i promised, after some testing i changed the 1.2 version from #development blessing to #release. Here some more changes: - reorganized categories (mainly cleaning up the NativeBoost-Core category and move some stuff to NativeBoost-Core-FFI category) But this one might be interesting even to non-NB users: i added a fresh implementation of weak finalization registry (NBFinalizationRegistry). In short: - it's not using WeakIdentityDictionary to store registered objects - it produces less garbage, under high add/finalization pressure - it does not allows removal of object once it is registered, since it doesn't pretends to be a Collection, like WeakRegistry does. - if you really need to avoid executing the finalization, use the object returned by #add: method, which is an instance of NBWeakFinalizerItem and send #doNotFinalize message to it (note, any other methods are considered private and messing up with them can be fatal) You can extract implementation and use it without loading NB, but it will work only under VMs with support of new finalization scheme. (but since NB-enabled VMs already having it, i can afford to use it, without having crippling backwards compatibility schemes) -- Best regards, Igor Stasenko.

June 2, 2012