[Pharo-project] NB FileSystem Primitives Speedup
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time. best cami
yeah that's great! ...but we need a speed up of FS even without that. There are some simple protocols that needs to be enhanced, like the one that checks all entries in a root directory if I execute: '/tmp' asFileReference isDirectory :P On Jun 1, 2012, at 9:17 PM, Camillo Bruni wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
fstat is your friend :) i don't know if primitives providing access to that very useful function. On 1 June 2012 21:21, Esteban Lorenzano <estebanlm@gmail.com> wrote:
yeah that's great! ...but we need a speed up of FS even without that. There are some simple protocols that needs to be enhanced, like the one that checks all entries in a root directory if I execute: '/tmp' asFileReference isDirectory :P
On Jun 1, 2012, at 9:17 PM, Camillo Bruni wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
-- Best regards, Igor Stasenko.
On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
Can you share this work? I've been interested in this particular use for NB for a while now, but haven't gotten around to trying it (or, unfortunately, NB itself) yet. This would probably push me into a user role - and is definitely something I'd like to see. -Chris
On 1 June 2012 22:47, Chris Cunningham <cunningham.cb@gmail.com> wrote:
On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
Can you share this work? Â I've been interested in this particular use for NB for a while now, but haven't gotten around to trying it (or, unfortunately, NB itself) yet. Â This would probably push me into a user role - and is definitely something I'd like to see.
hehe.. there's almost nothing to show about.. we're just making calls to two C functions: opendir() and readdir() from standard C library. a very basic stuff, which you would do if your programming language is C :) anyways, see attached. If you on mac, it shall work.. try something like: (UnixDir on: '/usr/bin') entries
-Chris
-- Best regards, Igor Stasenko.
ah.. sorry it will not work without these patches. These are just couple minor refactorings for accepting '$' as valid part of function name. Because we discovered that we had to use 'readdir$INODE64' function instead of 'readdir' actually we could use older one, but 64 is a bit better :) (i will do a new snapshot of Nativeboost-Core soon) -- Best regards, Igor Stasenko.
On Fri, Jun 01, 2012 at 11:01:29PM +0200, Igor Stasenko wrote:
On 1 June 2012 22:47, Chris Cunningham <cunningham.cb@gmail.com> wrote:
On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
Can you share this work? ??I've been interested in this particular use for NB for a while now, but haven't gotten around to trying it (or, unfortunately, NB itself) yet. ??This would probably push me into a user role - and is definitely something I'd like to see.
hehe.. there's almost nothing to show about.. we're just making calls to two C functions: opendir() and readdir() from standard C library.
That will definitely make a huge performance improvement. Here is something similar (probably bit rotted but maybe still of interest): http://wiki.squeak.org/squeak/2274
a very basic stuff, which you would do if your programming language is C :)
anyways, see attached.
If you on mac, it shall work.. try something like:
(UnixDir on: '/usr/bin') entries
-Chris
-- Best regards, Igor Stasenko.
On 2 June 2012 15:23, David T. Lewis <lewis@mail.msen.com> wrote:
On Fri, Jun 01, 2012 at 11:01:29PM +0200, Igor Stasenko wrote:
On 1 June 2012 22:47, Chris Cunningham <cunningham.cb@gmail.com> wrote:
On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
Can you share this work? ??I've been interested in this particular use for NB for a while now, but haven't gotten around to trying it (or, unfortunately, NB itself) yet. ??This would probably push me into a user role - and is definitely something I'd like to see.
hehe.. there's almost nothing to show about.. we're just making calls to two C functions: opendir() and readdir() from standard C library.
That will definitely make a huge performance improvement. Here is something similar (probably bit rotted but maybe still of interest):
it is a pity that this stuff was never integrated into VMs.. or actually never added to file plugin replacing it's naive implementation.
a very basic stuff, which you would do if your programming language is C :)
anyways, see attached.
If you on mac, it shall work.. try something like:
(UnixDir on: '/usr/bin') entries
-Chris
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
That will definitely make a huge performance improvement. Here is something similar (probably bit rotted but maybe still of interest):
it is a pity that this stuff was never integrated into VMs.. or actually never added to file plugin replacing it's naive implementation.
I do not understand the process of VM enh. David what prevent it to get used? May be we should add it the pharo version and suddenly it will kick the system. But I'm rather puzzled and sad. Stef
On Sat, Jun 02, 2012 at 04:11:16PM +0200, St?phane Ducasse wrote:
That will definitely make a huge performance improvement. Here is something similar (probably bit rotted but maybe still of interest):
it is a pity that this stuff was never integrated into VMs.. or actually never added to file plugin replacing it's naive implementation.
I do not understand the process of VM enh. David what prevent it to get used? May be we should add it the pharo version and suddenly it will kick the system. But I'm rather puzzled and sad.
Stef
That is easy to explain ... I was lazy and did not bother to maintain it ;) As for process, it is simple. Just build the plugin (though it probably will need maintenance for bit rot), and add it to your VM builds. On the image side, add the support to use the plugin if present, else fall back on old methods. I like small plugins because they are modular and reliable. They can be added and removed without a lot of coordination with VM builders, platform maintainers, etc. On the image side, you can easily detect if the plugin is present, so it is not necessary to worry so much about what version of VM or FilePlugin is running. So the answer is ... Just Do It :) <OT> IMHO more people need to learn how to write small plugins. It is important for modularity and reliability, and for trying new VM experiments without waiting for a "VM guru" to help. Maybe you can also provide a way for people to add their experimental plugins to the Jenkins builds too, so that they can be made quickly available to anyone with an interest. </OT> Dave
That is easy to explain ... I was lazy and did not bother to maintain it ;)
I do not buy such easy excuse. This is not your fault. It means that nobody cares.
As for process, it is simple. Just build the plugin (though it probably will need maintenance for bit rot), and add it to your VM builds. On the image side, add the support to use the plugin if present, else fall back on old methods.
So like that bugs will just be more difficult to track? I like the idea everybody doing its own little hack in his corner. Sounds like software engineering but not with the right definition to me.
I like small plugins because they are modular and reliable. They can be added and removed without a lot of coordination with VM builders, platform maintainers, etc. On the image side, you can easily detect if the plugin is present, so it is not necessary to worry so much about what version of VM or FilePlugin is running.
This was the same shitty story as having a slow socketStream in the image and that everybody had to load the FastSocketStream. Hopefully an idiot like me or marcus renamed FastSocketStream into SocketStream and did not say it and suddenly we got a decent socket stream. Else we would all have again a bad one.
So the answer is ... Just Do It :) Don't you think that this is what we are trying to do since ages: improving the infrastructure but not by piling junk on junk.
David we are talking about File operations not funky experimental stuff. To me it just shows that there is no concerted momentum around VMs. Now people will shoot at me but I do not care just immensely sad. When I imagine that I'm spending my time trying to make sure that companies can built software around Smalltalk. Sometimes I think that I'm in the wrong community. Probably Pythoners and ruby people are more concerned.
<OT> IMHO more people need to learn how to write small plugins. It is important for modularity and reliability, and for trying new VM experiments without waiting for a "VM guru" to help. Maybe you can also provide a way for people to add their experimental plugins to the Jenkins builds too, so that they can be made quickly available to anyone with an interest. </OT>
You mean with an svn structure where guys like igor do not have commit rights? Probably but this is difficult and not really welcoming. Stef
Dave
Stéphane Ducasse wrote
To me it just shows that there is no concerted momentum around VMs... Probably Pythoners and ruby people are more concerned.
Yes. Probably they are concerned because they are so much slower than us ;) http://shootout.alioth.debian.org/u32/performance.php?test=meteor Don't get discouraged. The VM is frightening and mysterious to many, but with the help of Igor, Mariano, and others around to write blogs, answer questions, improve the build process, and lead the charge to improve, it seems that working on the VM is becoming more and more accessible/common... Even last year at ESUG, with 1 year using Smalltalk, I found building and working on the VM frustrating and confusing. But now, esp. with Jenkins/git/etc, it's a snap, for example, to build debug VMs, step through execution, and even submit VM patches. Let's keep our great energy and progress multiplying... Sean -- View this message in context: http://forum.world.st/NB-FileSystem-Primitives-Speedup-tp4632820p4632911.htm... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
You know, Sean, what i like about C that every time i entering C domain i need to solve yet another hard to crack puzzle which takes unpredictably big amount of time. And you can never be sure that there is no hidden surprises behind next turn.. something which conceptually trivial to implement and even can fit on one blank of paper in C code, could take days before you can make it working. Because C programming practices strictly following WYSIWYDG principle (What You See Is What You DONT Get). So, every time you feel like you lack of confusing moments during your day, just go to /usr/include directory, open any header and try to read it ... :) Or if you prefer C++, best is to read STL library code, or Boost code. On 2 June 2012 17:51, Sean P. DeNigris <sean@clipperadams.com> wrote:
Stéphane Ducasse wrote
To me it just shows that there is no concerted momentum around VMs... Probably Pythoners and ruby people are more concerned.
Yes. Probably they are concerned because they are so much slower than us ;) http://shootout.alioth.debian.org/u32/performance.php?test=meteor
Don't get discouraged. The VM is frightening and mysterious to many, but with the help of Igor, Mariano, and others around to write blogs, answer questions, improve the build process, and lead the charge to improve, it seems that working on the VM is becoming more and more accessible/common... Even last year at ESUG, with 1 year using Smalltalk, I found building and working on the VM frustrating and confusing. But now, esp. with Jenkins/git/etc, it's a snap, for example, to build debug VMs, step through execution, and even submit VM patches.
Let's keep our great energy and progress multiplying...
Sean
-- View this message in context: http://forum.world.st/NB-FileSystem-Primitives-Speedup-tp4632820p4632911.htm... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
-- Best regards, Igor Stasenko.
2012/6/2 Igor Stasenko <siguctua@gmail.com>:
You know, Sean, what i like about C that every time i entering C domain i need to solve yet another hard to crack puzzle which takes unpredictably big amount of time. And you can never be sure that there is no hidden surprises behind next turn.. something which conceptually trivial to implement and even can fit on one blank of paper in C code, could take days before you can make it working.
Because C programming practices strictly following WYSIWYDG principle (What You See Is What You DONT Get).
I would say, that you MIGHT get, depending on compiler and options...
So, every time you feel like you lack of confusing moments during your day, just go to /usr/include directory, open any header and try to read it ... :)
Or if you prefer C++, best is to read STL library code, or Boost  code.
+1, one of my favourite argument against C++ I always wondered which part of such obfuscation was intentional (vs necessary) C++ adds complexity for sure, some totally useless like distinction between delete and delete[], since underneath you still have dumb pointers, and the compiler can't make a distinction, it's just one more responsibility for the programmer. Nicolas
On 2 June 2012 17:51, Sean P. DeNigris <sean@clipperadams.com> wrote:
Stéphane Ducasse wrote
To me it just shows that there is no concerted momentum around VMs... Probably Pythoners and ruby people are more concerned.
Yes. Probably they are concerned because they are so much slower than us ;) http://shootout.alioth.debian.org/u32/performance.php?test=meteor
Don't get discouraged. The VM is frightening and mysterious to many, but with the help of Igor, Mariano, and others around to write blogs, answer questions, improve the build process, and lead the charge to improve, it seems that working on the VM is becoming more and more accessible/common... Even last year at ESUG, with 1 year using Smalltalk, I found building and working on the VM frustrating and confusing. But now, esp. with Jenkins/git/etc, it's a snap, for example, to build debug VMs, step through execution, and even submit VM patches.
Let's keep our great energy and progress multiplying...
Sean
-- View this message in context: http://forum.world.st/NB-FileSystem-Primitives-Speedup-tp4632820p4632911.htm... Sent from the Pharo Smalltalk mailing list archive at Nabble.com.
-- Best regards, Igor Stasenko.
On 2 June 2012 17:26, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Don't you think that this is what we are trying to do since ages: improving the infrastructure but not by piling junk on junk.
hehe stef.. I can say this happens everywhere. Piles of junk on top of each other.. Like figuring out, why output of readdir() function having strange results.. To figure it out, you have to read this code, and understand what's going on: this is part of standard C library on mac (/usr/include/dirent.h on your machine, you can check it) __BEGIN_DECLS #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int alphasort(const void *, const void *) __DARWIN_INODE64(alphasort); #endif /* not POSIX */ int closedir(DIR *) __DARWIN_ALIAS(closedir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int getdirentries(int, char *, int, long *) #if __DARWIN_64_BIT_INO_T /* * getdirentries() doesn't work when 64-bit inodes is in effect, so we * generate a link error. */ __asm("_getdirentries_is_not_available_when_64_bit_inodes_are_in_effect") #else /* !__DARWIN_64_BIT_INO_T */ __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0,__MAC_10_6, __IPHONE_2_0,__IPHONE_2_0) #endif /* __DARWIN_64_BIT_INO_T */ ; #endif /* not POSIX */ DIR *opendir(const char *) __DARWIN_ALIAS_I(opendir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) DIR *__opendir2(const char *, int) __DARWIN_ALIAS_I(__opendir2); #endif /* not POSIX */ struct dirent *readdir(DIR *) __DARWIN_INODE64(readdir); int readdir_r(DIR *, struct dirent *, struct dirent **) __DARWIN_INODE64(readdir_r); void rewinddir(DIR *) __DARWIN_ALIAS_I(rewinddir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int scandir(const char *, struct dirent ***, int (*)(struct dirent *), int (*)(const void *, const void *)) __DARWIN_INODE64(scandir); #ifdef __BLOCKS__ int scandir_b(const char *, struct dirent ***, int (^)(struct dirent *), int (^)(const void *, const void *)) __DARWIN_INODE64(scandir_b) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2); #endif /* __BLOCKS__ */ #endif /* not POSIX */ void seekdir(DIR *, long) __DARWIN_ALIAS_I(seekdir); long telldir(DIR *) __DARWIN_ALIAS_I(telldir); __END_DECLS #endif /* !KERNEL */ -- Best regards, Igor Stasenko.
2012/6/2 Igor Stasenko <siguctua@gmail.com>:
On 2 June 2012 17:26, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Don't you think that this is what we are trying to do since ages: improving the infrastructure but not by piling junk on junk.
hehe stef.. I can say this happens everywhere. Piles of junk on top of each other..
Like figuring out, why output of readdir() function having strange results.. To figure it out, you have to read this code, and understand what's going on: Â this is part of standard C library on mac (/usr/include/dirent.h on your machine, you can check it)
__BEGIN_DECLS #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int alphasort(const void *, const void *) __DARWIN_INODE64(alphasort); #endif /* not POSIX */ int closedir(DIR *) __DARWIN_ALIAS(closedir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int getdirentries(int, char *, int, long *) #if __DARWIN_64_BIT_INO_T /*  * getdirentries() doesn't work when 64-bit inodes is in effect, so we  * generate a link error.  */                         __asm("_getdirentries_is_not_available_when_64_bit_inodes_are_in_effect") #else /* !__DARWIN_64_BIT_INO_T */                         __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0,__MAC_10_6, __IPHONE_2_0,__IPHONE_2_0) #endif /* __DARWIN_64_BIT_INO_T */ ; #endif /* not POSIX */ DIR *opendir(const char *) __DARWIN_ALIAS_I(opendir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) DIR *__opendir2(const char *, int) __DARWIN_ALIAS_I(__opendir2); #endif /* not POSIX */ struct dirent *readdir(DIR *) __DARWIN_INODE64(readdir); int readdir_r(DIR *, struct dirent *, struct dirent **) __DARWIN_INODE64(readdir_r); void rewinddir(DIR *) __DARWIN_ALIAS_I(rewinddir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int scandir(const char *, struct dirent ***,   int (*)(struct dirent *), int (*)(const void *, const void *)) __DARWIN_INODE64(scandir); #ifdef __BLOCKS__ int scandir_b(const char *, struct dirent ***,   int (^)(struct dirent *), int (^)(const void *, const void *)) __DARWIN_INODE64(scandir_b) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2); #endif /* __BLOCKS__ */ #endif /* not POSIX */ void seekdir(DIR *, long) __DARWIN_ALIAS_I(seekdir); long telldir(DIR *) __DARWIN_ALIAS_I(telldir); __END_DECLS
#endif /* !KERNEL */
Ouch, Igor, you gonna give us a headache. That's exatly the kind of niceties that makes FFI impracticle without a pre-processor, remember our discussion? Nicolas
On 2 June 2012 18:42, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote: ...
Ouch, Igor, you gonna give us a headache. That's exatly the kind of niceties that makes FFI impracticle without a pre-processor, remember our discussion?
That kind of niceties is a poor man's meta programming. I don't see it as an argument against FFI. For me it is a clear argument against system design. Because look.. most of the times you don't care what happens under the hood of your sport car.. until it stops working, or no longer meets your demands.
From position of consumer , it is clear what you do next - you go to market and buy a new one, or give it to engineers who can fix it. So, either there is someone who can produce better cars (engineers), or someone who can fix your car (engineers). But if there nobody left, and only consumers around, then you are in trouble.
And FFI actually points to that trouble: you cannot communicate with system without having a highly sophisticated compiler and preprocessor and also without knowledge about which bells you should ring and what whistles you should whistle.. this is road to nowhere. This is clearly not FFI's fault. It is fault of system design. For example, look at config.h file, which produced by autoconfig utility.. can you tell me, what all those flags controlling? Ok, name me the purpose of half of them.. Can you? And then tell me, how you can write a "highly portable" C software and guarantee that all those bells and whistles are correctly arranged?
From position of an engineer it is wrong to be on the side of consumers (i don't care, it is something under the hood of my car, which makes it run). And for sure, nobody cares about insignificant little fly such as Cog VM, except from few of us. So, you have to go and open a hood of a car and remove a rotten cats & rats from there, this is how you can move to better future. Otherwise at some point, cars will stop working, planes will stop flying, computers will stop working, because nobody will be left who would understand how to make or fix them.
So, this is a right question, posed by Stephane: how we can make sure that we will be able to improve system in future, once its creators are gone, and nobody can understand it anymore? Only by simplifying and cleaning all layers of a system and without relying on good Santa, which will come one day and clean this mess for us. -- Best regards, Igor Stasenko.
2012/6/3 Igor Stasenko <siguctua@gmail.com>:
On 2 June 2012 18:42, Nicolas Cellier <nicolas.cellier.aka.nice@gmail.com> wrote: ...
Ouch, Igor, you gonna give us a headache. That's exatly the kind of niceties that makes FFI impracticle without a pre-processor, remember our discussion?
That kind of niceties is a poor man's meta programming. I don't see it as an argument against FFI. For me it is a clear argument against system design.
Because look.. most of the times you don't care what happens under the hood of your sport car.. until it stops working, or no longer meets your demands. From position of consumer , it is clear what you do next - you go to market and buy a new one, or give it to engineers who can fix it. So, either there is someone who can produce better cars (engineers), or someone who can fix your car (engineers). But if there nobody left, and only consumers around, then you are in trouble.
And FFI actually points to that trouble: you cannot communicate with system without having a highly sophisticated compiler and preprocessor and also without knowledge about which bells you should ring and what whistles you should whistle.. this is road to nowhere.
This is clearly not FFI's fault. It is fault of system design. For example, look at config.h file, which produced by autoconfig utility.. can you tell me, what all those flags controlling? Ok, name me the purpose of half of them.. Can you? And then tell me, how you can write a "highly portable" C software and guarantee that all those bells and whistles are correctly arranged?
Sure, I don't blame FFI for being mud, just for exposing such mud. As I told you, I doubt I'd like it to be mirrored in image. The mud is everywhere, and there ain't much we can do. So how are we supposed to fix this? Throw away linux/FreeBSD/MacOS/WinXX and start an OS from scratch? Fight with more mud crystallised in hardware with SqueakNOS? This purpose of FFI is to open the gates to the external world, not to increase our autism. There is a solution in such cases: we can wrap mud in a nice packet by writing an interface library that uses highly sophisticated compiler and preprocessor, advanced knowledge of C code hackers, and community support for building various distributions (youpi with autoconf Cmake and the like)... It's a bit simpler than writing a plugin or VM code because no knowledge of Smalltalk/VMMaker/slang is required. But, not very much different at the end. Nicolas
From position of an engineer it is wrong to be on the side of consumers (i don't care, it is something under the hood of my car, which makes it run). And for sure, nobody cares about insignificant little fly such as Cog VM, except from few of us. So, you have to go and open a hood of a car and remove a rotten cats & rats from there, this is how you can move to better future. Otherwise at some point, cars will stop working, planes will stop flying, computers will stop working, because nobody will be left who would understand how to make or fix them.
So, this is a right question, posed by Stephane: how we can make sure that we will be able to improve system in future, once its creators are gone, and nobody can understand it anymore? Only by simplifying and cleaning all layers of a system and without relying on good Santa, which will come one day and clean this mess for us.
-- Best regards, Igor Stasenko.
Depends on
On Jun 2, 2012, at 5:52 PM, Igor Stasenko wrote:
On 2 June 2012 17:26, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
Don't you think that this is what we are trying to do since ages: improving the infrastructure but not by piling junk on junk.
hehe stef.. I can say this happens everywhere. Piles of junk on top of each other..
still my point is valid. How can we move on at the VM level by step by step improving it How can we make sure that the code of david from 2006 which speeds up directories will be integrated? How can we reduce the complexity of the vm? This is why I'm working on the consortium to get a chance to the community. Stef
Like figuring out, why output of readdir() function having strange results.. To figure it out, you have to read this code, and understand what's going on: this is part of standard C library on mac (/usr/include/dirent.h on your machine, you can check it)
__BEGIN_DECLS #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int alphasort(const void *, const void *) __DARWIN_INODE64(alphasort); #endif /* not POSIX */ int closedir(DIR *) __DARWIN_ALIAS(closedir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int getdirentries(int, char *, int, long *) #if __DARWIN_64_BIT_INO_T /* * getdirentries() doesn't work when 64-bit inodes is in effect, so we * generate a link error. */ __asm("_getdirentries_is_not_available_when_64_bit_inodes_are_in_effect") #else /* !__DARWIN_64_BIT_INO_T */ __OSX_AVAILABLE_BUT_DEPRECATED(__MAC_10_0,__MAC_10_6, __IPHONE_2_0,__IPHONE_2_0) #endif /* __DARWIN_64_BIT_INO_T */ ; #endif /* not POSIX */ DIR *opendir(const char *) __DARWIN_ALIAS_I(opendir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) DIR *__opendir2(const char *, int) __DARWIN_ALIAS_I(__opendir2); #endif /* not POSIX */ struct dirent *readdir(DIR *) __DARWIN_INODE64(readdir); int readdir_r(DIR *, struct dirent *, struct dirent **) __DARWIN_INODE64(readdir_r); void rewinddir(DIR *) __DARWIN_ALIAS_I(rewinddir); #if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE) int scandir(const char *, struct dirent ***, int (*)(struct dirent *), int (*)(const void *, const void *)) __DARWIN_INODE64(scandir); #ifdef __BLOCKS__ int scandir_b(const char *, struct dirent ***, int (^)(struct dirent *), int (^)(const void *, const void *)) __DARWIN_INODE64(scandir_b) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2); #endif /* __BLOCKS__ */ #endif /* not POSIX */ void seekdir(DIR *, long) __DARWIN_ALIAS_I(seekdir); long telldir(DIR *) __DARWIN_ALIAS_I(telldir); __END_DECLS
#endif /* !KERNEL */
-- Best regards, Igor Stasenko.
On 2 June 2012 17:01, David T. Lewis <lewis@mail.msen.com> wrote:
<OT> IMHO more people need to learn how to write small plugins. It is important for modularity and reliability, and for trying new VM experiments without waiting for a "VM guru" to help. Maybe you can also provide a way for people to add their experimental plugins to the Jenkins builds too, so that they can be made quickly available to anyone with an interest. </OT>
Dave
yes, i would wellcome that. And if you know, for instance we got this lately: https://ci.lille.inria.fr/pharo/view/Cog/job/CogDroid/ but sometimes i feel like crying it the desert. 1. There's not many people who can be attracted in hacking VM(s) 2. And even among those who can, not many feel excited about idea of improving and reimplementing it The principle "don't fix it if it not broken" works pretty well in VM domain , because of factor of 10 in having fun & time spent when hacking smalltalk code vs hacking VM code. -- Best regards, Igor Stasenko.
sure you are welcome. we would also love to get OSProcess plugin rewritten. Stef On Jun 1, 2012, at 10:47 PM, Chris Cunningham wrote:
On Fri, Jun 1, 2012 at 12:17 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
For those complaining lately about the new FS being a bit slow ;) Igor and I hacked together a test version for accessing files using NativeBoost.
From our stupid example (listing /usr/bin with ~1k Files) we were incredible 40 times! faster than the file primitives in the current image (5ms instead of 200ms).
With the current FS infrastructure such a migration can be done absolutely painless by simply replacing one back-end a time.
best cami
Can you share this work? I've been interested in this particular use for NB for a while now, but haven't gotten around to trying it (or, unfortunately, NB itself) yet. This would probably push me into a user role - and is definitely something I'd like to see.
-Chris
participants (8)
-
Camillo Bruni -
Chris Cunningham -
David T. Lewis -
Esteban Lorenzano -
Igor Stasenko -
Nicolas Cellier -
Sean P. DeNigris -
Stéphane Ducasse