- Pharo-dev - lists.pharo.org

Re: [Pharo-project] optimizing primitiveFindFirstInString

by Philippe Marschall

On 15.08.2011 00:21, Levente Uzonyi wrote: > On Sun, 14 Aug 2011, Philippe Marschall wrote: > >> On 14.08.2011 22:00, Levente Uzonyi wrote: >>> On Sun, 14 Aug 2011, Philippe Marschall wrote: >>> >>>> Hi >>>> >>>> In Seaside we get a lot of performance gains out of >>>> primitiveFindFirstInString. One thing that always annoyed me a bit is >>>> that it's not as optimized as it could be. >>>> >>>> The inclusionMap you give it are 256 consecutive boolean values (0 or >>>> 1). There is no need for this to be a 256 element ByteArray when each >>>> element can only be 0 or 1. We could as well make it a 32 element >>>> ByteArray and each byte holding eight bit values. Instead of using the >>>> asciiValue to directly index into the inclusionMap we would use the top >>>> five bits to index into the inclusionMap and the bottom 3 bits to >>>> "index >>>> into the byte". >>>> >>>> Did that make any sense? >>> >>> Do you want to save space? >> >> I'm trying to trade memory access (which is slow) for a bit shift and >> two bit ands (which is fast). > > 256 bytes + object header easily fit into the L1 cache of any recent x86 > CPU, even the old P4's had 8kB of it. Sure once it's in the cache then it's in the cache. But a cache line is only 64 bytes. So a 32 element array should fit and a 256 element doesn't. So a 32 element array should be there in one memory access while a 256 element array probably takes five. > Moving less bytes around may make > some difference though, but if you create your ByteArrays right before > you use them, then it's pretty likely that your object is already in the > cache. I don't of course. >> >>> Are you storing lots of inclusion maps (maybe >>> CharacterSets)? If not, then IMHO it's not worth to adding this feature >>> to this primitive, because runtime performance will be worse on both the >>> image side and the VM side. >> >> Why? > > Filling the contents of a compact 32 sized ByteArray in the image takes > more time than a 256 byte ByteArray. On the VM side memory access + > shifts + bitands cost more than just memory access. Uhm, it's a primitive. My understanding is this would result in "C shifts + bitands". > But (as usual) a > benchmark can tell if it's really worth to do it or not. This there somewhere some documentation how one adds a primitive and compiles a VM (Cog ideally)? Cheers Philippe

Aug. 15, 2011

Re: [Pharo-project] citezen-pier broken

by Damien Pollet

Let's have a look during ESUG, it shouldn't be that difficult to fix :) On 14 August 2011 22:11, Tudor Girba <tudor(a)tudorgirba.com> wrote: > Hi, > > Citezen-Pier does not seem to work anymore. The first thing I noticed is that it still relies on Rio for accessing the bib file. I guess we should update it to work with Filesystem. > > Anyone happens to have an interest to work on it? > > Cheers, > Doru > > > -- > www.tudorgirba.com > > "Some battles are better lost than fought." > > > > > -- Damien Pollet type less, do more [ | ] http://people.untyped.org/damien.pollet

Aug. 15, 2011

Re: [Pharo-project] optimizing primitiveFindFirstInString

by Levente Uzonyi

On Sun, 14 Aug 2011, Philippe Marschall wrote: > On 14.08.2011 22:00, Levente Uzonyi wrote: >> On Sun, 14 Aug 2011, Philippe Marschall wrote: >> >>> Hi >>> >>> In Seaside we get a lot of performance gains out of >>> primitiveFindFirstInString. One thing that always annoyed me a bit is >>> that it's not as optimized as it could be. >>> >>> The inclusionMap you give it are 256 consecutive boolean values (0 or >>> 1). There is no need for this to be a 256 element ByteArray when each >>> element can only be 0 or 1. We could as well make it a 32 element >>> ByteArray and each byte holding eight bit values. Instead of using the >>> asciiValue to directly index into the inclusionMap we would use the top >>> five bits to index into the inclusionMap and the bottom 3 bits to "index >>> into the byte". >>> >>> Did that make any sense? >> >> Do you want to save space? > > I'm trying to trade memory access (which is slow) for a bit shift and > two bit ands (which is fast). 256 bytes + object header easily fit into the L1 cache of any recent x86 CPU, even the old P4's had 8kB of it. Moving less bytes around may make some difference though, but if you create your ByteArrays right before you use them, then it's pretty likely that your object is already in the cache. > >> Are you storing lots of inclusion maps (maybe >> CharacterSets)? If not, then IMHO it's not worth to adding this feature >> to this primitive, because runtime performance will be worse on both the >> image side and the VM side. > > Why? Filling the contents of a compact 32 sized ByteArray in the image takes more time than a 256 byte ByteArray. On the VM side memory access + shifts + bitands cost more than just memory access. But (as usual) a benchmark can tell if it's really worth to do it or not. Levente > > Cheers > Philippe > > >

Aug. 14, 2011

Re: [Pharo-project] [Esug-list] [ANN] SmallHarbour ALPHA release

by Mariano Martinez Peck

Excllent!! This sounds awesome. Continue with your effort :) On Sun, Aug 14, 2011 at 11:23 PM, laurent laffont <laurent.laffont(a)gmail.com > wrote: > Hi, > > We are pleased to announce the first version of SmallHarbour. > > SmallHarbour is a public fork of SeasideHosting, running on top of Pharo, > Seaside and Cog - driven by Romain Verduci and Laurent Laffont for ESUG > SummerTalk 2011. > > > The main goals are: > - people should be able to (easily) setup their own web application > deployment platform > - provide a public service for hosting web applications. > - be able to setup a web application without Smalltalk knowledge in one > click. > > Actually SmallHarbour support Pharo and Seaside, but we expect to support > Iliad and AidaWeb framework soon. > > When an account is created, you can go in Home and click on "Create blog" > to deploy a Pier image. More templates to come. > > The alpha public server is at: http://www.alpha.smallharbour.org. Feel > free to upload images. BE CAREFUL: that may be deleted / lost without > warning. Do not put sensible informations. > > > To setup your own SmallHarbour, > - read documentation at > http://project.alpha.smallharbour.org/seaside/pier/Documentation > - see screencast > http://project.alpha.smallharbour.org/seaside/pier/blog/2011-08-11 > > Feedback is welcome. We still need to find bugs, fix them and learn. > > Romain will present SmallHarbour at ESUG. > > > Thanks ESUG, Adrian and Netstyle for sharing code, Mariano for help, Claire > and Patrick who are working on visual identity. > > Cheers, > > Laurent Laffont - @lolgzs <http://twitter.com/#%21/lolgzs> > > Pharo Smalltalk Screencasts: http://www.pharocasts.com/ > Blog: http://magaloma.blogspot.com/ > Developer group: http://cara74.seasidehosting.st > > > _______________________________________________ > Esug-list mailing list > Esug-list(a)lists.esug.org > http://lists.esug.org/mailman/listinfo/esug-list_lists.esug.org > > -- Mariano http://marianopeck.wordpress.com

Aug. 14, 2011

[Pharo-project] [ANN] SmallHarbour ALPHA release

by laurent laffont

Hi, We are pleased to announce the first version of SmallHarbour. SmallHarbour is a public fork of SeasideHosting, running on top of Pharo, Seaside and Cog - driven by Romain Verduci and Laurent Laffont for ESUG SummerTalk 2011. The main goals are: - people should be able to (easily) setup their own web application deployment platform - provide a public service for hosting web applications. - be able to setup a web application without Smalltalk knowledge in one click. Actually SmallHarbour support Pharo and Seaside, but we expect to support Iliad and AidaWeb framework soon. When an account is created, you can go in Home and click on "Create blog" to deploy a Pier image. More templates to come. The alpha public server is at: http://www.alpha.smallharbour.org. Feel free to upload images. BE CAREFUL: that may be deleted / lost without warning. Do not put sensible informations. To setup your own SmallHarbour, - read documentation at http://project.alpha.smallharbour.org/seaside/pier/Documentation - see screencast http://project.alpha.smallharbour.org/seaside/pier/blog/2011-08-11 Feedback is welcome. We still need to find bugs, fix them and learn. Romain will present SmallHarbour at ESUG. Thanks ESUG, Adrian and Netstyle for sharing code, Mariano for help, Claire and Patrick who are working on visual identity. Cheers, Laurent Laffont - @lolgzs <http://twitter.com/#!/lolgzs> Pharo Smalltalk Screencasts: http://www.pharocasts.com/ Blog: http://magaloma.blogspot.com/ Developer group: http://cara74.seasidehosting.st

Aug. 14, 2011

Re: [Pharo-project] optimizing primitiveFindFirstInString

by Philippe Marschall

On 14.08.2011 22:00, Levente Uzonyi wrote: > On Sun, 14 Aug 2011, Philippe Marschall wrote: > >> Hi >> >> In Seaside we get a lot of performance gains out of >> primitiveFindFirstInString. One thing that always annoyed me a bit is >> that it's not as optimized as it could be. >> >> The inclusionMap you give it are 256 consecutive boolean values (0 or >> 1). There is no need for this to be a 256 element ByteArray when each >> element can only be 0 or 1. We could as well make it a 32 element >> ByteArray and each byte holding eight bit values. Instead of using the >> asciiValue to directly index into the inclusionMap we would use the top >> five bits to index into the inclusionMap and the bottom 3 bits to "index >> into the byte". >> >> Did that make any sense? > > Do you want to save space? I'm trying to trade memory access (which is slow) for a bit shift and two bit ands (which is fast). > Are you storing lots of inclusion maps (maybe > CharacterSets)? If not, then IMHO it's not worth to adding this feature > to this primitive, because runtime performance will be worse on both the > image side and the VM side. Why? Cheers Philippe

Aug. 14, 2011

[Pharo-project] citezen-pier broken

by Tudor Girba

Hi, Citezen-Pier does not seem to work anymore. The first thing I noticed is that it still relies on Rio for accessing the bib file. I guess we should update it to work with Filesystem. Anyone happens to have an interest to work on it? Cheers, Doru -- www.tudorgirba.com "Some battles are better lost than fought."

Aug. 14, 2011

Re: [Pharo-project] [Vm-dev] building Cog on 64 bits debian

by Igor Stasenko

On 14 August 2011 18:39, laurent laffont <laurent.laffont(a)gmail.com> wrote: > > Thanks Levente. > After searching & trying finally I've passed this problem by: > mv /usr/bin/gcc /usr/bin/real.gcc > and then creating /usr/bin/gcc : > #!/bin/sh > exec real.gcc -m32 $@ > and chmod +x /usr/bin/gcc > > Now I have: > [ 94%] Built target vm-display-null > Scanning dependencies of target vm-display-X11 > [ 94%] Building C object vm-display-X11/CMakeFiles/vm-display-X11.dir/root/securedvm/platforms/unix/vm-display-X11/sqUnixX11.c.o > [ 95%] Building C object vm-display-X11/CMakeFiles/vm-display-X11.dir/root/securedvm/platforms/unix/vm-display-X11/sqUnixMozilla.c.o > Linking C shared library /root/securedvm/results/vm-display-X11 > /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../libSM.so when searching for -lSM > /usr/bin/ld: skipping incompatible /usr/lib/gcc/x86_64-linux-gnu/4.4.5/../../../libSM.a when searching for -lSM > /usr/bin/ld: skipping incompatible /usr/lib/libSM.so when searching for -lSM > /usr/bin/ld: skipping incompatible /usr/lib/libSM.a when searching for -lSM > /usr/bin/ld: cannot find -lSM > collect2: ld returned 1 exit status > looks like you don't have 32bit version of this library. That's why it refusing to link. As for compiler options, i don't think that you need to use workarounds (like creating real.gcc etc) , because the -m32 flag is there: # This is automatically generated file using CogUnixConfig on 14 August 2011 10:01:34 pm ... message("Adding module: vm-sound-null") add_definitions( -DLSB_FIRST=1 -DUSE_GLOBAL_STRUCT=0 -DCOGMTVM=0 -m32 -g0 -O2 -msse2 -D_GNU_SOURCE -DNDEBUG -DITIMER_HEARTBEAT=1 -DNO_VM_PROFILE=1 -DDEBUGVM=0) ... Try building using: make VERBOSE=1 to see what flags are passed to compiler. And please, create a subclass of CogUnixConfig , like Cog32OnDebian64Config and play with definitions there. In image you can do: Cog32OnDebian64Config generateWithSources or Cog32OnDebian64Config generate the difference that first will generate cmake files + source files, while second one is only cmake files (takes less than second) I suspecting that problem with linker flags, which tries to link 32-bit output with 64-bit binary. It seems that we're missing the flag there that it should use 32-bit binary for output. > Laurent > > On Sat, Aug 13, 2011 at 4:36 PM, Levente Uzonyi <leves(a)elte.hu> wrote: >> >> On Sat, 13 Aug 2011, laurent laffont wrote: >> >> (no quotes thx to gmail...) >> >> This issue was discussed several times. The simplest solution is to define CC as "gcc -m32" (and optionally CXX as "g++ -m32" if you need g++). Another solution is to check how cmake supports cross compilation. IIRC autotools has some flags for this. >> >> >> Levente > > > -- Best regards, Igor Stasenko AKA sig.

Aug. 14, 2011

Re: [Pharo-project] optimizing primitiveFindFirstInString

by Levente Uzonyi

On Sun, 14 Aug 2011, Philippe Marschall wrote: > Hi > > In Seaside we get a lot of performance gains out of > primitiveFindFirstInString. One thing that always annoyed me a bit is > that it's not as optimized as it could be. > > The inclusionMap you give it are 256 consecutive boolean values (0 or > 1). There is no need for this to be a 256 element ByteArray when each > element can only be 0 or 1. We could as well make it a 32 element > ByteArray and each byte holding eight bit values. Instead of using the > asciiValue to directly index into the inclusionMap we would use the top > five bits to index into the inclusionMap and the bottom 3 bits to "index > into the byte". > > Did that make any sense? Do you want to save space? Are you storing lots of inclusion maps (maybe CharacterSets)? If not, then IMHO it's not worth to adding this feature to this primitive, because runtime performance will be worse on both the image side and the VM side. Levente > > Cheers > Philippe > > >

Aug. 14, 2011

[Pharo-project] Fwd: [Vm-dev] Socket's readSemaphore is losing signals with Cog on Linux

by StÃ©phane Ducasse

can be a valuable information for some of you. Would be good to fix it for real. Stef Begin forwarded message: > From: Andreas Raab <andreas.raab(a)gmx.de> > Date: August 14, 2011 7:58:06 PM GMT+02:00 > To: vm-dev(a)lists.squeakfoundation.org > Subject: Re: [Vm-dev] Socket's readSemaphore is losing signals with Cog on Linux > Reply-To: Squeak Virtual Machine Development Discussion <vm-dev(a)lists.squeakfoundation.org> > > On 8/13/2011 13:42, Levente Uzonyi wrote: >> Socket's readSemaphore is losing signals with CogVMs on linux. We found several cases (RFB, PostgreSQL) when processes are stuck in the following method: >> >> Socket >> waitForDataIfClosed: closedBlock >> "Wait indefinitely for data to arrive. This method will block until >> data is available or the socket is closed." >> >> [ >> (self primSocketReceiveDataAvailable: socketHandle) >> ifTrue: [^self]. >> self isConnected >> ifFalse: [^closedBlock value]. >> self readSemaphore wait ] repeat >> >> When we inspect the contexts, the process is waiting for the readSemaphore, but evaluating (self primSocketReceiveDataAvailable: socketHandle) yields true. Signaling the readSemaphore makes the process running again. As a workaround we replaced #wait with #waitTimeoutMSecs: and all our problems disappeared. >> >> The interpreter VM doesn't seem to have this bug, so I guess the bug was introduced with the changes of aio.c. > > Oh, interesting. We know this problem fairly well and have always worked around by changing the wait in the above to a "waitTimeoutMSecs: 500" which turns it into a soft busy loop. It would be interesting to see if there's a bug in Cog which causes this. FWIW, here is the relevant portion: > > "Soft 500ms busy loop - to protect against AIO probs; > occasionally, VM-level AIO fails to trip the semaphore" > self readSemaphore waitTimeoutMSecs: 500. > > Cheers, > - Andreas >

Aug. 14, 2011