December 2012 - Pharo-dev - lists.pharo.org

[Pharo-project] using a long running native thread - NativeBoost or Plugin?

by drush66

Would it be possible to start native thread inside pharo? This thread would run (for quite a long time) some event loop (like the libev), and post some info to the SharedQueue. Smalltalk code would in turn be reading stuff from this queue. You could say it would be pumping events from libev to SharedQueue instance for Smalltalk code to consume. Is this doable? With NativeBoost or Plugin? I have also found some info on threaded VM, but if I understand it correctly it seems more oriented to provide a thread to execute FFI call on it. ----- http://www.cloud208.com/ -- View this message in context: http://forum.world.st/using-a-long-running-native-thread-NativeBoost-or-Plu… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Dec. 2, 2012

Re: [Pharo-project] Performance Tip: Using buffered streams

by Mariano Martinez Peck

On Sun, Dec 2, 2012 at 11:09 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Hi, > > <short-version> > > When doing complex IO, like writing or parsing protocols, to non-memory > streams, like file streams (and probably socket streams), buffering can > make an important difference. ZnBufferedWriteStream and > ZnBufferedReadStream are an easy solution to experience the difference. > Yes, I noticed that in Fuel and that's why we use the buffered write stream always. It was a huge difference. > > </short-version> > > And now the long version: first some measurements (code based on latest > versions of Zinc, NeoCSV, NeoJSON and STON): > > NeoJSON > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.json' asFileReference writeStreamDo: [ :stream | > (NeoJSONWriter on: stream) nextPut: data ] ] timeToRun. 22152 > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.json' asFileReference writeStreamDo: [ :fstream | > ZnBufferedWriteStream on: fstream do: [ :stream | > (NeoJSONWriter on: stream) nextPut: data ] ] ] timeToRun. > 5147 > > [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | > (NeoJSONReader on: stream) next ] ] timeToRun. 3502 > > [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | > ZnBufferedReadStream on: fstream do: [ :stream | > (NeoJSONReader on: stream) next ] ] ] timeToRun. 1214 > > STON > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.ston' asFileReference writeStreamDo: [ :stream | > STON writer on: stream; nextPut: data ] ] timeToRun. 22275 > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.ston' asFileReference writeStreamDo: [ :fstream | > ZnBufferedWriteStream on: fstream do: [ :stream | > STON writer on: stream; nextPut: data ] ] ] timeToRun. 5501 > > [ '/tmp/numbers.ston' asFileReference readStreamDo: [ :stream | > STON reader on: stream; next ] ] timeToRun. 3632 > > [ '/tmp/numbers.ston' asFileReference readStreamDo: [ :fstream | > ZnBufferedReadStream on: fstream do: [ :stream | > STON reader on: stream; next ] ] ] timeToRun. 1367 > > NeoCSV > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :stream | > (NeoCSVWriter on: stream) nextPutAll: data ] ] timeToRun. 20916 > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each reciprocal asFloat. each sqrt. each > asWords } ]. > [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :fstream | > ZnBufferedWriteStream on: fstream do: [ :stream | > (NeoCSVWriter on: stream) nextPutAll: data ] ] ] > timeToRun. 5559 > > [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :stream | > (NeoCSVReader on: stream) > addIntegerField; addIntegerField; addFloatField; > addFloatField; addField; > upToEnd ] ] timeToRun. 1290 > > [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :fstream | > ZnBufferedReadStream on: fstream do: [ :stream | > (NeoCSVReader on: stream) > addIntegerField; addIntegerField; addFloatField; > addFloatField; addField; > upToEnd ] ] ] timeToRun. 970 > > > The reason for these non-trivial speedups is that buffering is missing > and/or buffer management is suboptimal in the standard file stream classes. > Simply wrapping the stream with a 64Kb buffer yields results like this. We > already knew this for writing (and Fuel adapted it as well), but it seems > to make a similar, although smaller difference when reading. > This is really really interesting, because it means a speed up of 2x when reading :) Where can I get the last version of ZnBufferedReadStream? > > <side-note> > > Attentive, curious readers might ask why in this particular benchmark the > reading is so much slower than the writing - normally it should be the > inverse or at least they should be closer to each other. Lo and behold: > another performance issue: float printing (as opposed to parsing) is > extremely slow (probably partially due to its reliance on LargeInteger > arithmetic) ! > I remember Nicolas Cieller doing something about Floats and printing...maybe he improved something? > > Here is a similar benchmark with only Integers: > > | data | > data := (1 to: 25000) collect: [ :each | > { each. each negated. each + 100000. each - 100000. each asWords } > ]. > [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :fstream | > ZnBufferedWriteStream on: fstream do: [ :stream | > (NeoCSVWriter on: stream) nextPutAll: data ] ] ] > timeToRun. 311 > > [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :fstream | > ZnBufferedReadStream on: fstream do: [ :stream | > (NeoCSVReader on: stream) > addIntegerField; addIntegerField; addIntegerField; > addIntegerField; addField; > upToEnd ] ] ] timeToRun. 396 > > </side-note> > > If ZnBufferedWriteStream or ZnBufferedReadStream were useful in speading > up your code, please let us know. > > Sven > > PS: There is one ZnBufferedReadStream limitation: by design, #position, > #position: or #skip: -1 and everything on top of that are NOT supported - > IMHO real streams should not (and cannot) provide these operations. A sad > consequence is that NumberParser and its subclasses (and thus Number > class>>#readFrom:) do not work. NeoJSON and STON do their own number > parsing, NeoCSV relies on a new class, NeoNumberParser, that is > independently usable, though restricted to normal numbers as opposed to the > full Smalltalk syntax. > > I want to give it a try for Fuel. > -- > Sven Van Caekenberghe > http://stfx.eu > Smalltalk is the Red Pill > > > > > -- Mariano http://marianopeck.wordpress.com

Dec. 2, 2012

[Pharo-project] Performance Tip: Using buffered streams

by Sven Van Caekenberghe

Hi, <short-version> When doing complex IO, like writing or parsing protocols, to non-memory streams, like file streams (and probably socket streams), buffering can make an important difference. ZnBufferedWriteStream and ZnBufferedReadStream are an easy solution to experience the difference. </short-version> And now the long version: first some measurements (code based on latest versions of Zinc, NeoCSV, NeoJSON and STON): NeoJSON | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.json' asFileReference writeStreamDo: [ :stream | (NeoJSONWriter on: stream) nextPut: data ] ] timeToRun. 22152 | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.json' asFileReference writeStreamDo: [ :fstream | ZnBufferedWriteStream on: fstream do: [ :stream | (NeoJSONWriter on: stream) nextPut: data ] ] ] timeToRun. 5147 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :stream | (NeoJSONReader on: stream) next ] ] timeToRun. 3502 [ '/tmp/numbers.json' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoJSONReader on: stream) next ] ] ] timeToRun. 1214 STON | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.ston' asFileReference writeStreamDo: [ :stream | STON writer on: stream; nextPut: data ] ] timeToRun. 22275 | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.ston' asFileReference writeStreamDo: [ :fstream | ZnBufferedWriteStream on: fstream do: [ :stream | STON writer on: stream; nextPut: data ] ] ] timeToRun. 5501 [ '/tmp/numbers.ston' asFileReference readStreamDo: [ :stream | STON reader on: stream; next ] ] timeToRun. 3632 [ '/tmp/numbers.ston' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | STON reader on: stream; next ] ] ] timeToRun. 1367 NeoCSV | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :stream | (NeoCSVWriter on: stream) nextPutAll: data ] ] timeToRun. 20916 | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each reciprocal asFloat. each sqrt. each asWords } ]. [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :fstream | ZnBufferedWriteStream on: fstream do: [ :stream | (NeoCSVWriter on: stream) nextPutAll: data ] ] ] timeToRun. 5559 [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :stream | (NeoCSVReader on: stream) addIntegerField; addIntegerField; addFloatField; addFloatField; addField; upToEnd ] ] timeToRun. 1290 [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoCSVReader on: stream) addIntegerField; addIntegerField; addFloatField; addFloatField; addField; upToEnd ] ] ] timeToRun. 970 The reason for these non-trivial speedups is that buffering is missing and/or buffer management is suboptimal in the standard file stream classes. Simply wrapping the stream with a 64Kb buffer yields results like this. We already knew this for writing (and Fuel adapted it as well), but it seems to make a similar, although smaller difference when reading. <side-note> Attentive, curious readers might ask why in this particular benchmark the reading is so much slower than the writing - normally it should be the inverse or at least they should be closer to each other. Lo and behold: another performance issue: float printing (as opposed to parsing) is extremely slow (probably partially due to its reliance on LargeInteger arithmetic) ! Here is a similar benchmark with only Integers: | data | data := (1 to: 25000) collect: [ :each | { each. each negated. each + 100000. each - 100000. each asWords } ]. [ '/tmp/numbers.csv' asFileReference writeStreamDo: [ :fstream | ZnBufferedWriteStream on: fstream do: [ :stream | (NeoCSVWriter on: stream) nextPutAll: data ] ] ] timeToRun. 311 [ '/tmp/numbers.csv' asFileReference readStreamDo: [ :fstream | ZnBufferedReadStream on: fstream do: [ :stream | (NeoCSVReader on: stream) addIntegerField; addIntegerField; addIntegerField; addIntegerField; addField; upToEnd ] ] ] timeToRun. 396 </side-note> If ZnBufferedWriteStream or ZnBufferedReadStream were useful in speading up your code, please let us know. Sven PS: There is one ZnBufferedReadStream limitation: by design, #position, #position: or #skip: -1 and everything on top of that are NOT supported - IMHO real streams should not (and cannot) provide these operations. A sad consequence is that NumberParser and its subclasses (and thus Number class>>#readFrom:) do not work. NeoJSON and STON do their own number parsing, NeoCSV relies on a new class, NeoNumberParser, that is independently usable, though restricted to normal numbers as opposed to the full Smalltalk syntax. -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Dec. 2, 2012

Re: [Pharo-project] CI server down?

by Sebastian Nozzi

Re-forwarding to the list. Might be useful. 2012/12/2 Camillo Bruni <camillobruni(a)gmail.com>: > With a change of repeating myself, this is the main use case of http://pharo.gforge.inria.fr/ci > > wget --quiet -qO - http://pharo.gforge.inria.fr/ci/ciPharo20NBCog.sh | sh > > ./vm.sh Pharo.image save $JOB_NAME > > REPO=http://squeaksource.com/Athens > ./vm.sh $JOB_NAME.image config $REPO ConfigurationOfAthens --install=1.4 > ./vm.sh $JOB_NAME.image test --junit-xml-output "Athens-.*" > > => Zero-Conf setup with the latest VMs and the Latest images > > On 2012-12-02, at 19:03, Sebastian Nozzi <sebnozzi(a)gmail.com> wrote: >>> so don't use it to get images. All images and VMs are available offsite at >>> http://pharo.gforge.inria.fr/ci/ >> >> Cool, that's *exactly* what I was using it for... thanks for the tip! :-) >

Dec. 2, 2012

Re: [Pharo-project] CI server down?

by Camillo Bruni

yes it's dead, as usual... the old ci is veeeery unstable. so don't use it to get images. All images and VMs are available offsite at http://pharo.gforge.inria.fr/ci/ => no https transfers => faster download speed => higher availability => simple predictable urls On 2012-12-02, at 18:51, Sebastian Nozzi <sebnozzi(a)gmail.com> wrote: > I cannot access: > > http://ci.lille.inria.fr/pharo/ > > (anyone has nagios on that? ;-) >

Dec. 2, 2012

[Pharo-project] CI server down?

by Sebastian Nozzi

I cannot access: http://ci.lille.inria.fr/pharo/ (anyone has nagios on that? ;-)

Dec. 2, 2012

Re: [Pharo-project] just help :)

by Mariano Martinez Peck

On Fri, Nov 30, 2012 at 10:06 AM, Sebastian Nozzi <sebnozzi(a)gmail.com>wrote: > 2012/11/30 Marcus Denker <marcus.denker(a)inria.fr>: > > On Nov 29, 2012, at 11:55 PM, dimitris chloupis <thekilon(a)yahoo.co.uk> > > wrote: > > > > * Framework for seamless data/object-synchronization. I suggested this a > > while ago, didn't get much response. I don't blame anyone, there is not > much > > need for this right now. What I would want is to be able to modify > certain > > objects in my image, while offline, and connect to the Internet, and > these > > objects be synchronized between other images. This way Internet-enabled > apps > > could be built on top of Pharo in a better way that web-apps are built > now > > (because, in Pharo you can not hit F5 and get fresh data ;-). But this is > > low-priority for me." > > > > This is a hard problemâ¦ but if would be nice to have, yes. > > I think it would make a nice research project ;-) > > Maybe as an extension(*) to Fuel/Tanker? > I guess Fuel/Tanker also has to deal with conflicts, right? > > Right. This already happens and Fuel deals with some of them. > > > * - "Extension" or "on-top-of" > > -- Mariano http://marianopeck.wordpress.com

Dec. 2, 2012

Re: [Pharo-project] [[ false ] whileFalse: [ ]] in workspace

by Guillermo Polito

Hi! I've hacked a fix, but I don't know if it is correct... can somebody check it so I make a Slice? Guille On Sun, Dec 2, 2012 at 8:56 PM, Guillermo Polito <guillermopolito(a)gmail.com>wrote: > Ok, I've found the cause :). > > From update 20043: > > Issue 5744: Clean up Temp Embedding 2): DoIts > http://code.google.com/p/pharo/issues/detail?id=5744 > > > - Doits are now compiled with source embedded > - embedding sources for doits pretty prints from AST as doits are compiled > specially (with a return added) > > CompiledMethod: > > - remove blockExtendsToTempsMap: Not needed anymore, all calls go the > MethodNode > - remove #copyWitTempsFromMethodNode: All senders use #copyWithSource > - remove #holdsTempNames. Temp name embedding not needed > - add isDoit > - schematicTempNamesString always returns the schematic string > representation of all temps > - tempNames forwards to methodNode > - remove tempNamesString. Concatenate yourself if you need that. > > Debugger>>selectedMessage asks the selectedContext directly for the > source. No need for the methodMap. > > DebuggerMethodMap > > - no special handling of methods that encode temps. Just as for methodNode > always. > > MethodNode > > - generateWithSource. embedd pretty printed code in case of doit > - remove generateWithTempNames > > In general: rewrite code to use #generateWithSource > > > On Sat, Dec 1, 2012 at 6:24 PM, StÃ©phane Ducasse < > stephane.ducasse(a)inria.fr> wrote: > >> Sebastian >> >> I can tell you that what you did is **useful** because we are all working >> like mad >> and such kind of help is really welcome. >> >> Stef >> >> On Dec 1, 2012, at 3:24 PM, Sebastian Nozzi wrote: >> >> > It's the #7076. >> > >> > 2012/11/29 Camillo Bruni <camillobruni(a)gmail.com>: >> >> perfect! can you open a bug report with this information? :) >> > >> >> >> >

Dec. 2, 2012

Re: [Pharo-project] DynamicVariable>>value:during:

by Sven Van Caekenberghe

Yes, I think so too, but maybe there was some reason for not doing it ? On 02 Dec 2012, at 20:52, Sebastian Sastre <sebastian(a)flowingconcept.com> wrote: > Hey guys, > > would not make much much more sense if we return the result of evaluating the block when we use dynamic variables? > > Here is the patch I'm using these days: > > DynamicVariable>>value: anObject during: aBlock > > | p oldValue result | > p := Processor activeProcess. > oldValue := (p psValueAt: index) ifNil: [self default]. > [ > p psValueAt: index put: anObject. > result := aBlock value. > ] ensure: [ > p psValueAt: index put: oldValue > ]. > > ^ result -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Dec. 2, 2012

Re: [Pharo-project] [[ false ] whileFalse: [ ]] in workspace

by Mariano Martinez Peck

On Sat, Dec 1, 2012 at 6:24 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>wrote: > Sebastian > > I can tell you that what you did is **useful** because we are all working > like mad > and such kind of help is really welcome. > > +9999 > Stef > > On Dec 1, 2012, at 3:24 PM, Sebastian Nozzi wrote: > > > It's the #7076. > > > > 2012/11/29 Camillo Bruni <camillobruni(a)gmail.com>: > >> perfect! can you open a bug report with this information? :) > > > > > -- Mariano http://marianopeck.wordpress.com

Dec. 2, 2012