July 2013 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Pharo performance

by Chris

On 29/07/2013 21:33, Sven Van Caekenberghe wrote: > Chris, > > On 29 Jul 2013, at 20:52, Chris <cpmbailey(a)btinternet.com> wrote: > >> I've been getting a little concerned with certain aspects of performance recently. Just a couple of examples off the top of my head were trying to do a printString on 200000 floats which takes over 3 seconds. If I do the same in Python it is only 0.25 seconds. Similarly reading 65000 points from a database with the PostgresV2 driver was about 800m/s and only 40 with psycopg. I'd have to try it again but am pretty sure going native was faster than OpenDBX as well. I appreciate Pharo is never going to be able to compete with the static-typed heavyweight languages but would hope we can get performance at least comparable to other dynamic languages :) Is it just that some method implementations are in need of some TLC; more things moved on top of C libraries and primitives and so forth rather than anything with the VM itself? >> >> Cheers >> Chris > You could try: > > | floats | > floats := (1 to: 200000) collect: #asFloat. > [ FloatPrintPolicy > value: InexactFloatPrintPolicy new > during: [ > String new: 1500000 streamContents: [ :stream | > floats do: [ :each | each printOn: stream ] ] ] ] timeToRun > > => 796 ms > > I haven't looked at the Postgresql driver in detail, but I would guess that PostgresV2 reads from a String somewhere, which is slow unless care is taken, while psycopg probably does a binary read. That last thing can be done in Pharo as well, but not if the driver is text based somehow. > > You are right: Pharo should definitively be in the same range as other dynamically typed languages. But dynamic languages are dangerous: user become lazy / ignorant about performance. > > Thanks for pushing this. > > Sven > > -- > Sven Van Caekenberghe > http://stfx.eu > Smalltalk is the Red Pill > > > Thanks for the float tip :) I think I need to investigate the Postgres thing a bit further. I thought it was a fairly native driver but the double array type may well be using a string

July 29, 2013

Re: [Pharo-users] Pharo performance

by Sven Van Caekenberghe

Chris, On 29 Jul 2013, at 20:52, Chris <cpmbailey(a)btinternet.com> wrote: > I've been getting a little concerned with certain aspects of performance recently. Just a couple of examples off the top of my head were trying to do a printString on 200000 floats which takes over 3 seconds. If I do the same in Python it is only 0.25 seconds. Similarly reading 65000 points from a database with the PostgresV2 driver was about 800m/s and only 40 with psycopg. I'd have to try it again but am pretty sure going native was faster than OpenDBX as well. I appreciate Pharo is never going to be able to compete with the static-typed heavyweight languages but would hope we can get performance at least comparable to other dynamic languages :) Is it just that some method implementations are in need of some TLC; more things moved on top of C libraries and primitives and so forth rather than anything with the VM itself? > > Cheers > Chris You could try: | floats | floats := (1 to: 200000) collect: #asFloat. [ FloatPrintPolicy value: InexactFloatPrintPolicy new during: [ String new: 1500000 streamContents: [ :stream | floats do: [ :each | each printOn: stream ] ] ] ] timeToRun => 796 ms I haven't looked at the Postgresql driver in detail, but I would guess that PostgresV2 reads from a String somewhere, which is slow unless care is taken, while psycopg probably does a binary read. That last thing can be done in Pharo as well, but not if the driver is text based somehow. You are right: Pharo should definitively be in the same range as other dynamically typed languages. But dynamic languages are dangerous: user become lazy / ignorant about performance. Thanks for pushing this. Sven -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

July 29, 2013

Re: [Pharo-users] Pharo performance

by Mariano Martinez Peck

I remember Nicolas Cieller did something about improving performance of Float printing. Not sure what is the state yet.... On Mon, Jul 29, 2013 at 4:50 PM, Chris <cpmbailey(a)btinternet.com> wrote: > I often use the profiler :) > > The float printString is all fairly evenly split which is why I mention > about whether another implementation may be required. I'll always raise > anything much more obvious that I see! > > Hi Chris, > > My recommendation in this case is always do not spend a single second > trying to figure out what is the bottleneck by yourself. First thing ever > to do, is to run a profiler. You can do that very easily in Pharo. > > TimeProfiler spyOn: [ Transcript show: 'do something for real here to > benchmark'. > Object allSubclasses. ] > > replace the closure for what you want to benchmark. > > Then, share with us if you have interesting finds.... > > Cheers, > > > > On Mon, Jul 29, 2013 at 3:52 PM, Chris <cpmbailey(a)btinternet.com> wrote: > >> I've been getting a little concerned with certain aspects of performance >> recently. Just a couple of examples off the top of my head were trying to >> do a printString on 200000 floats which takes over 3 seconds. If I do the >> same in Python it is only 0.25 seconds. Similarly reading 65000 points from >> a database with the PostgresV2 driver was about 800m/s and only 40 with >> psycopg. I'd have to try it again but am pretty sure going native was >> faster than OpenDBX as well. I appreciate Pharo is never going to be able >> to compete with the static-typed heavyweight languages but would hope we >> can get performance at least comparable to other dynamic languages :) Is it >> just that some method implementations are in need of some TLC; more things >> moved on top of C libraries and primitives and so forth rather than >> anything with the VM itself? >> >> Cheers >> Chris >> >> > > > -- > Mariano > http://marianopeck.wordpress.com > > > -- Mariano http://marianopeck.wordpress.com

July 29, 2013

Re: [Pharo-users] Pharo performance

by Chris

I often use the profiler :) The float printString is all fairly evenly split which is why I mention about whether another implementation may be required. I'll always raise anything much more obvious that I see! > Hi Chris, > > My recommendation in this case is always do not spend a single second > trying to figure out what is the bottleneck by yourself. First thing > ever to do, is to run a profiler. You can do that very easily in Pharo. > > TimeProfiler spyOn: [ Transcript show: 'do something for real here to > benchmark'. > Object allSubclasses. ] > > replace the closure for what you want to benchmark. > > Then, share with us if you have interesting finds.... > > Cheers, > > > > On Mon, Jul 29, 2013 at 3:52 PM, Chris <cpmbailey(a)btinternet.com > <mailto:cpmbailey@btinternet.com>> wrote: > > I've been getting a little concerned with certain aspects of > performance recently. Just a couple of examples off the top of my > head were trying to do a printString on 200000 floats which takes > over 3 seconds. If I do the same in Python it is only 0.25 > seconds. Similarly reading 65000 points from a database with the > PostgresV2 driver was about 800m/s and only 40 with psycopg. I'd > have to try it again but am pretty sure going native was faster > than OpenDBX as well. I appreciate Pharo is never going to be able > to compete with the static-typed heavyweight languages but would > hope we can get performance at least comparable to other dynamic > languages :) Is it just that some method implementations are in > need of some TLC; more things moved on top of C libraries and > primitives and so forth rather than anything with the VM itself? > > Cheers > Chris > > > > > -- > Mariano > http://marianopeck.wordpress.com

July 29, 2013

Re: [Pharo-users] Pharo performance

by Mariano Martinez Peck

Hi Chris, My recommendation in this case is always do not spend a single second trying to figure out what is the bottleneck by yourself. First thing ever to do, is to run a profiler. You can do that very easily in Pharo. TimeProfiler spyOn: [ Transcript show: 'do something for real here to benchmark'. Object allSubclasses. ] replace the closure for what you want to benchmark. Then, share with us if you have interesting finds.... Cheers, On Mon, Jul 29, 2013 at 3:52 PM, Chris <cpmbailey(a)btinternet.com> wrote: > I've been getting a little concerned with certain aspects of performance > recently. Just a couple of examples off the top of my head were trying to > do a printString on 200000 floats which takes over 3 seconds. If I do the > same in Python it is only 0.25 seconds. Similarly reading 65000 points from > a database with the PostgresV2 driver was about 800m/s and only 40 with > psycopg. I'd have to try it again but am pretty sure going native was > faster than OpenDBX as well. I appreciate Pharo is never going to be able > to compete with the static-typed heavyweight languages but would hope we > can get performance at least comparable to other dynamic languages :) Is it > just that some method implementations are in need of some TLC; more things > moved on top of C libraries and primitives and so forth rather than > anything with the VM itself? > > Cheers > Chris > > -- Mariano http://marianopeck.wordpress.com

July 29, 2013

Pharo performance

by Chris

I've been getting a little concerned with certain aspects of performance recently. Just a couple of examples off the top of my head were trying to do a printString on 200000 floats which takes over 3 seconds. If I do the same in Python it is only 0.25 seconds. Similarly reading 65000 points from a database with the PostgresV2 driver was about 800m/s and only 40 with psycopg. I'd have to try it again but am pretty sure going native was faster than OpenDBX as well. I appreciate Pharo is never going to be able to compete with the static-typed heavyweight languages but would hope we can get performance at least comparable to other dynamic languages :) Is it just that some method implementations are in need of some TLC; more things moved on top of C libraries and primitives and so forth rather than anything with the VM itself? Cheers Chris

July 29, 2013

Re: [Pharo-users] [Voyage] Using OID as a domain object identifier

by Norbert Hartl

Am 29.07.2013 um 19:34 schrieb Esteban A. Maringolo <emaringolo(a)gmail.com>: > 2013/7/29 Norbert Hartl <norbert(a)hartl.name>: >> Am 29.07.2013 um 15:46 schrieb "Esteban A. Maringolo" <emaringolo(a)gmail.com>: >>> Is it a good practice to use the OID value (anInteger) as an >>> "external" identifier of your domain object? >>> ... >>> I like OID/UUIDs, and I don't want to duplicate that logic in my app. >>> >> I think for your purpose OID will be fine but I'm not sure. I use UUIDs. You can instruct mongo to use a UUID instead of an OID. Works flawlessly. > > How do you do that? Because OID's are a BSON native artifact, > efficient in terms of storage and lookup (a 12 byte integer in the > end...) > > Is there a way to have an smaller UUID? more like the first eight > chars of a Git commit or similarâ¦ If you look at [1] you can see that UUID is a native BSON artefact as well. Pharo has a UUID class. I extended the Mongo driver to serialize UUID properly. So basically all you need is to put a UUID object in your model object and serialization should be automatic. Of course you need to tell the serialization layer to serialize it as _id. But it is fully supported on mongo. Performance wise it is said it might be a tad slower than object ids. But if you have doubts about object ids you should be willing to take the payoff. I can't help here much because I have my own serialization layer that is pretty much superseded by Voyage. Do you have really performance problems why you like to restrict the key size? That would be a bad because you would restrict the uniqueness to a useless artefact. IIRC mongo does that for you anyway. There is a certain size of key mongo evaluates first and rechecks on collision. But I'm not sure I had only the impression the same techniques are in charge as for other hashing problems. hope this helps, Norbert [1] http://bsonspec.org/#/specification

July 29, 2013

Re: [Pharo-users] [Voyage] Using OID as a domain object identifier

by Esteban A. Maringolo

2013/7/29 Norbert Hartl <norbert(a)hartl.name>: > Am 29.07.2013 um 15:46 schrieb "Esteban A. Maringolo" <emaringolo(a)gmail.com>: >> Is it a good practice to use the OID value (anInteger) as an >> "external" identifier of your domain object? >> ... >> I like OID/UUIDs, and I don't want to duplicate that logic in my app. >> > I think for your purpose OID will be fine but I'm not sure. I use UUIDs. You can instruct mongo to use a UUID instead of an OID. Works flawlessly. How do you do that? Because OID's are a BSON native artifact, efficient in terms of storage and lookup (a 12 byte integer in the end...) Is there a way to have an smaller UUID? more like the first eight chars of a Git commit or similar... Regards, Esteban A. Maringolo

July 29, 2013

Re: [Pharo-users] Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

by phil＠highoctane.be

The image and the changes file are here: (uploading... will take a while - 30 min or so) https://www.dropbox.com/sh/wfxep3jm5v27szt/5keLUTFvGz Phil On Mon, Jul 29, 2013 at 5:06 PM, Sebastian Tleye <stleye(a)gmail.com> wrote: > Yes please (if it is not too much work :) ) > I would like to check something. > > > 2013/7/29 phil(a)highoctane.be <phil(a)highoctane.be> > >> I can send you my image but need to remove all my credentials from there >> first. >> >> --- >> Philippe Back >> Dramatic Performance Improvements >> Mob: +32(0) 478 650 140 | Fax: +32 (0) 70 408 027 >> Mail:phil@highoctane.be | Web: http://philippeback.eu >> Blog: http://philippeback.be | Twitter: @philippeback >> Youtube: http://www.youtube.com/user/philippeback/videos >> >> High Octane SPRL >> rue cour Boisacq 101 | 1301 Bierges | Belgium >> >> Featured on the Software Process and Measurement Cast >> http://spamcast.libsyn.com >> Sparx Systems Enterprise Architect and Ability Engineering EADocX Value >> Added Reseller >> >> >> >> >> On Mon, Jul 29, 2013 at 4:43 PM, StÃ©phane Ducasse < >> stephane.ducasse(a)inria.fr> wrote: >> >>> >> >>> >> Look at the top consumers, especially the Semaphore thing. It is way >>> beyond whatever was expected. There is something wrong there. >>> >> >>> >> There must be a leak of Points, MorphExtensions and I can't just >>> figure out why there are so many Floats. >>> >> >>> > MorphExtension should only be referenced by the Morph it extends. So >>> there should always be more Morphs than extensions. >>> > This looks really strange. >>> >>> yes this is really strange. >>> >>> > >>> > Marcus >>> > >>> > >>> >>> >>> >>> >> >

July 29, 2013

Re: [Pharo-users] Voyage: Circular references

by Sabine KnÃ¶fel

Hi esteban, Thank you for answering. The Problem persists i was waiting for your answer and it would be great if you could answer my Last Post in the Forum (how to tell voyage the "boundary"). I am also in vacation now and back mid of next week. Sorry for the spelling i write from Mobile phone. Sabine Am 29.07.2013 15:13 schrieb "EstebanLM [via Smalltalk]" < ml-node+s1294792n4701282h37(a)n4.nabble.com>: > Hi Sabine, > > I'm really sorry, I just went on vacations when you sent this mail :( > > Is this problem persisting? > Did you reach a solution? > Can I help you with something? > > Esteban > > > On Jul 19, 2013, at 12:56 PM, Sabine KnÃ¶fel <[hidden email]<http://user/SendEmail.jtp?type=node&node=4701282&i=0>> > wrote: > > Hi Esteban, > > thanks! > > I think the solution of Bernat will not be enough for me. > I added the class methods isVoyageRoot to the root objects (like trip, > country and so on). > > But I have instances which should never be persistent, e.g. the > country I mentioned. The Country has currencies and expense rates and > this is a lot of stuff...this should be only transient in the image. > > The transient country is referenced by persistent instances of trip. > (countrie does naturally not point to trips) > > Trip (persistant) -> Country (transient) > > If I try to save a trip, voyage at some point enters > >>rawReference: description: > and there, it looks whether anObject (a country) isNew: > isNew means the other object, the country has an _id (mongo id). > > The country does not have an id and for this reason, voyage tries to > make it persistent although its class has isVogageRoot false. > And than the system enters a point where it tries to make so many > objects persistent that the image does not respond anymore. > > So, my question is: is it possible to define > "In class trip, there is an inst var country -> voyage please save > only a symbol here" > and > "voyage, if loading a trip, for inst var country, do shting" > > Sure, I could save symbols within the trips country inst var all the time > and get the country it if needed, but it would be nicer if voyage coud > do that for me .-) > > Sabine > > > > > > On Fri, Jul 19, 2013 at 11:52 AM, EstebanLM [via Smalltalk] > <<a > href="x-msg://4592/user/SendEmail.jtp?type=node&node=4699517&i=0" > target="_top" rel="nofollow" link="external">[hidden email]> wrote: > > > Hi Sabine, > > > > If Bernat help is not enough, I would like to have some more insight on > your > > problem, so If you can put here a better description of your model and > the > > problem you are having, I will have a look :) > > > > Esteban > > > > On Jul 19, 2013, at 10:42 AM, Bernat Romagosa <[hidden email]> wrote: > > > > Hi Sabine! > > > > If you want an object to be referenced by another, and not embedded in > it, > > you need to give that object root entity. As a root, the object will > have a > > collection of its own, and will be referenced by other objects by id. If > the > > object is not root, it will be embedded inside the referring object as a > > property. > > > > So, to be safe, always set #isVoyageRoot to return true at class side of > all > > objects you don't want to have embedded. In general, "big enough" > objects > > who are referenced by different objects should be root. > > > > In my case, I had a centre which had users which had centres. Users also > had > > projects, and projects also had users. Big mess, I know. Besides that, > all > > users, centres and projects had profiles. My solution was to make > centres, > > projects and users be roots, and let only the profiles be embedded. > > > > I hope this helped! > > > > Bernat. > > > > > > 2013/7/19 Sabine KnÃ¶fel <[hidden email]> > >> > >> Hi Esteban, > >> > >> yes, now the Rectangle Example works after doing the reset. > >> > >> But saving my model still leads to a non responding image because > >> Voyage is trying to write json for all model stuf deep deep deep and > >> circular. Perhaps same like: > >> > http://forum.world.st/Voyage-Circular-references-td4691940.html#a4699436 > >> > >> I dont know anything about Magritte (yet) and I need a hint what to > >> define in my model so that it stops creating json at certain points > >> and creates only e.g. a Symbol. > >> > >> Example: My model instance of trip has a connection to an instance of > >> country. The country should not be written in json with all its > >> definitions. Voyage should only write e.g. #germany in the json and > >> later, when getting the instance back from the database, I want to > >> reassign the trip to the country. > >> > >> I already implemented this stuff when working witk Mongotalk the last > >> months and it worked fine. But my question yesterday: > >> > >> > http://forum.world.st/MongoDB-open-close-in-production-singleton-or-not-td4… > >> lead to my impression that I should use voyage. > >> > >> So, how can I define the "borders"? > >> > >> Sabine > >> > >> On Thu, Jul 18, 2013 at 7:45 PM, EstebanLM [via Smalltalk] > >> <[hidden email]> wrote: > >> > >> > Hi Sabine, > >> > > >> > Probably you do not missed anything. > >> > Probably I missed to update the voyage configuration :) > >> > > >> > can you try doing > >> > > >> > VOMongoRepository allInstancesDo: #reset. > >> > > >> > before trying again? > >> > > >> > Cheers, > >> > Esteban > >> > > >> > On Jul 18, 2013, at 6:31 PM, Sabine KnÃ¶fel <[hidden email]> wrote: > >> > >> > > >> >> I found this: > >> >> http://smallworks.eu/web/blog/2013-06-14-voyage-the-adventure > >> >> > >> >> Chapter "Enhancing storage" > >> >> But if I add the 3+1 methods to Rectangle, and save it again, the > json > >> >> did > >> >> not change. > >> >> > >> >> Did I miss something? > >> >> > >> >> > >> >> > >> >> > >> >> -- > >> >> View this message in context: > >> >> > http://forum.world.st/Voyage-Circular-references-tp4691940p4699436.html > >> >> Sent from the Pharo Smalltalk Users mailing list archive at > Nabble.com. > >> >> > >> > > >> > > >> > > >> > > >> > ________________________________ > >> > If you reply to this email, your message will be added to the > discussion > >> > below: > >> > > http://forum.world.st/Voyage-Circular-references-tp4691940p4699443.html > >> > To unsubscribe from Voyage: Circular references, click here. > >> > NAML > >> > >> ________________________________ > >> View this message in context: Re: Voyage: Circular references > >> > >> Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. > > > > > > > > > > > -- > > Bernat Romagosa. > > > > > > > > > > ________________________________ > > If you reply to this email, your message will be added to the discussion > > below: > > http://forum.world.st/Voyage-Circular-references-tp4691940p4699516.html > > To unsubscribe from Voyage: Circular references, click here. > > NAML > > ------------------------------ > View this message in context: Re: Voyage: Circular references<http://forum.world.st/Voyage-Circular-references-tp4691940p4699517.html> > Sent from the Pharo Smalltalk Users mailing list archive<http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html>at > Nabble.com. > > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > http://forum.world.st/Voyage-Circular-references-tp4691940p4701282.html > To unsubscribe from Voyage: Circular references, click here<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&no…> > . > NAML<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instan…> > -- View this message in context: http://forum.world.st/Voyage-Circular-references-tp4691940p4701356.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

July 29, 2013