- Pharo-users - lists.pharo.org

Re: [Pharo-users] Broken link for download Portable edition

by Marcus Denker

On Aug 1, 2013, at 1:25 PM, Gisela Decuzzi <giseladecuzzi(a)gmail.com> wrote: > Hi, I want to try the portable edition I saw in the download's page but is pointing to http://files.pharo.org/portable/Pharo3.0-portable.zip and in files.pharo.org seems missing the portable folder at all. > > I checked in the old server (http://pharo.gforge.inria.fr/ci/) and there is not portable folder also. > > Someone know where I can find it? Yes, this is now http://files.pharo.org/platform/ I will update the website. Marcus

Aug. 1, 2013

Broken link for download Portable edition

by Gisela Decuzzi

Hi, I want to try the portable edition I saw in the download's page but is pointing to http://files.pharo.org/portable/Pharo3.0-portable.zip and in files.pharo.org seems missing the portable folder at all. I checked in the old server (http://pharo.gforge.inria.fr/ci/) and there is not portable folder also. Someone know where I can find it?

Aug. 1, 2013

Re: [Pharo-users] [Pharo-dev] Large images reasons [WAS] Re: Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

by phil＠highoctane.be

This trimmed a couple of megs indeed. Then I did an ImageCleaner cleanUpForRelease. and saved. But nothing terrific as the size continues to grow. Need to see where those MorphExtensions are coming from... Phil On Thu, Aug 1, 2013 at 9:33 AM, Marcus Denker <marcus.denker(a)inria.fr>wrote: > > On Aug 1, 2013, at 8:56 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> > wrote: > > > since years marcus is telling that MC storing ancestor information is > doomed but we do not have something to really replace it. > > > > This is yet another thing. > > Monticello keep all history data of all package in image, this is not a > cache. When you delete it, your package loses > it's history and you can not merge anymore. > > MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' > put: nil ]. > > We did that on Pharo2 release and it saved 8MB (nearly a third of the > image). > > Marcus >

Aug. 1, 2013

Re: [Pharo-users] [Pharo-dev] Large images reasons [WAS] Re: Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

by Goubier Thierry

Hi Stef, With Dale, we're working on a FileTree / GitFileTree without ancestors info in the repository [1] :) ... ... but recreated on the fly by GitFileTree for the Monticello history browsing tools [2] :( [1] is underway and ... [2] is already working in https://github.com/dalehenrich/filetree/tree/pharo2.0/repository/Monticello… Thierry Le 01/08/2013 08:56, StÃ©phane Ducasse a Ã©crit : > since years marcus is telling that MC storing ancestor information is > doomed but we do not have something to really replace it. > > Stef > On Jul 31, 2013, at 8:53 PM, Mariano Martinez Peck > <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: > >> OK, this was was my experiment.... >> >> Image fresh with all my app and dependencies loades: 30MB >> >> After using it for some days/weeks: 160MB. >> >> SpaceTally new printSpaceAnalysis showed: >> >> Class code space # instances >> inst space percent inst average size >> ByteString 2785 413144 >> 116244078 69.90 281.36 >> Array 3712 181772 >> 8466668 5.10 46.58 >> ByteArray 8574 1319 >> 8186802 4.90 6206.82 >> Bitmap 3653 303 >> 6656340 4.00 21968.12 >> CompiledMethod 22467 90554 >> 5685374 3.40 62.78 >> >> >> After executing ImageCleaner cleanUpForRelease: 36MB >> >> Then...I searched which part of #cleanUpForRelease: was making the >> difference, and it was: >> >> Smalltalk cleanUp: true except: #() confirming: false. >> >> So now it was time to know WHICH class did the diference, so I modified >> #cleanUp: aggressive except: exclusions confirming: aBool >> >> in these lines: >> >> "Run the cleanup code" >> classes >> do:[:aClass| >> Transcript show: 'Image size before cleaning ', aClass name, ' : ', >> Smalltalk imagePath asFileReference size asString. >> aClass cleanUp: aggressive. >> 3 timesRepeat: [Smalltalk garbageCollect]. >> Smalltalk snapshot: true andQuit: false. >> Transcript show: 'Image size after cleaning ', aClass name, ' : ', >> Smalltalk imagePath asFileReference size asString. >> ] >> displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. >> I then opened a Transcript, and evaluated >> >> Smalltalk cleanUp: true except: #() confirming: false. >> >> I went to prepare Mate, and when I come back, the result was, of course: >> >> "Image size after cleaning MCFileBasedRepository : 39744008" >> >> That clean up ends up doing: >> >> flushAllCaches >> self allSubInstancesDo: [:ea | ea flushCache] >> So it sends #flushCache to all instances of MCHttpRepository and >> MCFileBasedRepository. >> >> Now what I wanted to see if it there was a particular repo that could >> take most of the space (like package-cache). >> And indeed, it was...I modified #flushCaches to: >> >> flushAllCaches >> | file | >> file := 'repos.txt' asFileReference writeStream text. >> self allSubInstancesDo: [:each | >> file nextPutAll: 'Image size before cleaning ', each printString, ' : >> ', Smalltalk imagePath asFileReference size asString; cr. >> each flushCache. >> 3 timesRepeat: [Smalltalk garbageCollect]. >> Smalltalk snapshot: true andQuit: false. >> file nextPutAll: 'Image size after cleaning ', each printString, ' : >> ', Smalltalk imagePath asFileReference size asString;cr. >> ]. >> file flush; close. >> And then I looked in the 'repos.txt' file. My package cache repo >> cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. >> So...cleaning cache of just 3 repos frees approx 100MB. >> >> The question is....can we flush the cache safely? If they are called >> "cache", then I guess yes, we can. >> >> Thoughts? >> >> Thanks, >> >> >> >> On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck >> <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: >> >> Guys, I have images also with seaside, magritte, glorp, >> postgresV2, etc and it is also around 200MB. >> I will try to do some research today and let you know. >> >> Cheers, >> >> >> On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker >> <marcus.denker(a)inria.fr <mailto:marcus.denker@inria.fr>> wrote: >> >> >> On Jul 30, 2013, at 1:49 PM, "phil(a)highoctane.be >> <mailto:phil@highoctane.be>" <phil(a)highoctane.be >> <mailto:phil@highoctane.be>> wrote: >> >> > the changes file contained passwords and I replaced the >> text. So offsets may be wrong due to that. >> > >> Yes, the first thing I wanted to do is to recompile >> everything. Does not work. >> >> > Memorymonitor is not doing fanct stuff. It just counts >> instances. >> > >> Yes, but maybe it holds on to these instances? >> >> Marcus >> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/> >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/> > -- Thierry Goubier CEA list Laboratoire des Fondations des SystÃ¨mes Temps RÃ©el EmbarquÃ©s 91191 Gif sur Yvette Cedex France Phone/Fax: +33 (0) 1 69 08 32 92 / 83 95

Aug. 1, 2013

Re: [Pharo-users] [Pharo-dev] Large images reasons [WAS] Re: Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

by Marcus Denker

On Aug 1, 2013, at 8:56 AM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: > since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it. > This is yet another thing. Monticello keep all history data of all package in image, this is not a cache. When you delete it, your package loses it's history and you can not merge anymore. MCVersionInfo allInstances do: [ :each | each instVarNamed: 'ancestors' put: nil ]. We did that on Pharo2 release and it saved 8MB (nearly a third of the image). Marcus

Aug. 1, 2013

Re: [Pharo-users] Pharo performance

by CHRIS BAILEY

I think my package is based on the one from the DBX suite. It has a few edits, some of which I'm confident should go in and others just for my needs which probably aren't fit to. Disappointly, I got the psyco test wrong and it is only taking 30m/s :( I guess this is the difference when something is implemented on top of C and libpq (I presume) ________________________________ From: Mariano Martinez Peck <marianopeck(a)gmail.com> To: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> Sent: Wednesday, 31 July 2013, 22:40 Subject: Re: [Pharo-users] Pharo performance On Wed, Jul 31, 2013 at 6:32 PM, Chris <cpmbailey(a)btinternet.com> wrote: It definitely seems to be because most things in the V2 driver are coming in as strings and then being converted. Running the same query on the three systems gives the following for a 65000 double array > >PGV2 770m/s >PGV3 200m/s >Psycopg2 130m/s > >Now I just need to find out how to get PGV3 as the backend for Glorp! > > PLease please please use the Glorp version you get from the DBX suite. You need to create a subclass ofÂ DatabaseDriver. You can take a look toÂ NativePostgresDriver and create a similar one for V3. I would also like to have Glorp + PostgresV3 running.Â Also...we should somehow use NativePostgresPoolingDriver and NativePostgresConnectionPool for V3... we need some refactor here... We should join forces!!Â Â Cheers >Chris > > > >Hi Yanni, >> >>On 30 Jul 2013, at 05:17, Yanni Chiu <yanni(a)rogers.com> wrote: >> >> >>On 29/07/13 7:08 PM, Sven Van Caekenberghe wrote: >>> >>>The explanation for the slowdown must be in the PgV2 driver. >>>> The PgV2 protocol is described at: >>>Â http://www.postgresql.org/docs/7.1/static/protocol-message-formats.html >>> >>>Have a glance at the "AsciiRow" and "BinaryRow" message formats. The driver reads the data off the socket, parsing the the data, as described as described by the message format. With the V2 protocol design, you have to read the result row, one field at a time. >>> >>>IIUC, in the newer V3? protocol, the AsciiRow/BinaryRow message is replaced by a DataRow message. The DataRow message has the message size included, which could allow the driver to read the entire set of fields for one data row, using a single socket read (or a few buffer sized reads). >>> >>>I recall seeing an experimental V3 protocol implementation, a few years back - sorry, no links handy. It would be nice to see some benchmarks. >>> >>>Hope that helps. >>> Thanks for the response. >> >>I believe the V3 project is here http://www.squeaksource.com/PostgresV3.html. >> >>Now, I probably spoke too fast and should have taken Mariano's advice to never speculate and first measure. Here is my quick test that, for me, shows that PostgresV2 seems more than fast enough (something that I had experienced before without doing benchmarks). >> >>[ self execute: 'select longitude,latitude from log_data limit 10000;' ] timeToRun. >> >>=> 76 ms >> >>[ self execute: 'select longitude,latitude from log_data limit 100000;' ] timeToRun. >> >>=> 765 ms >> >>This is querying for 2 floats from a huge table, over the network. Pretty fast ;-) >> >>So, back to Chris: what exactly are you doing that is (so) slow ? >> >>Anyway, thanks Yanni for all your work on the existing driver ! >> >>Sven >> >>-- >>Sven Van Caekenberghe >>Proudly supporting Pharo >>http://pharo.org >>http://association.pharo.org >>http://consortium.pharo.org >> >> >> >> >> >> >> > > -- Mariano http://marianopeck.wordpress.com

Aug. 1, 2013

Re: [Pharo-users] [Pharo-dev] Large images reasons [WAS] Re: Pharo 2.0 with Seaside + DBXTalk + GlorpDBX + Magritte 3 + TWBS is getting slower and slower

by StÃ©phane Ducasse

since years marcus is telling that MC storing ancestor information is doomed but we do not have something to really replace it. Stef On Jul 31, 2013, at 8:53 PM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > OK, this was was my experiment.... > > Image fresh with all my app and dependencies loades: 30MB > > After using it for some days/weeks: 160MB. > > SpaceTally new printSpaceAnalysis showed: > > Class code space # instances inst space percent inst average size > ByteString 2785 413144 116244078 69.90 281.36 > Array 3712 181772 8466668 5.10 46.58 > ByteArray 8574 1319 8186802 4.90 6206.82 > Bitmap 3653 303 6656340 4.00 21968.12 > CompiledMethod 22467 90554 5685374 3.40 62.78 > > > After executing ImageCleaner cleanUpForRelease: 36MB > > Then...I searched which part of #cleanUpForRelease: was making the difference, and it was: > > Smalltalk cleanUp: true except: #() confirming: false. > > So now it was time to know WHICH class did the diference, so I modified > #cleanUp: aggressive except: exclusions confirming: aBool > > in these lines: > > "Run the cleanup code" > classes > do:[:aClass| > Transcript show: 'Image size before cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. > aClass cleanUp: aggressive. > 3 timesRepeat: [Smalltalk garbageCollect]. > Smalltalk snapshot: true andQuit: false. > Transcript show: 'Image size after cleaning ', aClass name, ' : ', Smalltalk imagePath asFileReference size asString. > ] > displayingProgress: [:aClass| 'Cleaning up in ', aClass name]. > > > I then opened a Transcript, and evaluated > > Smalltalk cleanUp: true except: #() confirming: false. > > I went to prepare Mate, and when I come back, the result was, of course: > > "Image size after cleaning MCFileBasedRepository : 39744008" > > That clean up ends up doing: > > flushAllCaches > self allSubInstancesDo: [:ea | ea flushCache] > > So it sends #flushCache to all instances of MCHttpRepository and MCFileBasedRepository. > > Now what I wanted to see if it there was a particular repo that could take most of the space (like package-cache). > And indeed, it was...I modified #flushCaches to: > > flushAllCaches > | file | > file := 'repos.txt' asFileReference writeStream text. > self allSubInstancesDo: [:each | > > file nextPutAll: 'Image size before cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString; cr. > each flushCache. > 3 timesRepeat: [Smalltalk garbageCollect]. > Smalltalk snapshot: true andQuit: false. > file nextPutAll: 'Image size after cleaning ', each printString, ' : ', Smalltalk imagePath asFileReference size asString;cr. > > ]. > file flush; close. > > And then I looked in the 'repos.txt' file. My package cache repo cleaned 60 MB. Glorp cleaned 35MB. Seaside30 cleaned 10MB. > So...cleaning cache of just 3 repos frees approx 100MB. > > The question is....can we flush the cache safely? If they are called "cache", then I guess yes, we can. > > Thoughts? > > Thanks, > > > > On Wed, Jul 31, 2013 at 10:48 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > Guys, I have images also with seaside, magritte, glorp, postgresV2, etc and it is also around 200MB. > I will try to do some research today and let you know. > > Cheers, > > > On Tue, Jul 30, 2013 at 8:55 AM, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > On Jul 30, 2013, at 1:49 PM, "phil(a)highoctane.be" <phil(a)highoctane.be> wrote: > > > the changes file contained passwords and I replaced the text. So offsets may be wrong due to that. > > > Yes, the first thing I wanted to do is to recompile everything. Does not work. > > > Memorymonitor is not doing fanct stuff. It just counts instances. > > > Yes, but maybe it holds on to these instances? > > Marcus > > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > -- > Mariano > http://marianopeck.wordpress.com

Aug. 1, 2013

Re: [Pharo-users] Pharo performance

by Mariano Martinez Peck

On Wed, Jul 31, 2013 at 6:32 PM, Chris <cpmbailey(a)btinternet.com> wrote: > It definitely seems to be because most things in the V2 driver are coming > in as strings and then being converted. Running the same query on the three > systems gives the following for a 65000 double array > > PGV2 770m/s > PGV3 200m/s > Psycopg2 130m/s > > Now I just need to find out how to get PGV3 as the backend for Glorp! > > PLease please please use the Glorp version you get from the DBX suite. You need to create a subclass of DatabaseDriver. You can take a look to NativePostgresDriver and create a similar one for V3. I would also like to have Glorp + PostgresV3 running. Also...we should somehow use NativePostgresPoolingDriver and NativePostgresConnectionPool for V3... we need some refactor here... We should join forces!! > Cheers > Chris > > > Hi Yanni, >> >> On 30 Jul 2013, at 05:17, Yanni Chiu <yanni(a)rogers.com> wrote: >> >> On 29/07/13 7:08 PM, Sven Van Caekenberghe wrote: >>> >>>> The explanation for the slowdown must be in the PgV2 driver. >>>> >>> The PgV2 protocol is described at: >>> http://www.postgresql.org/**docs/7.1/static/protocol-** >>> message-formats.html<http://www.postgresql.org/docs/7.1/static/protocol-message-formats.html> >>> >>> Have a glance at the "AsciiRow" and "BinaryRow" message formats. The >>> driver reads the data off the socket, parsing the the data, as described as >>> described by the message format. With the V2 protocol design, you have to >>> read the result row, one field at a time. >>> >>> IIUC, in the newer V3? protocol, the AsciiRow/BinaryRow message is >>> replaced by a DataRow message. The DataRow message has the message size >>> included, which could allow the driver to read the entire set of fields for >>> one data row, using a single socket read (or a few buffer sized reads). >>> >>> I recall seeing an experimental V3 protocol implementation, a few years >>> back - sorry, no links handy. It would be nice to see some benchmarks. >>> >>> Hope that helps. >>> >> Thanks for the response. >> >> I believe the V3 project is here http://www.squeaksource.com/** >> PostgresV3.html <http://www.squeaksource.com/PostgresV3.html>. >> >> Now, I probably spoke too fast and should have taken Mariano's advice to >> never speculate and first measure. Here is my quick test that, for me, >> shows that PostgresV2 seems more than fast enough (something that I had >> experienced before without doing benchmarks). >> >> [ self execute: 'select longitude,latitude from log_data limit 10000;' ] >> timeToRun. >> >> => 76 ms >> >> [ self execute: 'select longitude,latitude from log_data limit 100000;' ] >> timeToRun. >> >> => 765 ms >> >> This is querying for 2 floats from a huge table, over the network. Pretty >> fast ;-) >> >> So, back to Chris: what exactly are you doing that is (so) slow ? >> >> Anyway, thanks Yanni for all your work on the existing driver ! >> >> Sven >> >> -- >> Sven Van Caekenberghe >> Proudly supporting Pharo >> http://pharo.org >> http://association.pharo.org >> http://consortium.pharo.org >> >> >> >> >> >> >> > > -- Mariano http://marianopeck.wordpress.com

July 31, 2013

Re: [Pharo-users] Pharo performance

by Chris

It definitely seems to be because most things in the V2 driver are coming in as strings and then being converted. Running the same query on the three systems gives the following for a 65000 double array PGV2 770m/s PGV3 200m/s Psycopg2 130m/s Now I just need to find out how to get PGV3 as the backend for Glorp! Cheers Chris > Hi Yanni, > > On 30 Jul 2013, at 05:17, Yanni Chiu <yanni(a)rogers.com> wrote: > >> On 29/07/13 7:08 PM, Sven Van Caekenberghe wrote: >>> The explanation for the slowdown must be in the PgV2 driver. >> The PgV2 protocol is described at: >> http://www.postgresql.org/docs/7.1/static/protocol-message-formats.html >> >> Have a glance at the "AsciiRow" and "BinaryRow" message formats. The driver reads the data off the socket, parsing the the data, as described as described by the message format. With the V2 protocol design, you have to read the result row, one field at a time. >> >> IIUC, in the newer V3? protocol, the AsciiRow/BinaryRow message is replaced by a DataRow message. The DataRow message has the message size included, which could allow the driver to read the entire set of fields for one data row, using a single socket read (or a few buffer sized reads). >> >> I recall seeing an experimental V3 protocol implementation, a few years back - sorry, no links handy. It would be nice to see some benchmarks. >> >> Hope that helps. > Thanks for the response. > > I believe the V3 project is here http://www.squeaksource.com/PostgresV3.html. > > Now, I probably spoke too fast and should have taken Mariano's advice to never speculate and first measure. Here is my quick test that, for me, shows that PostgresV2 seems more than fast enough (something that I had experienced before without doing benchmarks). > > [ self execute: 'select longitude,latitude from log_data limit 10000;' ] timeToRun. > > => 76 ms > > [ self execute: 'select longitude,latitude from log_data limit 100000;' ] timeToRun. > > => 765 ms > > This is querying for 2 floats from a huge table, over the network. Pretty fast ;-) > > So, back to Chris: what exactly are you doing that is (so) slow ? > > Anyway, thanks Yanni for all your work on the existing driver ! > > Sven > > -- > Sven Van Caekenberghe > Proudly supporting Pharo > http://pharo.org > http://association.pharo.org > http://consortium.pharo.org > > > > > >

July 31, 2013

Re: [Pharo-users] RFB on 2.0 + linux - semaphore primitive failed

by Igor Stasenko

On 31 July 2013 22:17, Paul DeBruicker <pdebruic(a)gmail.com> wrote: > I don't know. I don't get the errors that used to occur with the max > semaphores being exceeded. I thought Igor Stasenko fixed that issue in > the VM so the array that holds the semaphores grows without blocking. > Or something. > You can have as big as you want array with semaphores. There is no limits and maxExternalSemaphoresSilently is just does nothing and ignored. > > > What I do know is that if I 'abandon' that warning and save & quit and > then reopen the image then the UI is locked and I have to use Seaside's > WAScreenshot tool to suspend and resume the UI to unfreeze it. > > weird. i would start from figuring out, why primitive failed..and for which semaphore. -- Best regards, Igor Stasenko.

July 31, 2013