June 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Exploring Heap Size Limits

by Jan Vrany

On 29/06/13 17:45, Sven Van Caekenberghe wrote: > Hi Jan, > > On 29 Jun 2013, at 17:12, Jan Vrany <jan.vrany(a)fit.cvut.cz> wrote: > >> Hi, >> >> for 32bit application you can never allocate whole 4GB as some memory has to be reserved for kernel. The usual split is 2/2, sometimes 3/1. Plus, >> the actual code must be somewhere in the memory as well as the stack. >> Usually the real limit for 32bit application is somewhere like 1.8GB for >> heap. With 3/1 split you can have one more gig, but for linux that would >> require a 3/1 support in the kernel. If I'm not mistaken most stock kernels come with 2/2 split. >> >> Jan > > Thanks for the reply. Sounds pretty reasonable. > > Would this also be the case for a 32-bit app running on a 64-bit OS ? If I'm not much mistaken, yes. The crucial point are system calls which need to pass data to the kernel (such as read()/write() to name the most obvious ones). At some point the process is in funny state, memory-wise. It's in kernel mode already running the system call but CPU has still user-processes' page table active. At this point data are copied from user memory to kernel memory (to those reserved 2gigs, usually memory above 0x7FFFFFFF) Then the kernel install its own, kernel page table which maps some memory the very same physical page which was mapped in user process's page table to the address where the data were just copied. That's how the data are transferred between user space and kernel space. Or, at least that's my understanding how it's done :-) I was never particularly good at these low-level things. Best, Jan > > Sven > >> On 28/06/13 23:02, Sven Van Caekenberghe wrote: >>> Hi, >>> >>> I wrote a little test to explore the heap size limits of the current Pharo image+VM combination. I did some tests on a 8GB 64-bit Linux machine and the basic conclusion is that using a standard 1GB heap size you can allocate and GC with close to 1GB of actual data. I think that is pretty good news. The machine has 4 effective cores, so could easily run 4 busy images like that. >>> >>> Pushing the limits further, things get weird, up to the point of crashing. Sadly 2, 3 or 4 GB heap size, which should be theoretically possible, fail, thus limiting the amount of data that can be allocated and used. >>> >>> Anyway, my goal is to start the debate around this topic ;-) >>> >>> >>> Here is the test script: >>> >>> t3@coolermaster:smalltalk$ cat memtest.st >>> >>> NonInteractiveTranscript stdout install. >>> >>> ! >>> >>> Transcript show: 'memtest.st'; cr. >>> Smalltalk garbageCollect. >>> Transcript show: SmalltalkImage current vm statisticsReport; cr. >>> >>> ! >>> >>> | count size data timeToRun | >>> count := Smalltalk commandLine arguments first asInteger. >>> size := Smalltalk commandLine arguments second asInteger. >>> Transcript show: ('allocating: {1} times {2} bytes' format: { count. size }); cr. >>> timeToRun := [ >>> data := Array new: count streamContents: [ :out | >>> count timesRepeat: [ >>> out nextPut: (ByteArray new: size) ] ]. >>> ] timeToRun. >>> Transcript >>> show: ('allocated: {1}' format: { (data collect: #size) sum humanReadableSIByteSize }); >>> cr. >>> Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. >>> Transcript show: '3 times GC'; cr. >>> timeToRun := [ >>> 3 timesRepeat: [ Smalltalk garbageCollect ] ] timeToRun. >>> Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. >>> Transcript show: SmalltalkImage current vm statisticsReport; cr. >>> >>> ! >>> >>> SmalltalkImage current quitPrimitive. >>> >>> >>> And here is a normal result running get.pharo.org/20+vm >>> >>> $ ./pharo Pharo.image memtest.st 512 1024000 >>> >>> memtest.st >>> uptime 0h0m0s >>> memory 25,076,708 bytes >>> old 19,213,624 bytes (76.60000000000001%) >>> young 5,384 bytes (0.0%) >>> used 19,219,008 bytes (76.60000000000001%) >>> free 5,857,700 bytes (23.400000000000002%) >>> GCs 10 (12ms between GCs) >>> full 1 totalling 30ms (25.0% uptime), avg 30.0ms >>> incr 9 totalling 6ms (5.0% uptime), avg 0.7000000000000001ms >>> tenures 0 >>> >>> allocating: 512 times 1024000 bytes >>> allocated: 524.29 MB >>> time to run 0:00:00:02.852 >>> 3 times GC >>> time to run 0:00:00:00.196 >>> uptime 0h0m3s >>> memory 546,746,364 bytes >>> old 543,520,476 bytes (99.4%) >>> young 7,184 bytes (0.0%) >>> used 543,527,660 bytes (99.4%) >>> free 3,218,704 bytes (0.6000000000000001%) >>> GCs 356 (9ms between GCs) >>> full 89 totalling 2,743ms (86.4% uptime), avg 30.8ms >>> incr 267 totalling 223ms (7.0% uptime), avg 0.8ms >>> tenures 0 >>> Since last view 346 (9ms between GCs) >>> uptime 3.1s >>> full 88 totalling 2,713ms (88.80000000000001% uptime), avg 30.8ms >>> incr 258 totalling 217ms (7.1000000000000005% uptime), avg 0.8ms >>> tenures 0 >>> >>> >>> It takes about 2.5s to allocate about 512Mb. A GC then takes 30ms. >>> >>> You can play a bit with the 2 parameters to generate different datasets. >>> >>> Not changing any VM parameters, you can successfully do something like >>> >>> $ ./pharo Pharo.image memtest.st 1000 1024000 >>> >>> or >>> >>> $ ./pharo Pharo.image memtest.st 1024000 1000 >>> >>> thus allocating about 1GB. >>> >>> Playing with -mmap gives some weird results and unexpected slowness as well. Consider (the default -mmap is 1024m): >>> >>> $ ./pharo -mmap 1280m Pharo.image memtest.st 500 1024000 >>> >>> memtest.st >>> uptime 0h0m0s >>> memory 25,076,716 bytes >>> old 19,213,480 bytes (76.60000000000001%) >>> young 5,384 bytes (0.0%) >>> used 19,218,864 bytes (76.60000000000001%) >>> free 5,857,852 bytes (23.400000000000002%) >>> GCs 12 (13ms between GCs) >>> full 2 totalling 56ms (37.1% uptime), avg 28.0ms >>> incr 10 totalling 6ms (4.0% uptime), avg 0.6000000000000001ms >>> tenures 0 >>> >>> allocating: 500 times 1024000 bytes >>> allocated: 512.00 MB >>> time to run 0:00:00:13.66 >>> 3 times GC >>> time to run 0:00:00:00.192 >>> uptime 0h0m14s >>> memory 533,946,384 bytes >>> old 531,232,232 bytes (99.5%) >>> young 7,644 bytes (0.0%) >>> used 531,239,876 bytes (99.5%) >>> free 2,706,508 bytes (0.5%) >>> GCs 1,011 (14ms between GCs) >>> full 501 totalling 13,819ms (98.7% uptime), avg 27.6ms >>> incr 510 totalling 35ms (0.2% uptime), avg 0.1ms >>> tenures 0 >>> Since last view 999 (14ms between GCs) >>> uptime 13.9s >>> full 499 totalling 13,763ms (99.30000000000001% uptime), avg 27.6ms >>> incr 500 totalling 29ms (0.2% uptime), avg 0.1ms >>> tenures 0 >>> >>> >>> So increasing the heap from 1024 Mb to 1280 Mb slows the allocation down to 14s. >>> >>> Increasing the heap size to 1792 Mb gives the following result: >>> >>> t3@coolermaster:smalltalk$ ./pharo -mmap 1792m Pharo.image memtest.st 1280 1048576 >>> >>> memtest.st >>> uptime 0h0m0s >>> memory 25,076,708 bytes >>> old 19,213,592 bytes (76.60000000000001%) >>> young 5,384 bytes (0.0%) >>> used 19,218,976 bytes (76.60000000000001%) >>> free 5,857,732 bytes (23.400000000000002%) >>> GCs 12 (12ms between GCs) >>> full 2 totalling 55ms (37.9% uptime), avg 27.5ms >>> incr 10 totalling 6ms (4.1000000000000005% uptime), avg 0.6000000000000001ms >>> tenures 0 >>> >>> allocating: 1280 times 1048576 bytes >>> allocated: 1.34 GB >>> time to run 0:00:00:35.258 >>> 3 times GC >>> time to run 0:00:00:00.37 >>> uptime 0h0m35s >>> memory -783,340,292 bytes >>> old -786,061,544 bytes (100.30000000000001%) >>> young 7,460 bytes (0.0%) >>> used -786,054,084 bytes (100.30000000000001%) >>> free 2,713,792 bytes (-0.30000000000000004%) >>> GCs 2,572 (14ms between GCs) >>> full 1281 totalling 35,444ms (99.10000000000001% uptime), avg 27.700000000000003ms >>> incr 1291 totalling 93ms (0.30000000000000004% uptime), avg 0.1ms >>> tenures 0 >>> Since last view 2,560 (14ms between GCs) >>> uptime 35.6s >>> full 1279 totalling 35,389ms (99.30000000000001% uptime), avg 27.700000000000003ms >>> incr 1281 totalling 87ms (0.2% uptime), avg 0.1ms >>> tenures 0 >>> >>> >>> The allocation takes a lot longer, but the GC times are stable. >>> >>> I suspect that negative reporting has to do with SmallInteger limitations, but the question is if it is only a cosmetic problem or not. >>> >>> Again, I am not complaining, just exploring/wondering. >>> >>> >>> Sven >>> >>> >>> >> >> > > >

June 29, 2013

Re: [Pharo-dev] Exploring Heap Size Limits

by Sven Van Caekenberghe

Hi Jan, On 29 Jun 2013, at 17:12, Jan Vrany <jan.vrany(a)fit.cvut.cz> wrote: > Hi, > > for 32bit application you can never allocate whole 4GB as some memory has to be reserved for kernel. The usual split is 2/2, sometimes 3/1. Plus, > the actual code must be somewhere in the memory as well as the stack. > Usually the real limit for 32bit application is somewhere like 1.8GB for > heap. With 3/1 split you can have one more gig, but for linux that would > require a 3/1 support in the kernel. If I'm not mistaken most stock kernels come with 2/2 split. > > Jan Thanks for the reply. Sounds pretty reasonable. Would this also be the case for a 32-bit app running on a 64-bit OS ? Sven > On 28/06/13 23:02, Sven Van Caekenberghe wrote: >> Hi, >> >> I wrote a little test to explore the heap size limits of the current Pharo image+VM combination. I did some tests on a 8GB 64-bit Linux machine and the basic conclusion is that using a standard 1GB heap size you can allocate and GC with close to 1GB of actual data. I think that is pretty good news. The machine has 4 effective cores, so could easily run 4 busy images like that. >> >> Pushing the limits further, things get weird, up to the point of crashing. Sadly 2, 3 or 4 GB heap size, which should be theoretically possible, fail, thus limiting the amount of data that can be allocated and used. >> >> Anyway, my goal is to start the debate around this topic ;-) >> >> >> Here is the test script: >> >> t3@coolermaster:smalltalk$ cat memtest.st >> >> NonInteractiveTranscript stdout install. >> >> ! >> >> Transcript show: 'memtest.st'; cr. >> Smalltalk garbageCollect. >> Transcript show: SmalltalkImage current vm statisticsReport; cr. >> >> ! >> >> | count size data timeToRun | >> count := Smalltalk commandLine arguments first asInteger. >> size := Smalltalk commandLine arguments second asInteger. >> Transcript show: ('allocating: {1} times {2} bytes' format: { count. size }); cr. >> timeToRun := [ >> data := Array new: count streamContents: [ :out | >> count timesRepeat: [ >> out nextPut: (ByteArray new: size) ] ]. >> ] timeToRun. >> Transcript >> show: ('allocated: {1}' format: { (data collect: #size) sum humanReadableSIByteSize }); >> cr. >> Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. >> Transcript show: '3 times GC'; cr. >> timeToRun := [ >> 3 timesRepeat: [ Smalltalk garbageCollect ] ] timeToRun. >> Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. >> Transcript show: SmalltalkImage current vm statisticsReport; cr. >> >> ! >> >> SmalltalkImage current quitPrimitive. >> >> >> And here is a normal result running get.pharo.org/20+vm >> >> $ ./pharo Pharo.image memtest.st 512 1024000 >> >> memtest.st >> uptime 0h0m0s >> memory 25,076,708 bytes >> old 19,213,624 bytes (76.60000000000001%) >> young 5,384 bytes (0.0%) >> used 19,219,008 bytes (76.60000000000001%) >> free 5,857,700 bytes (23.400000000000002%) >> GCs 10 (12ms between GCs) >> full 1 totalling 30ms (25.0% uptime), avg 30.0ms >> incr 9 totalling 6ms (5.0% uptime), avg 0.7000000000000001ms >> tenures 0 >> >> allocating: 512 times 1024000 bytes >> allocated: 524.29 MB >> time to run 0:00:00:02.852 >> 3 times GC >> time to run 0:00:00:00.196 >> uptime 0h0m3s >> memory 546,746,364 bytes >> old 543,520,476 bytes (99.4%) >> young 7,184 bytes (0.0%) >> used 543,527,660 bytes (99.4%) >> free 3,218,704 bytes (0.6000000000000001%) >> GCs 356 (9ms between GCs) >> full 89 totalling 2,743ms (86.4% uptime), avg 30.8ms >> incr 267 totalling 223ms (7.0% uptime), avg 0.8ms >> tenures 0 >> Since last view 346 (9ms between GCs) >> uptime 3.1s >> full 88 totalling 2,713ms (88.80000000000001% uptime), avg 30.8ms >> incr 258 totalling 217ms (7.1000000000000005% uptime), avg 0.8ms >> tenures 0 >> >> >> It takes about 2.5s to allocate about 512Mb. A GC then takes 30ms. >> >> You can play a bit with the 2 parameters to generate different datasets. >> >> Not changing any VM parameters, you can successfully do something like >> >> $ ./pharo Pharo.image memtest.st 1000 1024000 >> >> or >> >> $ ./pharo Pharo.image memtest.st 1024000 1000 >> >> thus allocating about 1GB. >> >> Playing with -mmap gives some weird results and unexpected slowness as well. Consider (the default -mmap is 1024m): >> >> $ ./pharo -mmap 1280m Pharo.image memtest.st 500 1024000 >> >> memtest.st >> uptime 0h0m0s >> memory 25,076,716 bytes >> old 19,213,480 bytes (76.60000000000001%) >> young 5,384 bytes (0.0%) >> used 19,218,864 bytes (76.60000000000001%) >> free 5,857,852 bytes (23.400000000000002%) >> GCs 12 (13ms between GCs) >> full 2 totalling 56ms (37.1% uptime), avg 28.0ms >> incr 10 totalling 6ms (4.0% uptime), avg 0.6000000000000001ms >> tenures 0 >> >> allocating: 500 times 1024000 bytes >> allocated: 512.00 MB >> time to run 0:00:00:13.66 >> 3 times GC >> time to run 0:00:00:00.192 >> uptime 0h0m14s >> memory 533,946,384 bytes >> old 531,232,232 bytes (99.5%) >> young 7,644 bytes (0.0%) >> used 531,239,876 bytes (99.5%) >> free 2,706,508 bytes (0.5%) >> GCs 1,011 (14ms between GCs) >> full 501 totalling 13,819ms (98.7% uptime), avg 27.6ms >> incr 510 totalling 35ms (0.2% uptime), avg 0.1ms >> tenures 0 >> Since last view 999 (14ms between GCs) >> uptime 13.9s >> full 499 totalling 13,763ms (99.30000000000001% uptime), avg 27.6ms >> incr 500 totalling 29ms (0.2% uptime), avg 0.1ms >> tenures 0 >> >> >> So increasing the heap from 1024 Mb to 1280 Mb slows the allocation down to 14s. >> >> Increasing the heap size to 1792 Mb gives the following result: >> >> t3@coolermaster:smalltalk$ ./pharo -mmap 1792m Pharo.image memtest.st 1280 1048576 >> >> memtest.st >> uptime 0h0m0s >> memory 25,076,708 bytes >> old 19,213,592 bytes (76.60000000000001%) >> young 5,384 bytes (0.0%) >> used 19,218,976 bytes (76.60000000000001%) >> free 5,857,732 bytes (23.400000000000002%) >> GCs 12 (12ms between GCs) >> full 2 totalling 55ms (37.9% uptime), avg 27.5ms >> incr 10 totalling 6ms (4.1000000000000005% uptime), avg 0.6000000000000001ms >> tenures 0 >> >> allocating: 1280 times 1048576 bytes >> allocated: 1.34 GB >> time to run 0:00:00:35.258 >> 3 times GC >> time to run 0:00:00:00.37 >> uptime 0h0m35s >> memory -783,340,292 bytes >> old -786,061,544 bytes (100.30000000000001%) >> young 7,460 bytes (0.0%) >> used -786,054,084 bytes (100.30000000000001%) >> free 2,713,792 bytes (-0.30000000000000004%) >> GCs 2,572 (14ms between GCs) >> full 1281 totalling 35,444ms (99.10000000000001% uptime), avg 27.700000000000003ms >> incr 1291 totalling 93ms (0.30000000000000004% uptime), avg 0.1ms >> tenures 0 >> Since last view 2,560 (14ms between GCs) >> uptime 35.6s >> full 1279 totalling 35,389ms (99.30000000000001% uptime), avg 27.700000000000003ms >> incr 1281 totalling 87ms (0.2% uptime), avg 0.1ms >> tenures 0 >> >> >> The allocation takes a lot longer, but the GC times are stable. >> >> I suspect that negative reporting has to do with SmallInteger limitations, but the question is if it is only a cosmetic problem or not. >> >> Again, I am not complaining, just exploring/wondering. >> >> >> Sven >> >> >> > >

June 29, 2013

Manage a Printer device with Semaphore forMutualExclusion

by Dario Trussardi

Ciao, I work with a seaside application. I have a printer device and i need to manage the use of it at one work at time. I think to use one Semaphore forMutualExclusion defined as variable of the specific printerTicket device Now i defined this code to mange the access : aPrinterTicket accessSema critical:[ stampaok:= [ " i open a IP connection on the device. do the printer and manage relative errors " TicketBaseTipoCentriCarico newOn: dcmModel dettagli: tfDettagli onPrinter: aPrinterTicket ] on: Error do: [:ex | " for error i open dialog to advise the operator " self jqDialog: (DTRCassaDialogConferma openOnException: ex onModel: dcmModel). " repeat until the printer is do without error " ex retry ] ]. " When the printer is do i advise the operator of it and allow the next request " stampaok ifTrue: [self jqDialog:(DTRCassaDialogConferma openOnModel: dcmModel) title: 'Ordinazione confermata']. ] With this code the printer semaphore don't work. The semaphore don't manage the code of the request, and when i have a error, at any loop the excessSignal of the printer semaphore is incremented. Any idea ? Thanks, Dario

June 29, 2013

Re: [Pharo-dev] Exploring Heap Size Limits

by Jan Vrany

Hi, for 32bit application you can never allocate whole 4GB as some memory has to be reserved for kernel. The usual split is 2/2, sometimes 3/1. Plus, the actual code must be somewhere in the memory as well as the stack. Usually the real limit for 32bit application is somewhere like 1.8GB for heap. With 3/1 split you can have one more gig, but for linux that would require a 3/1 support in the kernel. If I'm not mistaken most stock kernels come with 2/2 split. Jan On 28/06/13 23:02, Sven Van Caekenberghe wrote: > Hi, > > I wrote a little test to explore the heap size limits of the current Pharo image+VM combination. I did some tests on a 8GB 64-bit Linux machine and the basic conclusion is that using a standard 1GB heap size you can allocate and GC with close to 1GB of actual data. I think that is pretty good news. The machine has 4 effective cores, so could easily run 4 busy images like that. > > Pushing the limits further, things get weird, up to the point of crashing. Sadly 2, 3 or 4 GB heap size, which should be theoretically possible, fail, thus limiting the amount of data that can be allocated and used. > > Anyway, my goal is to start the debate around this topic ;-) > > > Here is the test script: > > t3@coolermaster:smalltalk$ cat memtest.st > > NonInteractiveTranscript stdout install. > > ! > > Transcript show: 'memtest.st'; cr. > Smalltalk garbageCollect. > Transcript show: SmalltalkImage current vm statisticsReport; cr. > > ! > > | count size data timeToRun | > count := Smalltalk commandLine arguments first asInteger. > size := Smalltalk commandLine arguments second asInteger. > Transcript show: ('allocating: {1} times {2} bytes' format: { count. size }); cr. > timeToRun := [ > data := Array new: count streamContents: [ :out | > count timesRepeat: [ > out nextPut: (ByteArray new: size) ] ]. > ] timeToRun. > Transcript > show: ('allocated: {1}' format: { (data collect: #size) sum humanReadableSIByteSize }); > cr. > Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. > Transcript show: '3 times GC'; cr. > timeToRun := [ > 3 timesRepeat: [ Smalltalk garbageCollect ] ] timeToRun. > Transcript show: ('time to run {1}' format: { Duration milliSeconds: timeToRun }); cr. > Transcript show: SmalltalkImage current vm statisticsReport; cr. > > ! > > SmalltalkImage current quitPrimitive. > > > And here is a normal result running get.pharo.org/20+vm > > $ ./pharo Pharo.image memtest.st 512 1024000 > > memtest.st > uptime 0h0m0s > memory 25,076,708 bytes > old 19,213,624 bytes (76.60000000000001%) > young 5,384 bytes (0.0%) > used 19,219,008 bytes (76.60000000000001%) > free 5,857,700 bytes (23.400000000000002%) > GCs 10 (12ms between GCs) > full 1 totalling 30ms (25.0% uptime), avg 30.0ms > incr 9 totalling 6ms (5.0% uptime), avg 0.7000000000000001ms > tenures 0 > > allocating: 512 times 1024000 bytes > allocated: 524.29 MB > time to run 0:00:00:02.852 > 3 times GC > time to run 0:00:00:00.196 > uptime 0h0m3s > memory 546,746,364 bytes > old 543,520,476 bytes (99.4%) > young 7,184 bytes (0.0%) > used 543,527,660 bytes (99.4%) > free 3,218,704 bytes (0.6000000000000001%) > GCs 356 (9ms between GCs) > full 89 totalling 2,743ms (86.4% uptime), avg 30.8ms > incr 267 totalling 223ms (7.0% uptime), avg 0.8ms > tenures 0 > Since last view 346 (9ms between GCs) > uptime 3.1s > full 88 totalling 2,713ms (88.80000000000001% uptime), avg 30.8ms > incr 258 totalling 217ms (7.1000000000000005% uptime), avg 0.8ms > tenures 0 > > > It takes about 2.5s to allocate about 512Mb. A GC then takes 30ms. > > You can play a bit with the 2 parameters to generate different datasets. > > Not changing any VM parameters, you can successfully do something like > > $ ./pharo Pharo.image memtest.st 1000 1024000 > > or > > $ ./pharo Pharo.image memtest.st 1024000 1000 > > thus allocating about 1GB. > > Playing with -mmap gives some weird results and unexpected slowness as well. Consider (the default -mmap is 1024m): > > $ ./pharo -mmap 1280m Pharo.image memtest.st 500 1024000 > > memtest.st > uptime 0h0m0s > memory 25,076,716 bytes > old 19,213,480 bytes (76.60000000000001%) > young 5,384 bytes (0.0%) > used 19,218,864 bytes (76.60000000000001%) > free 5,857,852 bytes (23.400000000000002%) > GCs 12 (13ms between GCs) > full 2 totalling 56ms (37.1% uptime), avg 28.0ms > incr 10 totalling 6ms (4.0% uptime), avg 0.6000000000000001ms > tenures 0 > > allocating: 500 times 1024000 bytes > allocated: 512.00 MB > time to run 0:00:00:13.66 > 3 times GC > time to run 0:00:00:00.192 > uptime 0h0m14s > memory 533,946,384 bytes > old 531,232,232 bytes (99.5%) > young 7,644 bytes (0.0%) > used 531,239,876 bytes (99.5%) > free 2,706,508 bytes (0.5%) > GCs 1,011 (14ms between GCs) > full 501 totalling 13,819ms (98.7% uptime), avg 27.6ms > incr 510 totalling 35ms (0.2% uptime), avg 0.1ms > tenures 0 > Since last view 999 (14ms between GCs) > uptime 13.9s > full 499 totalling 13,763ms (99.30000000000001% uptime), avg 27.6ms > incr 500 totalling 29ms (0.2% uptime), avg 0.1ms > tenures 0 > > > So increasing the heap from 1024 Mb to 1280 Mb slows the allocation down to 14s. > > Increasing the heap size to 1792 Mb gives the following result: > > t3@coolermaster:smalltalk$ ./pharo -mmap 1792m Pharo.image memtest.st 1280 1048576 > > memtest.st > uptime 0h0m0s > memory 25,076,708 bytes > old 19,213,592 bytes (76.60000000000001%) > young 5,384 bytes (0.0%) > used 19,218,976 bytes (76.60000000000001%) > free 5,857,732 bytes (23.400000000000002%) > GCs 12 (12ms between GCs) > full 2 totalling 55ms (37.9% uptime), avg 27.5ms > incr 10 totalling 6ms (4.1000000000000005% uptime), avg 0.6000000000000001ms > tenures 0 > > allocating: 1280 times 1048576 bytes > allocated: 1.34 GB > time to run 0:00:00:35.258 > 3 times GC > time to run 0:00:00:00.37 > uptime 0h0m35s > memory -783,340,292 bytes > old -786,061,544 bytes (100.30000000000001%) > young 7,460 bytes (0.0%) > used -786,054,084 bytes (100.30000000000001%) > free 2,713,792 bytes (-0.30000000000000004%) > GCs 2,572 (14ms between GCs) > full 1281 totalling 35,444ms (99.10000000000001% uptime), avg 27.700000000000003ms > incr 1291 totalling 93ms (0.30000000000000004% uptime), avg 0.1ms > tenures 0 > Since last view 2,560 (14ms between GCs) > uptime 35.6s > full 1279 totalling 35,389ms (99.30000000000001% uptime), avg 27.700000000000003ms > incr 1281 totalling 87ms (0.2% uptime), avg 0.1ms > tenures 0 > > > The allocation takes a lot longer, but the GC times are stable. > > I suspect that negative reporting has to do with SmallInteger limitations, but the question is if it is only a cosmetic problem or not. > > Again, I am not complaining, just exploring/wondering. > > > Sven > > >

June 29, 2013

Re: [Pharo-dev] Exploring Heap Size Limits

by Sven Van Caekenberghe

On 29 Jun 2013, at 13:18, phil(a)highoctane.be wrote: > I did some more experiments and that's the most I can get from the script, no matter how much free memory there is on the machine. > Adding ./pharo --memory 2000m ... has no effect. Yes it seems that it is not possible to allocate more than half a GB on Mac OS X. The VM's are obviously different on each platform. BTW, I didn't know that get.pharo.org works so well on OS X too ;-) Now, for server production usage only Linux counts anyway. On Mac OS X, stability is way more important to me than heap size. But it would be nice to hear something from our VM experts. Sven

June 29, 2013

Re: [Pharo-dev] VM ppa question

by Sven Van Caekenberghe

Thanks, Theirry, Damien, I did $ sudo rm /etc/apt/sources.list.d/cassou-pharo-quantal.list $ sudo add-apt-repository ppa:pharo/stable $ sudo apt-get update $ sudo apt-get install pharo-vm-core And all seems fine $ pharo-vm-nox --version 3.9-7 #1 Mon Jun 17 09:28:43 UTC 2013 gcc 4.7.2 [Production ITHB VM] NBCoInterpreter NativeBoost-CogPlugin-GuillermoPolito.19 uuid: acc98e51-2fba-4841-a965-2975997bba66 Jun 17 2013 NBCogit NativeBoost-CogPlugin-GuillermoPolito.19 uuid: acc98e51-2fba-4841-a965-2975997bba66 Jun 17 2013 git://gitorious.org/cogvm/blessed.git Commit: 0505d0c93504af2ad6dbd4cc580e6732120363d7 Date: 2013-06-12 16:26:39 +0200 By: Esteban Lorenzano <estebanlm(a)gmail.com> Jenkins build #14600 Linux wani01 2.6.24-32-xen #1 SMP Mon Dec 3 16:12:25 UTC 2012 i686 i686 i686 GNU/Linux plugin path: /usr/lib/pharo-vm/ [default: /usr/lib/pharo-vm/] Maybe the last install was not needed, I don't know. On 29 Jun 2013, at 12:13, Damien Cassou <damien.cassou(a)gmail.com> wrote: > On Sat, Jun 29, 2013 at 11:39 AM, GOUBIER Thierry > <thierry.goubier(a)cea.fr> wrote: >> I believe Damien said a while ago to switch to an "official' pharo ppa instead of his: >> >> deb http://ppa.launchpad.net/pharo/stable/ubuntu raring main > > > I think you are right. Users need to delete the old PPA reference (in > /etc/apt/sources.list.d) and add the new one > http://www.pharo.org/pharo-download/ubuntu > > -- > Damien Cassou > http://damiencassou.seasidehosting.st > > "Success is the ability to go from one failure to another without > losing enthusiasm." > Winston Churchill >

June 29, 2013

Re: [Pharo-dev] Exploring Heap Size Limits

by phil＠highoctane.be

I did some more experiments and that's the most I can get from the script, no matter how much free memory there is on the machine. Adding ./pharo --memory 2000m ... has no effect. [PhilMac:~/Documents/Smalltalk/2-MyWorkspaces/workspaceMemTest philippeback$] ./pharo Pharo.image memtest.st 501 1024000 memtest.st uptime 0h0m0s memory 25,004,444 bytes old 19,213,688 bytes (76.80000000000001%) young 5,384 bytes (0.0%) used 19,219,072 bytes (76.9%) free 5,785,372 bytes (23.1%) GCs 5 (62ms between GCs) full 1 totalling 76ms (24.5% uptime), avg 76.0ms incr 4 totalling 10ms (3.2% uptime), avg 2.5ms tenures 0 allocating: 501 times 1024000 bytes allocated: 513.02 MB time to run 0:00:00:06.27 3 times GC time to run 0:00:00:00.494 uptime 0h0m7s memory 534,974,360 bytes old 532,256,456 bytes (99.5%) young 7,092 bytes (0.0%) used 532,263,548 bytes (99.5%) free 2,710,812 bytes (0.5%) GCs 344 (21ms between GCs) full 87 totalling 5,822ms (82.2% uptime), avg 66.9ms incr 257 totalling 659ms (9.3% uptime), avg 2.6ms tenures 0 Since last view 339 (20ms between GCs) uptime 6.800000000000001s full 86 totalling 5,746ms (84.9% uptime), avg 66.8ms incr 253 totalling 649ms (9.600000000000001% uptime), avg 2.6ms tenures 0

June 29, 2013

Re: [Pharo-dev] Exploring Heap Size Limits

by phil＠highoctane.be

On OSX Lion w/ 8GB RAM, the first call gives me: [PhilMac:~/Documents/Smalltalk/2-MyWorkspaces/workspaceMemTest philippeback$] ./run.sh memtest.st uptime 0h0m0s memory 25,004,440 bytes old 19,213,728 bytes (76.80000000000001%) young 5,384 bytes (0.0%) used 19,219,112 bytes (76.9%) free 5,785,328 bytes (23.1%) GCs 5 (127ms between GCs) full 1 totalling 72ms (11.4% uptime), avg 72.0ms incr 4 totalling 11ms (1.7000000000000002% uptime), avg 2.8000000000000003ms tenures 0 allocating: 512 times 1024000 bytes =============================================================================== Notice: Errors in script loaded from /Users/philippeback/Documents/Smalltalk/2-MyWorkspaces/workspaceMemTest/ memtest.st =============================================================================== Errors in script loaded from /Users/philippeback/Documents/Smalltalk/2-MyWorkspaces/workspaceMemTest/ memtest.st ==== Startup Error: OutOfMemory ByteArray class(Behavior)>>basicNew: ByteArray class(Behavior)>>new: UndefinedObject>>DoIt in Block: [t6... SmallInteger(Integer)>>timesRepeat: UndefinedObject>>DoIt in Block: [:t6 | t1... Array class(SequenceableCollection class)>>new:streamContents: UndefinedObject>>DoIt in Block: [t4 := Array... Time class>>millisecondsToRun: BlockClosure>>timeToRun UndefinedObject>>DoIt Compiler>>evaluate:in:to:notifying:ifFail:logged: Compiler class>>evaluate:for:notifying:logged: Compiler class>>evaluate:for:logged: Compiler class>>evaluate:logged: DoItDeclaration>>import CodeImporter>>evaluate in Block: [:decl | value := decl import] OrderedCollection>>do: CodeImporter>>evaluate BasicCodeLoader>>installSourceFile: in Block: [codeImporter evaluate] BlockClosure>>on:do: BasicCodeLoader>>handleErrorsDuring:reference: BasicCodeLoader>>installSourceFile: BasicCodeLoader>>installSourceFiles in Block: [:reference | self installSourceFile: reference] OrderedCollection>>do: BasicCodeLoader>>installSourceFiles in Block: [sourceFiles... BlockClosure>>ensure: BasicCodeLoader>>installSourceFiles BasicCodeLoader>>activate BasicCodeLoader class(CommandLineHandler class)>>activateWith: DefaultCommandLineHandler>>handleSubcommand Got startup errors: OutOfMemory Now, I did a purge. Same results. Need to pass a parameter or something? This is on a fresh 2.0 + vmLatest Phil

June 29, 2013

Re: [Pharo-dev] VM ppa question

by Damien Cassou

On Sat, Jun 29, 2013 at 11:39 AM, GOUBIER Thierry <thierry.goubier(a)cea.fr> wrote: > I believe Damien said a while ago to switch to an "official' pharo ppa instead of his: > > deb http://ppa.launchpad.net/pharo/stable/ubuntu raring main I think you are right. Users need to delete the old PPA reference (in /etc/apt/sources.list.d) and add the new one http://www.pharo.org/pharo-download/ubuntu -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill

June 29, 2013

Re: [Pharo-dev] Question about closure compilation

by Marcus Denker

On Jun 29, 2013, at 11:47 AM, Norbert Hartl <norbert(a)hartl.name> wrote: >> > > Thanks. So please let us separate problems. In the example from Sven there are two problems: > > - increase of memory per function invocation. That problem we don't have Yes, you are right. I am not really knowledgeable about JavaScriptâ¦ especially the strange stuff in there. ;-) > - easily keeping a reference to memory preventing it from being garbage collected. That is what you are talking about. And to me there is nothing wrong with it > Yes. It could lead to an object not being GCed, but this should not be problem in practice. Marcus

June 29, 2013