[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations. tx [image: Inline image 1]
Hi Usman, On 08 Jul 2013, at 13:05, Usman Bhatti <usman.bhatti@gmail.com> wrote:
[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations.
tx
<Screen Shot 2013-07-08 at 12.31.38 PM.png>
I know nothing about the (extra) limitations of the Windows VM. But on Linux you can allocate close to 1GB (see mailing list). On Mac you seem to be able to get pretty close to that (see Phil's answers). So switching hardware/OS is an option too ;-) Sven -- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org
On Mon, Jul 8, 2013 at 1:11 PM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi Usman,
On 08 Jul 2013, at 13:05, Usman Bhatti <usman.bhatti@gmail.com> wrote:
[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations.
tx
<Screen Shot 2013-07-08 at 12.31.38 PM.png>
I know nothing about the (extra) limitations of the Windows VM. But on Linux you can allocate close to 1GB (see mailing list). On Mac you seem to be able to get pretty close to that (see Phil's answers). So switching hardware/OS is an option too ;-)
I would be surprised if the client agrees to it :). So that I can't (SAS will take some time to come). So, may be currently, we'll have to resort to some optimizations through serialization and de-serialization, but memory limitation is going to be a recurrent problem for us.
Sven
-- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org
This is a standard problem on Windows whenever something tries to allocate contiguous memory due to the way it fragments your exe's memory with it's own dlls. I was able to get 1.4gb with some pretty nasty hacking of an exe once by rebasing the biggest offending dlls (it wasn't Pharo but the same principles should apply). You can find them using dependency walker. Be aware it can come with side effects as some really don't want to be moved, and as such I second Sven's OS comment ;-) ________________________________ From: Usman Bhatti <usman.bhatti@gmail.com> To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Sent: Monday, 8 July 2013, 12:05 Subject: [Pharo-users] More than 500MB for windows [I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations. tx
On Mon, Jul 8, 2013 at 1:26 PM, CHRIS BAILEY <cpmbailey@btinternet.com>wrote:
This is a standard problem on Windows whenever something tries to allocate contiguous memory due to the way it fragments your exe's memory with it's own dlls. I was able to get 1.4gb with some pretty nasty hacking of an exe once by rebasing the biggest offending dlls (it wasn't Pharo but the same principles should apply). You can find them using dependency walker. Be aware it can come with side effects as some really don't want to be moved, and as such I second Sven's OS comment ;-)
Tx for the hint. I'll first try image level optimizations before descending down to OS-level.
------------------------------ *From:* Usman Bhatti <usman.bhatti@gmail.com> *To:* Any question about pharo is welcome <pharo-users@lists.pharo.org> *Sent:* Monday, 8 July 2013, 12:05 *Subject:* [Pharo-users] More than 500MB for windows
[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations.
tx
[image: Inline image 1]
I remember there was a thread where Andreas explains why the windows VM would not alloc more than 500 MB. I think it's quite similar if not the same to what Chris has said. If this is the case, MAYBE, depending on which dlls/plugins you use, you may be safe and alloc more??? Don't know... I can search the email if you want. Cheers, On Mon, Jul 8, 2013 at 8:33 AM, Usman Bhatti <usman.bhatti@gmail.com> wrote:
On Mon, Jul 8, 2013 at 1:26 PM, CHRIS BAILEY <cpmbailey@btinternet.com>wrote:
This is a standard problem on Windows whenever something tries to allocate contiguous memory due to the way it fragments your exe's memory with it's own dlls. I was able to get 1.4gb with some pretty nasty hacking of an exe once by rebasing the biggest offending dlls (it wasn't Pharo but the same principles should apply). You can find them using dependency walker. Be aware it can come with side effects as some really don't want to be moved, and as such I second Sven's OS comment ;-)
Tx for the hint. I'll first try image level optimizations before descending down to OS-level.
------------------------------ *From:* Usman Bhatti <usman.bhatti@gmail.com> *To:* Any question about pharo is welcome <pharo-users@lists.pharo.org> *Sent:* Monday, 8 July 2013, 12:05 *Subject:* [Pharo-users] More than 500MB for windows
[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today]. I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations.
tx
[image: Inline image 1]
-- Mariano http://marianopeck.wordpress.com
It's bound to be the same. I encountered this quite extensively when using VSE on Win32. There were normally a couple of system DLL's that got plonked into the middle of it's address range and just moving those to one of the outer edges would mean the image could be started with a much bigger address space. Some would move without issues, some would just refuse to budge and some would appear to work until you actually called something in the offending dll :) The simplest quick gain I could get was to use the Windows /3gb switch. This would give you an extra gb of contiguous memory and mean you could run an image just shy of 1gb. This increases to just under 2gb when you run it on a 64-bit machine. Even though the VM itself wasn't 64-bit, you still get the memory benefits. ________________________________ From: Mariano Martinez Peck <marianopeck@gmail.com> To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Cc: CHRIS BAILEY <cpmbailey@btinternet.com> Sent: Monday, 8 July 2013, 12:46 Subject: Re: [Pharo-users] More than 500MB for windows I remember there was a thread where Andreas explains why the windows VM would not alloc more than 500 MB. I think it's quite similar if not the same to what Chris has said. If this is the case, MAYBE, depending on which dlls/plugins you use, you may be safe and alloc more??? Don't know... I can search the email if you want. Cheers, On Mon, Jul 8, 2013 at 8:33 AM, Usman Bhatti <usman.bhatti@gmail.com> wrote: On Mon, Jul 8, 2013 at 1:26 PM, CHRIS BAILEY <cpmbailey@btinternet.com> wrote:
This is a standard problem on Windows whenever something tries to allocate contiguous memory due to the way it fragments your exe's memory with it's own dlls. I was able to get 1.4gb with some pretty nasty hacking of an exe once by rebasing the biggest offending dlls (it wasn't Pharo but the same principles should apply). You can find them using dependency walker. Be aware it can come with side effects as some really don't want to be moved, and as such I second Sven's OS comment ;-)
Tx for the hint. I'll first try image level optimizations before descending down to OS-level. Â
________________________________ From: Usman Bhatti <usman.bhatti@gmail.com> To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Sent: Monday, 8 July 2013, 12:05 Subject: [Pharo-users] More than 500MB for windows
[I know there were some discussions on the topic and the topic of 64-bit VM but I'm not sure where we stand today].
I would like to know if we can allocate more than 500MB for pharo virtual machine in Windows? I am getting an "Out Of memory" msg and the following VM statistics when trying to parse a large application with Moose. Increasing the size would be the cheapest option (in terms of effort) before trying other optimizations.
tx
-- Mariano http://marianopeck.wordpress.com
Same question again. And same answer (i have no other yet). Why just don't use disk memory? I know it is extra effort, but that's what all software does, when it has to deal with big amounts of data. Why in smalltalk it should be different? Of course, on your place, i would also prefer that there is some magic fairy waves its magic wand and problem solved, but it is not going to be solved. If today you need >512Mb for your data, tomorrow you will need twice as much, and after tomorrow it will double again. So, let us be realistic: the solution to your problem lies not in VM, but in a way how to manage the data. -- Best regards, Igor Stasenko.
Hi, Igor, please get this point of view out of your system :). First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it. I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now. After all, we are here to change the world :). Cheers, Doru On Jul 8, 2013, at 4:50 PM, Igor Stasenko <siguctua@gmail.com> wrote:
Same question again. And same answer (i have no other yet).
Why just don't use disk memory? I know it is extra effort, but that's what all software does, when it has to deal with big amounts of data. Why in smalltalk it should be different?
Of course, on your place, i would also prefer that there is some magic fairy waves its magic wand and problem solved, but it is not going to be solved. If today you need >512Mb for your data, tomorrow you will need twice as much, and after tomorrow it will double again. So, let us be realistic: the solution to your problem lies not in VM, but in a way how to manage the data.
-- Best regards, Igor Stasenko.
-- www.tudorgirba.com "Sometimes the best solution is not the best solution."
I am with Doru here. Today, 500MB is NOTHINGGGGGG. NOTHIIIIIIIIING. NOTHING. A LATOP comes with 8GB. 16x more than the max the VM can allocate. For one of my clients, I have just set up a server with 32GB. It has 64x than the max a Pharo VM can allocate. Really? Going to disk is not always a possibility for many many reasons. And even if it were possible....if I have to spent XXX hours to workaround Pharo because it cannot use more than 500MB, then people will simply move to another language. And finally, wasn't a language like Smalltalk prepare for you and let you only focus in your app blah blah blah.. ? having to go to disk doesn't sound like that... So, Igor, I cannot stand that point of view. I can stand many others, like explaining why it's not Pharo fault, the problem window has, the money/time it would take for an engineer to fix it, etc etc etc. But not your position. Cheers, On Mon, Jul 8, 2013 at 5:43 PM, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Cheers, Doru
On Jul 8, 2013, at 4:50 PM, Igor Stasenko <siguctua@gmail.com> wrote:
Same question again. And same answer (i have no other yet).
Why just don't use disk memory? I know it is extra effort, but that's what all software does, when it has to deal with big amounts of data. Why in smalltalk it should be different?
Of course, on your place, i would also prefer that there is some magic fairy waves its magic wand and problem solved, but it is not going to be solved. If today you need >512Mb for your data, tomorrow you will need twice as much, and after tomorrow it will double again. So, let us be realistic: the solution to your problem lies not in VM, but in a way how to manage the data.
-- Best regards, Igor Stasenko.
-- www.tudorgirba.com
"Sometimes the best solution is not the best solution."
-- Mariano http://marianopeck.wordpress.com
On 8 July 2013 22:52, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
I am with Doru here. Today, 500MB is NOTHINGGGGGG. NOTHIIIIIIIIING. NOTHING. A LATOP comes with 8GB. 16x more than the max the VM can allocate. For one of my clients, I have just set up a server with 32GB. It has 64x than the max a Pharo VM can allocate. Really? Going to disk is not always a possibility for many many reasons. And even if it were possible....if I have to spent XXX hours to workaround Pharo because it cannot use more than 500MB, then people will simply move to another language. And finally, wasn't a language like Smalltalk prepare for you and let you only focus in your app blah blah blah.. ? having to go to disk doesn't sound like that...
So, Igor, I cannot stand that point of view. I can stand many others, like explaining why it's not Pharo fault, the problem window has, the money/time it would take for an engineer to fix it, etc etc etc. But not your position.
My point is that people should stop raising this topic again and again and do something about it. Either rewrite memory management in VM, to dynamically allocate address space, or rewrite own application to use disk memory. And i know that 512mb is nothing, especially for application which deals with big amounts of data. But from that perspective, 8GB is nothing as well. Be realistic: if you need to generate/process lots of data, you should not rely on just operative memory, it is pretty limited resource, comparing to disk memory. It was like that from the very beginning, and i think it will stay like that for a while.
Cheers, -- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On Jul 9, 2013, at 9:16 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:52, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
I am with Doru here. Today, 500MB is NOTHINGGGGGG. NOTHIIIIIIIIING. NOTHING. A LATOP comes with 8GB. 16x more than the max the VM can allocate. For one of my clients, I have just set up a server with 32GB. It has 64x than the max a Pharo VM can allocate. Really? Going to disk is not always a possibility for many many reasons. And even if it were possible....if I have to spent XXX hours to workaround Pharo because it cannot use more than 500MB, then people will simply move to another language. And finally, wasn't a language like Smalltalk prepare for you and let you only focus in your app blah blah blah.. ? having to go to disk doesn't sound like that...
So, Igor, I cannot stand that point of view. I can stand many others, like explaining why it's not Pharo fault, the problem window has, the money/time it would take for an engineer to fix it, etc etc etc. But not your position.
My point is that people should stop raising this topic again and again and do something about it. Either rewrite memory management in VM, to dynamically allocate address space, or rewrite own application to use disk memory.
And i know that 512mb is nothing, especially for application which deals with big amounts of data. But from that perspective, 8GB is nothing as well. Be realistic: if you need to generate/process lots of data, you should not rely on just operative memory, it is pretty limited resource, comparing to disk memory. It was like that from the very beginning, and i think it will stay like that for a while.
"640K ought to be enough for anybody." -- Bill Gates
Cheers, -- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
On 9 July 2013 10:01, Benjamin <benjamin.vanryseghem.pharo@gmail.com> wrote:
On Jul 9, 2013, at 9:16 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:52, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
I am with Doru here. Today, 500MB is NOTHINGGGGGG. NOTHIIIIIIIIING. NOTHING. A LATOP comes with 8GB. 16x more than the max the VM can allocate. For one of my clients, I have just set up a server with 32GB. It has 64x than the max a Pharo VM can allocate. Really? Going to disk is not always a possibility for many many reasons. And even if it were possible....if I have to spent XXX hours to workaround Pharo because it cannot use more than 500MB, then people will simply move to another language. And finally, wasn't a language like Smalltalk prepare for you and let you only focus in your app blah blah blah.. ? having to go to disk doesn't sound like that...
So, Igor, I cannot stand that point of view. I can stand many others, like explaining why it's not Pharo fault, the problem window has, the money/time it would take for an engineer to fix it, etc etc etc. But not your position.
My point is that people should stop raising this topic again and again and do something about it. Either rewrite memory management in VM, to dynamically allocate address space, or rewrite own application to use disk memory.
And i know that 512mb is nothing, especially for application which deals with big amounts of data. But from that perspective, 8GB is nothing as well. Be realistic: if you need to generate/process lots of data, you should not rely on just operative memory, it is pretty limited resource, comparing to disk memory. It was like that from the very beginning, and i think it will stay like that for a while.
"640K ought to be enough for anybody." -- Bill Gates
Ben, you wanted to add something constructive to discussion?
Cheers, -- Mariano http://marianopeck.wordpress.com
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
On 8 July 2013 22:43, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it? All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h And change this: #ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Cheers, Doru
-- Best regards, Igor Stasenko.
On Jul 9, 2013, at 9:10 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:43, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Which reason? (It is a real question, I do not remember why it is so low) Esteban
Cheers, Doru
-- Best regards, Igor Stasenko.
On 9 July 2013 09:13, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On Jul 9, 2013, at 9:10 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:43, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Which reason? (It is a real question, I do not remember why it is so low)
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
Esteban
-- Best regards, Igor Stasenko.
On Jul 9, 2013, at 9:22 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On Jul 9, 2013, at 9:10 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:43, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Which reason? (It is a real question, I do not remember why it is so low)
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
yep, but still could be doubled and have enough reserved memory for externals, I think. In any case, we need a solution, 512m is obviously too small (no matter if 1G is still not enough, it would be a lot better). Not to extend memory because "is still not enough" is like not healing a guy because "he will be dead old anyway" :)
Esteban
-- Best regards, Igor Stasenko.
On 9 July 2013 09:50, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On Jul 9, 2013, at 9:22 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On Jul 9, 2013, at 9:10 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 8 July 2013 22:43, Tudor Girba <tudor@tudorgirba.com> wrote:
Hi,
Igor, please get this point of view out of your system :).
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Which reason? (It is a real question, I do not remember why it is so low)
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
yep, but still could be doubled and have enough reserved memory for externals, I think. In any case, we need a solution, 512m is obviously too small (no matter if 1G is still not enough, it would be a lot better).
Not to extend memory because "is still not enough" is like not healing a guy because "he will be dead old anyway" :)
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745... please check
Esteban
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
But is it not the same on mac? I thought that it was a problem with the memory fragmentation on windows. Stef
In fact, problem is even worst. I wanted to watch how the GC behaves over 500m of memory occupied. I though on a test that would create/destroy objects in a regular basis, after filling the memory. So, to start I ran this simple script (it was just a start, not the real test I thought): objectQuantity := 1024 * 1024. objectSize := 1024. objects := Array new: objectQuantity. 1 to: objectQuantity do: [ :index | objects at: index put: (ByteArray new: objectSize) ]. objects inspect. FileStream stdout << Smalltalk vm statisticsReport << '=================' << String cr. The idea is to keep in memory a real big array with elements of 1024 bytes size each. That should fill memory up to 1G+size of array+rest of the system. Well, this are the results: OSX: VM crash (in 530m) Linux: VM crash (in ~1G) Win: Didn't even tried. Same script with: objectQuantity := 512 * 1024. OSX: VM crash (in 530m) Linux: PASSES! Win: Didn't even tried. Same script with: objectQuantity := 248 * 1024. OSX: PASSES! Linux: PASSES! Win: Didn't tried, but it should pass since memory is < 512m. Anyway... situation is far from good, IMO. Esteban On Jul 9, 2013, at 10:11 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
But is it not the same on mac? I thought that it was a problem with the memory fragmentation on windows.
Stef
Ouch this is why we will have to think hard on our next priorities. Stef On Jul 9, 2013, at 3:49 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
In fact, problem is even worst. I wanted to watch how the GC behaves over 500m of memory occupied. I though on a test that would create/destroy objects in a regular basis, after filling the memory.
So, to start I ran this simple script (it was just a start, not the real test I thought):
objectQuantity := 1024 * 1024. objectSize := 1024. objects := Array new: objectQuantity.
1 to: objectQuantity do: [ :index | objects at: index put: (ByteArray new: objectSize) ].
objects inspect.
FileStream stdout << Smalltalk vm statisticsReport << '=================' << String cr.
The idea is to keep in memory a real big array with elements of 1024 bytes size each. That should fill memory up to 1G+size of array+rest of the system.
Well, this are the results:
OSX: VM crash (in 530m) Linux: VM crash (in ~1G) Win: Didn't even tried.
Same script with:
objectQuantity := 512 * 1024.
OSX: VM crash (in 530m) Linux: PASSES! Win: Didn't even tried.
Same script with:
objectQuantity := 248 * 1024.
OSX: PASSES! Linux: PASSES! Win: Didn't tried, but it should pass since memory is < 512m.
Anyway... situation is far from good, IMO.
Esteban
On Jul 9, 2013, at 10:11 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
But is it not the same on mac? I thought that it was a problem with the memory fragmentation on windows.
Stef
By tweaking the plist on OSX I was able to allocate much more than what you mention. See other mails in the list. Phil On Tue, Jul 9, 2013 at 9:09 PM, Stéphane Ducasse <stephane.ducasse@inria.fr>wrote:
Ouch this is why we will have to think hard on our next priorities.
Stef
On Jul 9, 2013, at 3:49 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
In fact, problem is even worst. I wanted to watch how the GC behaves over 500m of memory occupied. I though on a test that would create/destroy objects in a regular basis, after filling the memory.
So, to start I ran this simple script (it was just a start, not the real test I thought):
objectQuantity := 1024 * 1024. objectSize := 1024. objects := Array new: objectQuantity.
1 to: objectQuantity do: [ :index | objects at: index put: (ByteArray new: objectSize) ].
objects inspect.
FileStream stdout << Smalltalk vm statisticsReport << '=================' << String cr.
The idea is to keep in memory a real big array with elements of 1024 bytes size each. That should fill memory up to 1G+size of array+rest of the system.
Well, this are the results:
OSX: VM crash (in 530m) Linux: VM crash (in ~1G) Win: Didn't even tried.
Same script with:
objectQuantity := 512 * 1024.
OSX: VM crash (in 530m) Linux: PASSES! Win: Didn't even tried.
Same script with:
objectQuantity := 248 * 1024.
OSX: PASSES! Linux: PASSES! Win: Didn't tried, but it should pass since memory is < 512m.
Anyway... situation is far from good, IMO.
Esteban
On Jul 9, 2013, at 10:11 AM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
If you reserve most of memory for objects, think how you could use things like freetype, opengl, cairo & any other library which allocates memory on conventional heap. also, think that DLLs and kernel needs breathing space as well.
But is it not the same on mac? I thought that it was a problem with the memory fragmentation on windows.
Stef
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it. Stef
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Stef
-- Best regards, Igor Stasenko.
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves. And then they will come back complaining that certain things don't work when they raise the limit too high ;-) One day we will have a proper 64-bit VM.
Stef
-- Best regards, Igor Stasenko.
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
Stef
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
On 9 July 2013 10:21, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
.. because reserving 512Mb for image which barely eats 30Mb is a huge waste.
Stef
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
Am 09.07.2013 um 11:01 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 10:21, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
.. because reserving 512Mb for image which barely eats 30Mb is a huge waste.
Yes, use cases are different. Therefor there should be a vm switch where you can choose the maximum memory to reserve, to heap size at start, the maximum heap size, etc. Norbert
On 9 July 2013 12:14, Norbert Hartl <norbert@hartl.name> wrote:
Am 09.07.2013 um 11:01 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 10:21, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
.. because reserving 512Mb for image which barely eats 30Mb is a huge waste.
Yes, use cases are different. Therefor there should be a vm switch where you can choose the maximum memory to reserve, to heap size at start, the maximum heap size, etc.
and so i added it. But i hate doing that. Because such switch should not exist and VM should be able to adapt to application's memory requirements automatically, without requiring user's input. Because then you have to write manual, explaining what is address space, and why users should be educated about it in order to use VM properly. And every other switch like this one you add, it makes manual bigger and bigger, up to the point, that it turns into complete mess of inter-depending, conflicting switches, some of them obsolete, some of them tricky and nobody knows how to use them properly (and that happens for sure).
Norbert
-- Best regards, Igor Stasenko.
Am 09.07.2013 um 13:43 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 12:14, Norbert Hartl <norbert@hartl.name> wrote:
Am 09.07.2013 um 11:01 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 10:21, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
.. because reserving 512Mb for image which barely eats 30Mb is a huge waste.
Yes, use cases are different. Therefor there should be a vm switch where you can choose the maximum memory to reserve, to heap size at start, the maximum heap size, etc.
and so i added it. But i hate doing that. Because such switch should not exist and VM should be able to adapt to application's memory requirements automatically, without requiring user's input. Because then you have to write manual, explaining what is address space, and why users should be educated about it in order to use VM properly. And every other switch like this one you add, it makes manual bigger and bigger, up to the point, that it turns into complete mess of inter-depending, conflicting switches, some of them obsolete, some of them tricky and nobody knows how to use them properly (and that happens for sure).
I disagree. I think it is good if the VM is able to adapt so in the case where no switch is used the VM does its best to adapt to the requirements. But there are _always_ use cases where you need to fine-tune. If you have an application that leaks memory you certainly want to restrict the maximum memory it can waste. Theoretically this happens when the OS runs out of memory and so the application will die. But before doing this secondary memory is allocated (mostly swap space) that when used kills the whole machine instead of just your application. The alternative is to use the OS to limit the resource (e.g. using ulimit) which is not working all the time. I think having those switches and being able to ignore them for most of the time is a good model to head for. Norbert
+1 and we are not users. We are programmers :) Stef
I disagree. I think it is good if the VM is able to adapt so in the case where no switch is used the VM does its best to adapt to the requirements. But there are _always_ use cases where you need to fine-tune. If you have an application that leaks memory you certainly want to restrict the maximum memory it can waste. Theoretically this happens when the OS runs out of memory and so the application will die. But before doing this secondary memory is allocated (mostly swap space) that when used kills the whole machine instead of just your application. The alternative is to use the OS to limit the resource (e.g. using ulimit) which is not working all the time.
I think having those switches and being able to ignore them for most of the time is a good model to head for.
Norbert
and so i added it. But i hate doing that. Because such switch should not exist and VM should be able to adapt to application's memory requirements automatically, without requiring user's input.
I'm like you but I built a shield of pragmatism else I would have never started pharo :)
Because then you have to write manual, explaining what is address space, and why users should be educated about it in order to use VM properly.
But we should first make sure people can get money for project in Pharo and after I'm sure that we will get money to build the next generation.
And every other switch like this one you add, it makes manual bigger and bigger, up to the point, that it turns into complete mess of inter-depending, conflicting switches, some of them obsolete, some of them tricky and nobody knows how to use them properly (and that happens for sure).
But this is better than do not existing because people can not deploy professional applications with Pharo. Life is a tradeoff
Norbert
-- Best regards, Igor Stasenko.
Am 09.07.2013 um 10:21 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
Igor, I don't understand your point here. In an environment where you can neither use nor reserve memory upon a certain limit I find the distinction useless. Usually you tend to tell that you don't want to think about the "far future right things" but the things that solve actual problems. Where has this attitude gone? :) Norbert
On 9 July 2013 12:12, Norbert Hartl <norbert@hartl.name> wrote:
Am 09.07.2013 um 10:21 schrieb Igor Stasenko <siguctua@gmail.com>:
On 9 July 2013 10:08, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 09 Jul 2013, at 10:03, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Yeah ! Now users can set the limit themselves.
And then they will come back complaining that certain things don't work when they raise the limit too high ;-)
One day we will have a proper 64-bit VM.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
Igor,
I don't understand your point here. In an environment where you can neither use nor reserve memory upon a certain limit I find the distinction useless. Usually you tend to tell that you don't want to think about the "far future right things" but the things that solve actual problems. Where has this attitude gone? :)
Strange, that you had such impression. :) Usually i tend a lot about future things, and that's why i really hate temporary solutions, which work today but break tomorrow. I am idealist, and that's why i trying to fix what not broken time to time.
Norbert
-- Best regards, Igor Stasenko.
Yes, i wish we can be there today. (And one day people will learn a difference between actual memory used and reserved address space :)
Igor this is not that we are not aware that Moose should be implemented differently but it is not simple. - the first design in 1998 was with a database access in mind so we were always doing aModel giveEntityWith: anId instead of using pointers. - I was fedup not to be able to pointers so we went to a pointer solution. - we have a mechanism to represent less objects but losing granularity. - Now cutting a graph of objects is not simple. We were thinking to see if we can use marea. The problem is that if one object of the graph is use then the part of the graph that was saved with it will be loaded back in memory and if we have not enough memory this is even more a problem. So I do not really know how to do it. Because
For the people that did not click on the link :) Tree SHA1: 26bf913 Parent SHA1: 14cc218 (- fixed save for 2.0) raw diff | raw patch Added option in windows .ini file to set the virtual memory address space limit. AddressSpaceLimit = <size in Mbytes> The default is 512Mb Thanks igor. Stef On Jul 9, 2013, at 10:03 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Stef
-- Best regards, Igor Stasenko.
Tx igor. Now I am waiting to get hold of the new VM (with this parameter) to test on Windows. On Tue, Jul 9, 2013 at 8:44 PM, Stéphane Ducasse <stephane.ducasse@inria.fr>wrote:
For the people that did not click on the link :)
- *Tree SHA1:* 26bf913<https://gitorious.org/cogvm/blessed/trees/341e6c2e150bcde80dfea6c890bab7745b...> - *Parent SHA1:* 14cc218<https://gitorious.org/cogvm/blessed/commit/14cc21817f90150b00f2c6ad2d069698f...> (- fixed save for 2.0) - raw diff<https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...> | raw patch<https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...>
Added option in windows .ini file to set the virtual memory address space limit.
AddressSpaceLimit = <size in Mbytes> The default is 512Mb
Thanks igor.
Stef
On Jul 9, 2013, at 10:03 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Stef
-- Best regards, Igor Stasenko.
On 10 July 2013 00:07, Usman Bhatti <usman.bhatti@gmail.com> wrote:
Tx igor. Now I am waiting to get hold of the new VM (with this parameter) to test on Windows.
yes, i am building it right now on my machine to test if it works, and jenkins refuses to build it (but for other reasons, which we need to fix asap)
On Tue, Jul 9, 2013 at 8:44 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
For the people that did not click on the link :)
Tree SHA1: 26bf913 Parent SHA1: 14cc218 (- fixed save for 2.0) raw diff | raw patch
Added option in windows .ini file to set the virtual memory address space limit.
AddressSpaceLimit = <size in Mbytes> The default is 512Mb
Thanks igor.
Stef
On Jul 9, 2013, at 10:03 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Stef
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
Wow! This is really exciting. Thanks a lot, Igor. I value this help even more knowing how you feel about the modification. Or maybe you changed your mind in the meantime? :) Cheers, Doru On Jul 10, 2013, at 1:35 PM, Igor Stasenko <siguctua@gmail.com> wrote:
On 10 July 2013 00:07, Usman Bhatti <usman.bhatti@gmail.com> wrote:
Tx igor. Now I am waiting to get hold of the new VM (with this parameter) to test on Windows.
yes, i am building it right now on my machine to test if it works, and jenkins refuses to build it (but for other reasons, which we need to fix asap)
On Tue, Jul 9, 2013 at 8:44 PM, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
For the people that did not click on the link :)
Tree SHA1: 26bf913 Parent SHA1: 14cc218 (- fixed save for 2.0) raw diff | raw patch
Added option in windows .ini file to set the virtual memory address space limit.
AddressSpaceLimit = <size in Mbytes> The default is 512Mb
Thanks igor.
Stef
On Jul 9, 2013, at 10:03 AM, Igor Stasenko <siguctua@gmail.com> wrote:
On 9 July 2013 09:13, Stéphane Ducasse <stephane.ducasse@inria.fr> wrote:
First, not all programming languages are like this: You can easily run other VMs (e.g., Java) with more than 3 GB. But, let's not even go there: I can run Pharo with 1Gb on Mac without problems. According to your reasoning we might end up downgrading the Mac VM. Some data does not fit in memory, but if I can get all my data in my image, I will choose to do it.
I am not saying that we should compare with Java, or that it is the end of the world that the Windows VM is highly restricted. I am simply saying that we should not dismiss this as a problem just because we do not know how, or do not have the resources to solve it right now.
After all, we are here to change the world :).
Yes, but this topic was raised multiple times already. Maybe we should stop wasting time on it?
All you need to do, to change the limit, go to platforms/win32/vm/sqWin32Alloc.h
And change this:
#ifndef MAX_VIRTUAL_MEMORY #define MAX_VIRTUAL_MEMORY 512*1024*1024 #endif
So, if you want more, build VM with any limit you see fit. But there's a reason why in official VM its 512.
Igor this is a real issue and we are all fighting to get more members in the consortium so that one day we can hire a guy like you to address it.
https://gitorious.org/cogvm/blessed/commit/341e6c2e150bcde80dfea6c890bab7745...
Stef
-- Best regards, Igor Stasenko.
-- Best regards, Igor Stasenko.
-- www.tudorgirba.com "Reasonable is what we are accustomed with."
On 10 July 2013 13:41, Tudor Girba <tudor@tudorgirba.com> wrote:
Wow!
This is really exciting.
Thanks a lot, Igor. I value this help even more knowing how you feel about the modification. Or maybe you changed your mind in the meantime? :)
No. It just a pain, knowing that it can do better, but to do it you have to change how VM manages the memory, which is much more work than what i did. I am not looking from position of "making customer happy", but "making myself happy", because apparently i am also using same VM and i want it to shine not just from outside, but inside as well. That's why i am grumbling :) So, okay.. i ran following: | objectQuantity objectSize objects fs | objectQuantity := 1024. objectSize := 1024*1024. objects := Array new: objectQuantity. fs := 'allocation.log' asFileReference ensureDeleted writeStream. 1 to: objectQuantity do: [ :index | objects at: index put: (ByteArray new: objectSize). fs nextPutAll: index asString; cr. fs flush ]. and run it on windows, and it actually behaved quite well, at 472 Mb allocated , i got a low-space warning popup. And no ugly crash. (note it cannot be strictly 512Mb, because some space already taken by image + jit + etc.. ) now, i set AddressSpaceLimit=1024 in .ini file and run the same again, and it stopped at 984 (megs allocated), again showing low-space warning (but it took a whiile to show it.. and image responding quite slow.. maybe because i run on virtual box and it is more memory than i gave to boxed windows, so it forced to do a lot of disk swapping. so, it works. -- Best regards, Igor Stasenko.
participants (11)
-
Benjamin -
CHRIS BAILEY -
Esteban Lorenzano -
Igor Stasenko -
Mariano Martinez Peck -
Norbert Hartl -
phil＠highoctane.be -
Stéphane Ducasse -
Sven Van Caekenberghe -
Tudor Girba -
Usman Bhatti