- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Pharo 6 crash when requesting memory from the JVM library

by Ben Coman

On Tue, May 16, 2017 at 11:24 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Ben, > > >> On May 16, 2017, at 7:42 AM, Ben Coman <btc(a)openinworld.com> wrote: >> >> On Tue, May 2, 2017 at 10:36 PM, Raffaello Giulietti >> <raffaello.giulietti(a)lifeware.ch> wrote: >>> Hello, >>> >>> I'm on Pharo 6 (32 bit)/Windows 10 (64 bit). >>> >>> I'm trying to load and use the (32 bit) JVM DLL into a running Pharo image >>> via the UFFI. >>> >>> Everything works correctly, including JVM method invocations, except when >>> trying to set the minimum and maximum JVM heap size by passing the >>> -Xms<size> and -Xmx<size> options upon JVM creation. With these options, >>> Pharo simply crashes without leaving any trace. >>> >>> Apparently, memory management requests from the JVM interfere with Pharo's >>> own memory management. >>> >>> Please note that we successfully use the same mechanism for both VisualWorks >>> (32 bit)/Windows 10 (64 bit) and Gemstone 64 bit/Linux, so it seems it has >>> to do with a Pharo limitation somehow. >>> >>> Further, I couldn't find any documentation on how to increase Pharo's >>> working set. >>> >>> So the questions are: >>> * What is the most probable cause of the crash described above? >>> * Where is there more doc about Pharo's memory configuration settings? >> >> I just bumped into this which you may find interesting... >> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/v… >> * The upshot of all this is that Squeak will claim (and hold on to) >> * ALL of the available virtual memory (or at least 75% of it) when >> * it starts up. If you can't live with that, use the -memory >> * option to allocate a fixed size heap. > > Please be careful to read the code. yes, guilty. I was skimming the code looking at something else and when it caught my eye. I didn't stop to examine it, just drop the link here. Thanks for the correction. cheers -ben > This is the situation on *non-Spur* VMs (Pharo 5 and earlier, Squeak 4.x and earlier). It is *not* the case on Spur. > > In Spur on start up the VM allocates enough memory for the heap plus one "growth increment" (currently 16mb) and new space (current default about 5mb on 32 bits, 9mb on 64 bits), and for the native code zone (1mb on x86, about 1.4mb on arm and x64). > > Spur *does not* reserve address space for the heap. It requests memory for the heap in segments (default 16mb; controllable via a vmParameterAt:put: send). It returns those segments to the is when GC frees segments (the threshold being controllable via a vmParameterAt:put: send). > >> >> cheers -ben >> >

May 16, 2017

Re: [Pharo-dev] Git breaks Monticello's version numbers

by Eliot Miranda

Hi Esteban, forgive me if what I say causes conflict. I do not wish to cause conflict; quite the opposite. But I must say something on this issue as it is very important to me. On Sat, May 13, 2017 at 7:21 AM, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > On 13 May 2017, at 16:17, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > > On 13 May 2017, at 15:51, Thierry Goubier <thierry.goubier(a)gmail.com> > wrote: > > Le 13/05/2017 Ã 15:43, Esteban Lorenzano a Ã©crit : > > > > On 13 May 2017, at 13:16, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> > wrote: > > Iâm not a bit expert, but if you donât use âmetadatalessâ format > everything works fine with monticello. I.e. each git commit > contains all the mc history. > > > yes, but with iceberg we did another choice: we force metadataless > and do not keep compatibility with monticello. this is like that by > design: keeping the metadata was too much noise and generated a lot > of conflicts we donât want. > > > The metadata-less format was experimented with before Iceberg because we > knew that recreation of MC-compatible metadata out of the vcs log was > already working. > > Then FileTree (and MC) was modified to make sure that the metadata-less > format would be compatible with MC. > > Now, Iceberg has decided to be non-compatible with MC. But this has > nothing to do with the underlying format. > > > Iceberg uses filetree metadataless as file format (it uses even the same > class to do the write) so what you are saying is not true ;) > What changes is that instead adding a random cypress.1 we add the a number > which is a timestamp of the commit. > > > So, using iceberg is not using git as âjust another repository format > as http, ftp, etc.â. If you want to keep both versions, then you > need to save in mc format then save again in iceberg (or viceversa). > > > Which looks clumsy at best and at worse, will make the transition to > Iceberg a pain. > > > You say: > I disagree. > the cost of keeping eternal backward compatibility is not moving forward. > > I don't see how maintaining compatibility between Monticello and git versions of packages will stop you from moving forward. It /will/ make it much more difficult for members of the other Smalltalk communities that use the same infrastructure to share code with Pharo. So this breaking of compatibility makes the lives of those of us who work on the VM and the beetled compiler more difficult. in fact, it feels like an attack. I know it is not intended as an attack but it hurts. Pharos uses the VM that I have built with others. That VM originates in Squeak and is developed in Squeak, and I will, to my dying breath, try and keep the VM as something that works for both Squeak and Pharo (not to mention Cuis and Newspeak). I believe that openness in the VM's construction and use makes it stronger and more useful to us all, and trying to fork it so that it is Pharo only is a mistake. Further, I believe that that's the case for the whole ecosystem and your desire to prevent interoperability with "older" or non-git-based versions of Monticello will in the long term hurt the Pharo community much more than it helps it. So I beg the Pharo community to try and maintain compatibility with their Squeak and Cuis cousins. See where we are today with a VM that is demonstrably faster than the best commercial competitors and what a benefit this is for commercial adoption of Pharo. We are in this situation because we have been open. Attempts to close off are in my opinion extremely unwise. > > > And Peter did a very good tool to easy migrations that respect history. > > Also, nobody is forced to use iceberg: you donât want it, do not use it. > You still have monticello. > > > as another point: nobody ask git to be compatible with svn or mercurial. > the only thing people ask is for migration tools. > That's not true. github makes it possible to run subversion against their git repositories. > and we have it. > What compatibility there is is patchy I do see git repositories not providing any compatibility with the Monticello tools to retrieve older versions. I see a confusion of different ways of interacting with git. What is the cost of maintaining compatibility and what are the benefits? What is the cost of not being compatible and what are the benefits? Esteban > > > > > Thierry > > Esteban > > > Uko > > On 13 May 2017, at 09:28, Thierry Goubier > <thierry.goubier(a)gmail.com> wrote: > > Le 13/05/2017 Ã 08:58, Stephane Ducasse a Ã©crit : > > My gut feeling is that it will be better not to mix git and > MC. > > > It is easy to make MC compatible with Git. > > It wasn't that hard in the past, but needed a community effort > (MC being a core part of the system). Now, with the > infrastructure underway (libgit, git fast-import) it looks very > easy to implement. > > Thierry > > > > On Fri, May 12, 2017 at 11:09 PM, Oleksandr Zaytsev > <olk.zaytsev(a)gmail.com <mailto:olk.zaytsev@gmail.com > <olk.zaytsev(a)gmail.com>>> wrote: > > Hello > > Two days ago I was trying to send the slice with my fix to > PolyMath using Monticello. But the version number got set to > 1494471195. Today I realized that all the packages to which I > commit are numbered like that. > > Cyril Ferlicot explained to me that this happens when I mix git > and Monticello commits. He suggested that I use a separate > image for committing to GitHub, or file out/file in if there is > a lot of changes to commit. > > Can this be considered a bug? Should I report it? > > I think it would be causing problems for many people. > > Oleks > > _,,,^..^,,,_ best, Eliot

May 16, 2017

Re: [Pharo-dev] Does special delay scheduler class is still needed for OSSubprocess?

by Mariano Martinez Peck

Denis, Max, On master you can find the scheduler change rollbacked. I will soon release a new version 0.2.5 but not just yet. So as for testing purpose, give it a try with master. Cheers, On Tue, May 16, 2017 at 10:07 AM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Tue, May 16, 2017 at 9:55 AM, Denis Kudriashov <dionisiydk(a)gmail.com> > wrote: > >> >> 2017-05-16 14:51 GMT+02:00 Denis Kudriashov <dionisiydk(a)gmail.com>: >> >>> We can do a new release with the undo of this scheduler change and give >>>> it a try. What do you think? >>> >>> >>> yes, why not >> >> >> But does original issue was related to itimer VM problems or not? >> > > No > > -- > Mariano > http://marianopeck.wordpress.com > -- Mariano http://marianopeck.wordpress.com

May 16, 2017

Re: [Pharo-dev] Pharo 6 crash when requesting memory from the JVM library

by Eliot Miranda

Hi Ben, > On May 16, 2017, at 7:42 AM, Ben Coman <btc(a)openinworld.com> wrote: > > On Tue, May 2, 2017 at 10:36 PM, Raffaello Giulietti > <raffaello.giulietti(a)lifeware.ch> wrote: >> Hello, >> >> I'm on Pharo 6 (32 bit)/Windows 10 (64 bit). >> >> I'm trying to load and use the (32 bit) JVM DLL into a running Pharo image >> via the UFFI. >> >> Everything works correctly, including JVM method invocations, except when >> trying to set the minimum and maximum JVM heap size by passing the >> -Xms<size> and -Xmx<size> options upon JVM creation. With these options, >> Pharo simply crashes without leaving any trace. >> >> Apparently, memory management requests from the JVM interfere with Pharo's >> own memory management. >> >> Please note that we successfully use the same mechanism for both VisualWorks >> (32 bit)/Windows 10 (64 bit) and Gemstone 64 bit/Linux, so it seems it has >> to do with a Pharo limitation somehow. >> >> Further, I couldn't find any documentation on how to increase Pharo's >> working set. >> >> So the questions are: >> * What is the most probable cause of the crash described above? >> * Where is there more doc about Pharo's memory configuration settings? > > I just bumped into this which you may find interesting... > https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/v… > * The upshot of all this is that Squeak will claim (and hold on to) > * ALL of the available virtual memory (or at least 75% of it) when > * it starts up. If you can't live with that, use the -memory > * option to allocate a fixed size heap. Please be careful to read the code. This is the situation on *non-Spur* VMs (Pharo 5 and earlier, Squeak 4.x and earlier). It is *not* the case on Spur. In Spur on start up the VM allocates enough memory for the heap plus one "growth increment" (currently 16mb) and new space (current default about 5mb on 32 bits, 9mb on 64 bits), and for the native code zone (1mb on x86, about 1.4mb on arm and x64). Spur *does not* reserve address space for the heap. It requests memory for the heap in segments (default 16mb; controllable via a vmParameterAt:put: send). It returns those segments to the is when GC frees segments (the threshold being controllable via a vmParameterAt:put: send). > > cheers -ben >

May 16, 2017

Re: [Pharo-dev] [Vm-dev] Is there a way to know if I am in iTimer VM?

by Tobias Pape

[sorry that was meant for vm-dev :(] > On 16.05.2017, at 17:20, Tobias Pape <Das.Linux(a)gmx.de> wrote: > > >> On 16.05.2017, at 17:14, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> >> Hi Mariano, >> >> On May 16, 2017, at 7:48 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: >> >>> >>> >>> On Tue, May 16, 2017 at 11:43 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>> Hi Mariano, >>> >>> On May 16, 2017, at 6:35 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: >>> >>>> Guys, >>>> >>>> As you know, OSSubprocess does not work correctly in Linux unless you use the iTimer VM. >>> >>> As others have pointed out, you mean that OSSubprocess does not work correctly unless you use the threaded heartbeat VM. That the itinerary heartbeat breaks OSSubprocess. >>> >>> >>> Yes, sorry about that. >>> >>> >>>> I would like to open a warning (that growl thingy on bottom left of the screen) or something at loading / install time if I can detect I am under Linux AND under traditional linux VM (not itimer). >>>> >>>> Do you know how can I do that? >>> I made a mistake in that there is no way, other than by using the FFI, to discover if the timer heartbeat is being used. Would it be satisfactory if I provide a flag in vm parameter 48 that, if set, indicates the itimer heartbeat is in use? This could be done in a few minutes and would be ready for the official release of Pharo 6. >>>> >>> >>> Well, it would be helpful as I may be able to throw a warning. Would this flag be set only under Linux VM? >> >> Effectively, yes. It would be set in VMs that use the itimer heartbeat. Currently that's only a subset of the Linux VMs. Hopefully it will not be set in any vm soon :-) > > > Can't guarantee that to you. > Not unless the threaded-heartbeat-vm can be used hazzle-free (ie, without using sudo/..) ;) > > SCNR, sorrry > >> >>> >>> Thanks, >>> >>> >>>> -- >>>> Mariano >>>> http://marianopeck.wordpress.com >>> >>> >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >

May 16, 2017

Re: [Pharo-dev] [Vm-dev] Is there a way to know if I am in iTimer VM?

by Tobias Pape

> On 16.05.2017, at 17:14, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Mariano, > > On May 16, 2017, at 7:48 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > >> >> >> On Tue, May 16, 2017 at 11:43 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> >> Hi Mariano, >> >> On May 16, 2017, at 6:35 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: >> >>> Guys, >>> >>> As you know, OSSubprocess does not work correctly in Linux unless you use the iTimer VM. >> >> As others have pointed out, you mean that OSSubprocess does not work correctly unless you use the threaded heartbeat VM. That the itinerary heartbeat breaks OSSubprocess. >> >> >> Yes, sorry about that. >> >> >>> I would like to open a warning (that growl thingy on bottom left of the screen) or something at loading / install time if I can detect I am under Linux AND under traditional linux VM (not itimer). >>> >>> Do you know how can I do that? >> I made a mistake in that there is no way, other than by using the FFI, to discover if the timer heartbeat is being used. Would it be satisfactory if I provide a flag in vm parameter 48 that, if set, indicates the itimer heartbeat is in use? This could be done in a few minutes and would be ready for the official release of Pharo 6. >>> >> >> Well, it would be helpful as I may be able to throw a warning. Would this flag be set only under Linux VM? > > Effectively, yes. It would be set in VMs that use the itimer heartbeat. Currently that's only a subset of the Linux VMs. Hopefully it will not be set in any vm soon :-) Can't guarantee that to you. Not unless the threaded-heartbeat-vm can be used hazzle-free (ie, without using sudo/..) ;) SCNR, sorrry > >> >> Thanks, >> >> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com

May 16, 2017

Re: [Pharo-dev] [Vm-dev] Is there a way to know if I am in iTimer VM?

by Eliot Miranda

Hi Mariano, > On May 16, 2017, at 7:48 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > > >> On Tue, May 16, 2017 at 11:43 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >> >> Hi Mariano, >> >>> On May 16, 2017, at 6:35 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: >>> >>> Guys, >>> >>> As you know, OSSubprocess does not work correctly in Linux unless you use the iTimer VM. >> >> As others have pointed out, you mean that OSSubprocess does not work correctly unless you use the threaded heartbeat VM. That the itinerary heartbeat breaks OSSubprocess. > > > Yes, sorry about that. > >> >>> I would like to open a warning (that growl thingy on bottom left of the screen) or something at loading / install time if I can detect I am under Linux AND under traditional linux VM (not itimer). >>> >>> Do you know how can I do that? >> I made a mistake in that there is no way, other than by using the FFI, to discover if the timer heartbeat is being used. Would it be satisfactory if I provide a flag in vm parameter 48 that, if set, indicates the itimer heartbeat is in use? This could be done in a few minutes and would be ready for the official release of Pharo 6. >>> > > Well, it would be helpful as I may be able to throw a warning. Would this flag be set only under Linux VM? Effectively, yes. It would be set in VMs that use the itimer heartbeat. Currently that's only a subset of the Linux VMs. Hopefully it will not be set in any vm soon :-) > > Thanks, > > >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >> > > > > -- > Mariano > http://marianopeck.wordpress.com

May 16, 2017

Re: [Pharo-dev] [Vm-dev] Is there a way to know if I am in iTimer VM?

by Mariano Martinez Peck

On Tue, May 16, 2017 at 11:43 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Mariano, > > On May 16, 2017, at 6:35 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> > wrote: > > Guys, > > As you know, OSSubprocess does not work correctly in Linux unless you use > the iTimer VM. > > > As others have pointed out, you mean that OSSubprocess does not work > correctly unless you use the threaded heartbeat VM. That the itinerary > heartbeat breaks OSSubprocess. > Yes, sorry about that. > > I would like to open a warning (that growl thingy on bottom left of the > screen) or something at loading / install time if I can detect I am under > Linux AND under traditional linux VM (not itimer). > > Do you know how can I do that? > > I made a mistake in that there is no way, other than by using the FFI, to > discover if the timer heartbeat is being used. Would it be satisfactory if > I provide a flag in vm parameter 48 that, if set, indicates the itimer > heartbeat is in use? This could be done in a few minutes and would be > ready for the official release of Pharo 6. > > > Well, it would be helpful as I may be able to throw a warning. Would this flag be set only under Linux VM? Thanks, > -- > Mariano > http://marianopeck.wordpress.com > > > -- Mariano http://marianopeck.wordpress.com

May 16, 2017

Re: [Pharo-dev] Is there a way to know if I am in iTimer VM?

by Eliot Miranda

Hi Mariano, On May 16, 2017, at 6:35 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: Guys, As you know, OSSubprocess does not work correctly in Linux unless you use the iTimer VM. As others have pointed out, you mean that OSSubprocess does not work correctly unless you use the threaded heartbeat VM. That the itinerary heartbeat breaks OSSubprocess. I would like to open a warning (that growl thingy on bottom left of the screen) or something at loading / install time if I can detect I am under Linux AND under traditional linux VM (not itimer). Do you know how can I do that? I made a mistake in that there is no way, other than by using the FFI, to discover if the timer heartbeat is being used. Would it be satisfactory if I provide a flag in vm parameter 48 that, if set, indicates the itimer heartbeat is in use? This could be done in a few minutes and would be ready for the official release of Pharo 6. -- Mariano http://marianopeck.wordpress.com

May 16, 2017

Re: [Pharo-dev] Pharo 6 crash when requesting memory from the JVM library

by Ben Coman

On Tue, May 2, 2017 at 10:36 PM, Raffaello Giulietti <raffaello.giulietti(a)lifeware.ch> wrote: > Hello, > > I'm on Pharo 6 (32 bit)/Windows 10 (64 bit). > > I'm trying to load and use the (32 bit) JVM DLL into a running Pharo image > via the UFFI. > > Everything works correctly, including JVM method invocations, except when > trying to set the minimum and maximum JVM heap size by passing the > -Xms<size> and -Xmx<size> options upon JVM creation. With these options, > Pharo simply crashes without leaving any trace. > > Apparently, memory management requests from the JVM interfere with Pharo's > own memory management. > > Please note that we successfully use the same mechanism for both VisualWorks > (32 bit)/Windows 10 (64 bit) and Gemstone 64 bit/Linux, so it seems it has > to do with a Pharo limitation somehow. > > Further, I couldn't find any documentation on how to increase Pharo's > working set. > > So the questions are: > * What is the most probable cause of the crash described above? > * Where is there more doc about Pharo's memory configuration settings? I just bumped into this which you may find interesting... https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/platforms/unix/v… * The upshot of all this is that Squeak will claim (and hold on to) * ALL of the available virtual memory (or at least 75% of it) when * it starts up. If you can't live with that, use the -memory * option to allocate a fixed size heap. cheers -ben

May 16, 2017