November 2013 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [Vm-dev] Frequent SegFaults in PharoVM with Pharo 3.0

by Eliot Miranda

On Mon, Nov 25, 2013 at 9:37 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 25 Nov 2013, at 17:56, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > > > Yeah you cannot compile without inlining specific messages. You can > disable inlining of #timesRepeat: at image level because we added it for > fun to see if it was easy to do in Opal. Now all other optimizations were > in the old compiler and are mandatories. However, you can now in Opal > disable these optimized messages in a restricted area (such as a class and > its subclasses). > > > > To me, it is kind of a bug that we cannot disable those optimizations, > we should be able to do it (so 1 day in the very very far future we could > have the JIT inlining these control structure allowing us to implement all > these inlined messages in all classes). But one thing is that some methods > such as #whileTrue are not implemented in a way they really work (they have > not stop condition, so if the compiler does not inline it, it gives you an > infinite loop). Another thing is the interrupt point problem as Marcus said. > > > > I don't think your other bug is related Opal optimization, only > #timesRepeat: is new and therefore can be faulty (and in this case, the > bytecode is correct, so I guess it is faulty because it removes an > interrupt point). > > It is a bit scary that there is no explicit, crystal clear list of those > interrupt critical points, especially if skipping them can lead to VM > crashes. > Before we raise the alarm the first thing is to identify the cause of the VM crash. It could still be a VM bug. First track down the bug. Don't try and find work-arounds. Try and find the cause. Only then can one develop a correct fix. Anything else buries the true cause under hacks, a process that is truly scary ;-) > > > 2013/11/25 Marcus Denker <marcus.denker(a)inria.fr> > > > > On 25 Nov 2013, at 16:26, Max Leske <maxleske(a)gmail.com> wrote: > > > > > Thanks ClÃ©ment, that seems to be it. Disabling the timesRepeat > inlining makes most of the builds run through but not all of them. It seems > like thereâs another (Opal related?) problem (with the exact same > symptomsâ¦). > > > > > > Iâve tried disabling all options, just to see what happens: all builds > fail with SegFaultsâ¦ > > > > You can not compile without optimisations. There are places in the image > that would not work with e.g. whileTrue: optimisation disabled as it would > e.g. add an interrupt point where none is now > > And other things, like performance, or e.g. the loop that goes over all > objects, if that creates objects while running you have a problemâ¦ > > > > Marcus > > > > > -- best, Eliot

Nov. 25, 2013

Re: [Pharo-dev] [Vm-dev] Frequent SegFaults in PharoVM with Pharo 3.0

by Sven Van Caekenberghe

On 25 Nov 2013, at 17:56, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > Yeah you cannot compile without inlining specific messages. You can disable inlining of #timesRepeat: at image level because we added it for fun to see if it was easy to do in Opal. Now all other optimizations were in the old compiler and are mandatories. However, you can now in Opal disable these optimized messages in a restricted area (such as a class and its subclasses). > > To me, it is kind of a bug that we cannot disable those optimizations, we should be able to do it (so 1 day in the very very far future we could have the JIT inlining these control structure allowing us to implement all these inlined messages in all classes). But one thing is that some methods such as #whileTrue are not implemented in a way they really work (they have not stop condition, so if the compiler does not inline it, it gives you an infinite loop). Another thing is the interrupt point problem as Marcus said. > > I don't think your other bug is related Opal optimization, only #timesRepeat: is new and therefore can be faulty (and in this case, the bytecode is correct, so I guess it is faulty because it removes an interrupt point). It is a bit scary that there is no explicit, crystal clear list of those interrupt critical points, especially if skipping them can lead to VM crashes. > 2013/11/25 Marcus Denker <marcus.denker(a)inria.fr> > > On 25 Nov 2013, at 16:26, Max Leske <maxleske(a)gmail.com> wrote: > > > Thanks ClÃ©ment, that seems to be it. Disabling the timesRepeat inlining makes most of the builds run through but not all of them. It seems like thereâs another (Opal related?) problem (with the exact same symptomsâ¦). > > > > Iâve tried disabling all options, just to see what happens: all builds fail with SegFaultsâ¦ > > You can not compile without optimisations. There are places in the image that would not work with e.g. whileTrue: optimisation disabled as it would e.g. add an interrupt point where none is now > And other things, like performance, or e.g. the loop that goes over all objects, if that creates objects while running you have a problemâ¦ > > Marcus >

Nov. 25, 2013

Re: [Pharo-dev] VM with fragmented memory

by Stefan Marr

Hi: On 25 Nov 2013, at 17:46, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > An interesting point in V8 is that the data and pointers are not in the same spaces. It would mean in Cog, if I understand correctly, that byte and word objects are not in the same space as other objects. I guess this simplifies the GC logic. I don't know if they did that in VW. > > > I have a general question regarding VM. Are you aware of other Virtual machines supporting this? Does the Java VM support fragmented memory? .Net? V8? > > > > Any VM that uses the train algorithm supports segmented memory. Any VM that uses "pools" or "regions" uses segmented memory. VisualWorks' VM supports segments. I'm pretty sure that V8 .Net & HotSpot provide segmented memory. > > > > > > I would like to mention this in a research paper I am working on. A very general reference could be the Garbage Collection Handbook (Jones, Hosking, Moss) chapter 8 and 10: Partitioning the heap 8.1 Terminology 8.2 Why to partition Partitioning by mobility Partitioning by size Partitioning for space Partitioning by kind Partitioning for yield Partitioning to reduce pause time Partitioning for locality Partitioning by thread Partitioning by availability Partitioning by mutability 8.3 How to partition 8.4 When to partition Chapter 10 elaborates on large objects and pointer-free objects (as well as algorithms that use partitioning). Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525

Nov. 25, 2013

Re: [Pharo-dev] [Vm-dev] Frequent SegFaults in PharoVM with Pharo 3.0

by ClÃ©ment Bera

Yeah you cannot compile without inlining specific messages. You can disable inlining of #timesRepeat: at image level because we added it for fun to see if it was easy to do in Opal. Now all other optimizations were in the old compiler and are mandatories. However, you can now in Opal disable these optimized messages in a restricted area (such as a class and its subclasses). To me, it is kind of a bug that we cannot disable those optimizations, we should be able to do it (so 1 day in the very very far future we could have the JIT inlining these control structure allowing us to implement all these inlined messages in all classes). But one thing is that some methods such as #whileTrue are not implemented in a way they really work (they have not stop condition, so if the compiler does not inline it, it gives you an infinite loop). Another thing is the interrupt point problem as Marcus said. I don't think your other bug is related Opal optimization, only #timesRepeat: is new and therefore can be faulty (and in this case, the bytecode is correct, so I guess it is faulty because it removes an interrupt point). 2013/11/25 Marcus Denker <marcus.denker(a)inria.fr> > > On 25 Nov 2013, at 16:26, Max Leske <maxleske(a)gmail.com> wrote: > > > Thanks ClÃ©ment, that seems to be it. Disabling the timesRepeat inlining > makes most of the builds run through but not all of them. It seems like > thereâs another (Opal related?) problem (with the exact same symptomsâ¦). > > > > Iâve tried disabling all options, just to see what happens: all builds > fail with SegFaultsâ¦ > > You can not compile without optimisations. There are places in the image > that would not work with e.g. whileTrue: optimisation disabled as it would > e.g. add an interrupt point where none is now > And other things, like performance, or e.g. the loop that goes over all > objects, if that creates objects while running you have a problemâ¦ > > Marcus >

Nov. 25, 2013

Re: [Pharo-dev] VM with fragmented memory

by ClÃ©ment Bera

Hello, To answer your question, V8 definitely supports memory segments. I don't know about papers but in the code each segment is an instance of Heap (see heap.cc and heap.h in the source). The 2 young spaces are in another contiguous segment of 8Mb. Each segment of V8 are one of this kind (all instances of Heap): * NewSpace: New object allocations go here. Capacity is 8MB. It's divided into two contiguous semispaces, and periodically collected using Cheney's algorithm. Uses bump-allocation. * OldSpaces: These allocate memory in 64-128KB chunks of 8KB pages. They come in the following flavors: * MapSpace: Exclusively for Maps (similar to js::Shape), and Maps *always* go here. * CodeSpace: Exclusively for Code. * DataSpace: For flat strings and numbers; can't reference objects, except for maps. * PointerSpace: For anything not a Map or Code that can reference other objects. * CellSpace: Seems to store global properties or something. * LOSpace: For large objects, where "large" means it needs more than ~8KB of storage. I have no proof for hotspot (their code is too complex) but I'm sure they have it too. For the other question (adding elements to OrderedCollection), Eliot will answer it in more details, but the main idea is that you have the heap chopped into segments of a fixed size (1Mb in Cog, and 64-128KB chunks in V8), therefore for object bigger than a segment, you need to handle specifically the object, i.e. having a segment of the size of the object. If you add 30 000 elements to an OrderedCollection, the object will still be under 1MB so it is ok, now if you a 3 million of elements to this OrderedCollection, you need to handle specifically, Therefore you have performance loss for object of more than 1Mb but there are not common so you will not notice it (you can hardly have thousands of objects bigger than 1Mb allocated and deallocated regularly, and you need this amount to notice the performance loss). An interesting point in V8 is that the data and pointers are not in the same spaces. It would mean in Cog, if I understand correctly, that byte and word objects are not in the same space as other objects. I guess this simplifies the GC logic. I don't know if they did that in VW. 2013/11/25 Alexandre Bergel <alexandre.bergel(a)me.com> > Hi Eliot, > > I have a question still on segmented memory. > > Consider an object âc := OrderedCollection newâ. It is likely that the > internal "arrayâ and the object c will be in the same generation and in the > same memory segment. > If I add, letâs say, 30 000 000 elements into c. After the addition the > array will be very very big. Which means that it may be in a different > segment memory than the one that contains âcâ. Do you feel this may be a > problem? I am thinking about a long jump between c and array. Long jumps > cost more than short jumps right? > > Do you feel there are some suboptimal situation with this situation? > > Alexandre > > NB: I cc the mailing list to see whether other may help in understand > better whatâs going on with segment and generations > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > On Nov 19, 2013, at 1:37 PM, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > > Hi Alexandre, > > > > > > On Mon, Nov 18, 2013 at 7:40 AM, Alexandre Bergel < > alexandre.bergel(a)me.com> wrote: > > Hi Eliot, > > > > Stef told me you are working hard on improving the Pharo VM. > > In particular making the VM support fragmented memory. If I understand > correctly, this will relieve the VM from having a continuous block of > memory to hold the whole image. Really cool! > > > > A better term is segmented memory. fragmentation in memory management > means free space getting divided into too small pieces to be useful. > > > > I have a general question regarding VM. Are you aware of other Virtual > machines supporting this? Does the Java VM support fragmented memory? .Net? > V8? > > > > Any VM that uses the train algorithm supports segmented memory. Any VM > that uses "pools" or "regions" uses segmented memory. VisualWorks' VM > supports segments. I'm pretty sure that V8 .Net & HotSpot provide > segmented memory. > > > > > > I would like to mention this in a research paper I am working on. > > > > Cheers, > > Alexandre > > -- > > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > > Alexandre Bergel http://www.bergel.eu > > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > > > > > > > > > > > > > -- > > best, > > Eliot > > >

Nov. 25, 2013

Re: [Pharo-dev] Correction for XML-Writer

by Tudor Girba

Indeed, it seems to work fine. Thanks Stephan and Diego. Doru On Mon, Nov 25, 2013 at 2:25 PM, Camillo Bruni <camillobruni(a)gmail.com>wrote: > Doru uploaded the new version, seems to work again: > > https://ci.inria.fr/pharo-contribution/job/XMLWriter/ > https://ci.inria.fr/pharo-contribution/job/XMLParser/ > > also I closed the original issue again > https://pharo.fogbugz.com/f/cases/12229/XML-Tools-improvement > > On 2013-11-25, at 13:40, Stephan Eggermont <stephan(a)stack.nl> wrote: > > > The configuration of XMLWriter refers to the wrong repository. > > That breaks the moose build. > > I donât seem to have commit rights to PharoExtras, so > > > > > > <ConfigurationOfXMLWriter-StephanEggermont.41.mcz> > > -- www.tudorgirba.com "Every thing has its own flow"

Nov. 25, 2013

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/30596 Home: https://github.com/pharo-project/pharo-core

Nov. 25, 2013

[pharo-project/pharo-core] aa6074: 30596

by GitHub

Branch: refs/heads/3.0 Home: https://github.com/pharo-project/pharo-core Commit: aa60745ad2bcc2724c1de09046f66e582bec08b1 https://github.com/pharo-project/pharo-core/commit/aa60745ad2bcc2724c1de090… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2013-11-25 (Mon, 25 Nov 2013) Changed paths: M NautilusRefactoring.package/ChangesBrowser.class/class/specs/defaultSpec.st M NautilusRefactoring.package/ChangesBrowser.class/definition.st A NautilusRefactoring.package/ChangesBrowser.class/instance/accessing/scopeChooser.st A NautilusRefactoring.package/ChangesBrowser.class/instance/private/updateChanges.st A NautilusRefactoring.package/EnvironmentChooser.class/README.md A NautilusRefactoring.package/EnvironmentChooser.class/class/specs/defaultSpec.st A NautilusRefactoring.package/EnvironmentChooser.class/definition.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/enableCustom.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/enableCustom_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/enableGlobal.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/enableGlobal_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/environment.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/environment_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/environments.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/environmentsView.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/environments_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/label.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/labelView.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/accessing/label_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/actions/chooseNewEnvironment.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/events/whenEnvironmentChanged_.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/initialization/initialize.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/initialization/initializeWidgets.st A NautilusRefactoring.package/EnvironmentChooser.class/instance/updating/update.st A NautilusRefactoring.package/PackageAndClassChooser.class/README.md A NautilusRefactoring.package/PackageAndClassChooser.class/class/specs/defaultSpec.st A NautilusRefactoring.package/PackageAndClassChooser.class/definition.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/childrenFor_.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/filteredRoots.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/iconFor_.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/labelFor_.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/roots.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing - tree/selectionChanged.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing/searchField.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing/searchPattern.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing/selectedItem.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing/selectedItems.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/accessing/tree.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/actions/choose.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/actions/chooseModalTo_.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/initialization/initializeWidgets.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/initialization/setFocus.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/testing/hasSearch.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/updating/searchUpdate_.st A NautilusRefactoring.package/PackageAndClassChooser.class/instance/updating/update.st M NautilusRefactoring.package/extension/RBCreateAccessorsForVariableRefactoring/instance/whatToDisplayIn_.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - scripts/script251.st A ScriptLoader30.package/ScriptLoader.class/instance/pharo - updates/update30596.st M ScriptLoader30.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st A Spec-MorphicAdapters.package/NameFilteringTreeNode.class/README.md A Spec-MorphicAdapters.package/NameFilteringTreeNode.class/definition.st A Spec-MorphicAdapters.package/NameFilteringTreeNode.class/instance/filtering/matchTokenFilter_.st Log Message: ----------- 30596 11682 Enable Scoped Refactoring Changes Step 1 https://pharo.fogbugz.com/f/cases/11682 http://files.pharo.org/image/30/30596.zip

Nov. 25, 2013

Re: [Pharo-dev] WhatsUp from: 2013-11-18 until: 2013-11-30

by Martin Dias

On Mon, Nov 18, 2013 at 7:00 AM, <seaside(a)rmod.lille.inria.fr> wrote: > Hi! We're sending this automatic email twice a month, to give the > community an opportunity to easily know what's happening and to coordinate > efforts. Just answer informally, and feel free to spawn discussions > thereafter! > > ### Here's what I've been up to since the last WhatsUp: > > - Reported and fixed several small issues (Marcus' "triviality a day") -- a couple of issues in Ring -- clean-up on .changes and .sources files and logging Cheers, MartÃn

Nov. 25, 2013

Re: [Pharo-dev] PharoLauncher

by Damien Cassou

On Thu, Nov 21, 2013 at 4:00 PM, Sean P. DeNigris <sean(a)clipperadams.com> wrote: > One thing is that I'd like to be able to choose the default image location. > It feels wrong to install them under ~/Library/Preferences on the Mac... done. Please open the launcher, then its preference settings, and change the directory. -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." Winston Churchill

Nov. 25, 2013