- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [Vm-dev] Re: Proxy DNU vs CannotInterpret. (Ghost related)

by ClÃ©ment Bera

2016-02-15 19:34 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > > Hi Stef, > > On Mon, Feb 15, 2016 at 8:18 AM, stepharo <stepharo(a)free.fr> wrote: > >> Hi eliot >> >> I think that the difference is really what guillermo explained. >> With DNU you can only trapped methods that are not defined while if you >> want to implement true proxy you do not want holes. >> I think that also with DNU proxy there were some limits realyed to classe >> but I do not remember. >> > > As far as the VM goes, there is /one/ minor issue with special selector > #== and #class (*). But the real issue is that ProtoObject implements far > too much protocol. If one creates a class that inherits from nil all it > needs to do is implement doersNotUnderstand: and it will catch every > message (*). For this to work the debugger and basic inspectors must use > the mirror primitives otherwise there will be infinite recursion of MNUs. > > Now the issue with special selector #== and #class is that these bytecodes > are specified as no lookup. So if one uses bytecodes 182 (#==) or 199 > (#class) these operate without sending messages and will compare the object > or answer the object's class without sending #doesNotUnderstand:. > > In the VW VM I added a flag so one could turn this behaviour off, and > that's certainly easy to do in our VM. Another approach is to have the > bytecode compiler not send these. > > But given our VM it is certainly possible to use doesNotUnderstand: > proxies, using a much slimmer class than ProtoObject. In fact a good > project would be to try and shrink ProtoObject until it provides the > minimum, which would be a #doesNotUnderstand: method that raises an error > that reminds the programmer that they need to implement their own > doesNotUnderstand: handler in subclasses of ProtoObject. > > Hello, The #class primitive bytecode is not used in Pharo to avoid this kind of problem. It is indeed possible to add a setting in Opal compiler not to compile #== using the inlined selector to avoid issues there too. However, even in proxies, it may make sense to have this inlined selector. > >> Stef >> >> Le 15/2/16 16:06, Eliot Miranda a Ã©crit : >> >> Hi Denis, >> >> On Feb 15, 2016, at 6:53 AM, Denis Kudriashov < <dionisiydk(a)gmail.com> >> dionisiydk(a)gmail.com> wrote: >> >> >> 2016-02-15 15:32 GMT+01:00 Mariano Martinez Peck < >> <marianopeck(a)gmail.com>marianopeck(a)gmail.com>: >> >>> As far as I can remember (this was about 4 years ago) The use of >>> #cannotInterpret: was the only way to be able to intercept everything >>> without breaking the system. Subclassing from ProtoObject make your proxy >>> to understand some messages. Subclassing from nil was technically possible, >>> but I found out the system would simply crash as it didn't know at all how >>> to handle other subclasses of nil. Even with the debugging support I added >>> as you can read in the paper. Maybe things have changed. >>> >>> If subclassing from nil does not break the system and you can still >>> inspect, debug proxies, then sure, you can give a try to Ghost using #dnu >>> rather than #cannotInterpret:. In fact, I would like to have a dnu-based >>> approeach that is as reliable as it was #cannotInterpret: in the sense of >>> how much I can intercept. With #cannotInterpret I could trap EVERYTHING >>> (everything but #==) and then decide what to do. Subclassing from nil would >>> get you there too if you make it work. >>> >> >> Actually I already implemented it many yeas ago for Mocketry. But >> subclassing from nil works correctly only in VW. I got problems when >> porting it to Squeak (at that time). >> Now I want to give DNU another chance. It will simplify logic very much. >> It will remove strange hierarchy of proxies which is needed for trick but >> raises many questions. >> >> >> There is no reason in principle why Squeak/Pharo should not be able to >> use MNU proxies in the same way as VW. The key is to use the mirror >> primitives in basic inspectors that are used to inspect the proxies and >> in the debugger to simulate code. Without the mirror primitives many >> sends to a proxy are likely to create an infinite recursion. As yet we >> don't have a robust low space handler that catches infinite recursion so >> debugging can be frustrating. But we can work on this. IMO the MNU >> approach is to be preferred. >> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot > >

Feb. 15, 2016

Re: [Pharo-dev] Compiling the unix vm / spur

by Eliot Miranda

Hi Marcus, On Mon, Feb 15, 2016 at 10:37 AM, Markus Fritsche <mfritsche(a)reauktion.de> wrote: > Hi, > > I was just going to update the ubuntu ppa an wanted to supply a spur vm > image. The compilation stopped however: > > -- Configuring done > CMake Error at IA32FFIPlugin/CMakeLists.txt:11 (add_library): > Cannot find source file: > > > /home/mfritsche/src/pharo-vm-spur-2016.02.13/src/plugins/IA32FFIPlugin/IA32FFIPlugin.c > That's a bug in the CMake code (which I hope will be history in a few months). tt should be looking *only* for src/plugins/SqueakFFIPlugin/SqueakFFIPlugin.c which should include src/plugins/SqueakFFIPlugin/IA32FFIPlugin.c > Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp > .hxx .in .txx > > > Any hints? > > Kind regards, > Markus > > -- _,,,^..^,,,_ best, Eliot

Feb. 15, 2016

New Cog VMs available as per VMMaker.oscog-eem.1679/r3602

by Eliot Miranda

Hi All, first of all, apologies for the long wait for new VMs. I know there are bugs that these VMs address and that it's been 5 months since VMs were last made is something I regret. The wait was due to my developing the 64-bit Spur Cog JIT VM which required surgery not just to the Cogit but also to Slang. I beg your forgiveness. Secondly, the Mac VMs are much changed, using John McIntosh's Cocoa support libraries. The old Carbon UI VMs are history. That means that Cog supports building on contemporary Mac OS releases. These VMs were built on 10.9; I like the 3D look much more than the 10.10 one ;-). The VMs are also signed, so installing them should be easier. Thirdly, these VMs include 64-bit JIT support. There is an updated squeak trunk image at http://www.mirandabanda.org/files/Cog/SpurImages for you to try. As yet the 64-bit Vms do not include an FFI plugin. I hope to rectify this soon. Finally I have included Pharo Mac VMs, not to displace the official Pharo VMs, but to test my build process. I hope that in very few months the svn repository will be moed to githup and that Esteban and I will build official VMs from the same repository. The VMs are here: http://www.mirandabanda.org/files/Cog/VM/VM.r3602 CogVM binaries as per VMMaker.oscog-eem.1679/r3602 Include 64-bit Spur VMs for Mac OS X x64 and Linux x64. N.B. As yet these VMs lack FFI support. Move all Mac OS X VMs to the Cocoa libraries. Minimum supported version should be 10.6.x. The Mac OS X VMs are signed and so should install properly without having to disable the launch security checks. Bug Fixes and Improvements: General: Add vmParameter 20 to answer the utc microseconds at startup. CoInterpreter: Remove the atCache from the CoInterpreter. It's of little benefit given the Cogit. ceSend: et al need to add looked up methods to the method cache, otherwise methods only found through machine code sends will not be jitted. Fix slip in maybeSelectorOfMethod: which will improve back traces including methods containing pragmas (including primitives). Modify secret primtiive 161, primitiveSetIdentityHash so that with 0 args it answers whether an object has an identity hash. This is vacuously true for V3, but is meaningful in Spur. Cogit: Fix /horrible/ bug with primitive error codes and fixups. The adjustment of the initialPC to skip the primitive and error code, if any, was done after generating fixups for backward branches, and hence these branches were bogus. Object>>shallowCopy in Spur is subject to this, but amazingly enough it has not surfaced before now. Fix this by adjusting the initialPC before scanMethod scans for fixups. Fix bug in Spur machine-code at:put: on 32-bit bits objects; method failed to fail for negative values. Fill unused portions of methods with the stop instruction, not nop. Explicitly initialize the code zone with stops. Explicitly clear the reclaimed portion of the method zone after a compaction. Include the reclaimed portion of the method zone in the I-cache flush after a compaction: no point in leaving it in cache since it isn't a valid call/jump target. Spur: Maintain the survivor count on scavenge. Fix start up of images containing >= 16 segments. The old code assumed numSegments < 16 and failed to grow the segment records, resulting in objects in segments greater than 15 to not be swizzled, and a resulting crash. Spur Cogit: Support CompiledMethod indexing in genPrimitiveAt[Put]. Hence fix the regression in at:[put:] whereby in Spur one could access the literals part of a method as bytes. Add a machine-code primitive for objectAt: to provide fast literal and header access on Spur. Add 16-bit indexability to Spur. Generate slimmer code for positive32BitValueOf: on 64-bit Spur. Fix a bug with out-of-place two-way become, which got the hash copy the wrong way round. By default, the hash stays with the reference in two-way become. Fix what appears to be a compiler bug with in-place two-way-become by treating the headers to be swapped so that remembered flags are swapped and hash bits are swapped if required. This fixes a bug with in-place two-way become of objects that are the same size on the heap. One object's contents would not be updated correctly. Implement ephemeron queue primitive. Add an image flag that controls the finalization support (StackInterpreter instVar newFinalization). If unset, finalization is as currently expected; WeakArray finalizes all WeakArrays in its registries on each finalization (which does not scale!). If set, then both fired ephemerons and bereaved weak arrays are added to the finalization queue and the finalization semaphore is signalled. In this regime the image is expected to send mourn to each element of the queue, and weak arrays will send themselves finalize in mourn, side-stepping the finalizeValues steps and rendering WeakArray's FinalizationDependents and WeakRegistry obsolete. The bit is set via Smalltalk vmParameterAt: 48 put: ((Smalltalk vmParameterAt: 48) bitOr: 64). Slang: Fix bad regression in type inference from VMMaker.oscog-eem.1587. returnTypeForSend:in: can only default return types to sqInt for unknown selectors. The rewrite in VMMaker.oscog-eem.1587 inadvertently defaulted the return type of known methods whose return type was yet to be determined to sqInt. The fix is to only default to sqInt if there is no known method for the given selector. This fixes, for instance, the weird flipping of the type of 32-bit Spur's headerWhileForwardingOf:,which should always be #sqLong. Fix bad bug in type inferrence. Avoid computing variable and return types prematurely i.e. don't derive types from as-yet-untyped methods. Fix baaaad bug in node:typeCompatibleWith:inliningInto:in: which was looking up the type of the formal parameter in inlining in the target method, not the method being inlined! Newspeak: Fix bug in MNU for cogged dynamic super sends - should start looking for #doesNotUnderstand: in the superclass not the receiver class. Unify MNU lookup between absent and present receiver sends as a result. Further fix outer send MNU by not confusing lkupClass and lkupClassTag. Push setting lkupClassTag down to clarify it is for new cache entries. Plugins: Fix the FilePlugin's unnecessary fullGC on Spur. Include the file sync prim in the FilePrims plugin. ThreadedFFIPlugin: - ExternalAddress now are taken correctly (nor as ByteArray or Alien, because they are different beasts). - allow pushing of pointers to any type (into a ByteArray, an ExternalAddress or an Alien), to allow passing parameters style int*, float*, etc. - allow reading in memory to allow read chunks of ByteArray Added ThreadedFFIPlugin>>#primitiveLoadSymbolFromModule (copy and adapt from IA32ABIPlugin). Fix coercion problems on FFI plugin (IA32 & ARM) - character were not correctly mapped back - floats were not being well checked - external addressed needs to be treated as aliens for the purpose of coercing them as pointers (bad regression by Eliot) Alien Plugins: Fix missing primitive failure for Alien typeAt: 0 [put:]. Add word accessors to Alien. _,,,^..^,,,_ best, Eliot

Feb. 15, 2016

Re: [Pharo-dev] Compiling the unix vm / spur

by Damien Cassou

On February 15, 2016 7:37:34 PM GMT+01:00, Markus Fritsche <mfritsche(a)reauktion.de> wrote: >Hi, > >I was just going to update the ubuntu ppa an wanted to supply a spur vm > >image. The compilation stopped however: I reported the same this morning : https://pharo.fogbugz.com/f/cases/17591/Can-t-compile-sources-2016-02-13-CM… -- Damien Cassou http://damiencassou.seasidehosting.st "Success is the ability to go from one failure to another without losing enthusiasm." --Winston Churchill

Feb. 15, 2016

Compiling the unix vm / spur

by Markus Fritsche

Hi, I was just going to update the ubuntu ppa an wanted to supply a spur vm image. The compilation stopped however: -- Configuring done CMake Error at IA32FFIPlugin/CMakeLists.txt:11 (add_library): Cannot find source file: /home/mfritsche/src/pharo-vm-spur-2016.02.13/src/plugins/IA32FFIPlugin/IA32FFIPlugin.c Tried extensions .c .C .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx Any hints? Kind regards, Markus

Feb. 15, 2016

Re: [Pharo-dev] Proxy DNU vs CannotInterpret. (Ghost related)

by Eliot Miranda

Hi Stef, On Mon, Feb 15, 2016 at 8:18 AM, stepharo <stepharo(a)free.fr> wrote: > Hi eliot > > I think that the difference is really what guillermo explained. > With DNU you can only trapped methods that are not defined while if you > want to implement true proxy you do not want holes. > I think that also with DNU proxy there were some limits realyed to classe > but I do not remember. > As far as the VM goes, there is /one/ minor issue with special selector #== and #class (*). But the real issue is that ProtoObject implements far too much protocol. If one creates a class that inherits from nil all it needs to do is implement doersNotUnderstand: and it will catch every message (*). For this to work the debugger and basic inspectors must use the mirror primitives otherwise there will be infinite recursion of MNUs. Now the issue with special selector #== and #class is that these bytecodes are specified as no lookup. So if one uses bytecodes 182 (#==) or 199 (#class) these operate without sending messages and will compare the object or answer the object's class without sending #doesNotUnderstand:. In the VW VM I added a flag so one could turn this behaviour off, and that's certainly easy to do in our VM. Another approach is to have the bytecode compiler not send these. But given our VM it is certainly possible to use doesNotUnderstand: proxies, using a much slimmer class than ProtoObject. In fact a good project would be to try and shrink ProtoObject until it provides the minimum, which would be a #doesNotUnderstand: method that raises an error that reminds the programmer that they need to implement their own doesNotUnderstand: handler in subclasses of ProtoObject. > > Stef > > Le 15/2/16 16:06, Eliot Miranda a Ã©crit : > > Hi Denis, > > On Feb 15, 2016, at 6:53 AM, Denis Kudriashov < <dionisiydk(a)gmail.com> > dionisiydk(a)gmail.com> wrote: > > > 2016-02-15 15:32 GMT+01:00 Mariano Martinez Peck < <marianopeck(a)gmail.com> > marianopeck(a)gmail.com>: > >> As far as I can remember (this was about 4 years ago) The use of >> #cannotInterpret: was the only way to be able to intercept everything >> without breaking the system. Subclassing from ProtoObject make your proxy >> to understand some messages. Subclassing from nil was technically possible, >> but I found out the system would simply crash as it didn't know at all how >> to handle other subclasses of nil. Even with the debugging support I added >> as you can read in the paper. Maybe things have changed. >> >> If subclassing from nil does not break the system and you can still >> inspect, debug proxies, then sure, you can give a try to Ghost using #dnu >> rather than #cannotInterpret:. In fact, I would like to have a dnu-based >> approeach that is as reliable as it was #cannotInterpret: in the sense of >> how much I can intercept. With #cannotInterpret I could trap EVERYTHING >> (everything but #==) and then decide what to do. Subclassing from nil would >> get you there too if you make it work. >> > > Actually I already implemented it many yeas ago for Mocketry. But > subclassing from nil works correctly only in VW. I got problems when > porting it to Squeak (at that time). > Now I want to give DNU another chance. It will simplify logic very much. > It will remove strange hierarchy of proxies which is needed for trick but > raises many questions. > > > There is no reason in principle why Squeak/Pharo should not be able to use > MNU proxies in the same way as VW. The key is to use the mirror > primitives in basic inspectors that are used to inspect the proxies and > in the debugger to simulate code. Without the mirror primitives many > sends to a proxy are likely to create an infinite recursion. As yet we > don't have a robust low space handler that catches infinite recursion so > debugging can be frustrating. But we can work on this. IMO the MNU > approach is to be preferred. > > > -- _,,,^..^,,,_ best, Eliot

Feb. 15, 2016

Re: [Pharo-dev] Improving Ghost proxies

by Eliot Miranda

On Mon, Feb 15, 2016 at 8:21 AM, stepharo <stepharo(a)free.fr> wrote: > > > Le 15/2/16 14:10, Mariano Martinez Peck a Ã©crit : > > > > On Mon, Feb 15, 2016 at 7:15 AM, Denis Kudriashov < <dionisiydk(a)gmail.com> > dionisiydk(a)gmail.com> wrote: > >> Hello. >> >> I think it is important to get single proxy framework for Pharo. It >> should provide required integration support for our tools. >> >> I want push Ghost in that direction. Debuggers and inspectors should work >> correctly with proxies which based on Ghost. >> >> > Hi Denis, > > I welcome you push on Ghost. The original source of Ghost was in SS3 and > then, the CAR team move it to Shub and hence why it is there. I THINK they > did not do many changes from the "upstream" project, so it should be safe > to take that one. I guess you should ask Luc Fabrese about that. > > Debugging, inspecting, etc was some of the problems that Ghost did solve. > I was able to do all that. And you were able to enable/disabled debugging > capabilities dynamically. > > What I really really really recommend you is that before doing anything > with Ghost, please read the whole journal paper as there is a lot of > discussion, possibilities, explanations of why we took path X, etc etc: > > https://hal.inria.fr/hal-01081236/file/Mart14z-Ghost-Final.pdf > > > >> Now Ghost is stored in CAR team repository. Should we move it to Pharo? >> >> > I know that is is subjective, but from my latest experience with git, if I > were to make the move of a repository (which requires some effort) I would > do it to github. > > > No core package of Pharo should be managed with git. > We will move all together and not pieces by pieces. > This is great to hear! > > Finally, let me say something.... what I got from my Ghost proxies > experience is that Ghost could serve as a proxy library for "common" usage. > The code of Ghost is very very little. What is most important is the idea, > not the code. This is why, many of the people that got interested in using > Ghost for a project X, they ended up "forking" the original Ghost code and > adapt it. So...while Ghost can be used and extended for different purposes, > I think it's not bad either that someone forks it and adjusts it for his > needs. > > > P.S. Ghost has one failed tests due to Spur problem 17536 >> <https://pharo.fogbugz.com/f/cases/17536/VM-Crash-when-adding-an-iVar-to-a-s…> >> . >> >> Best regards, >> Denis >> > > > > -- > Mariano > http://marianopeck.wordpress.com > > > -- _,,,^..^,,,_ best, Eliot

Feb. 15, 2016

Re: [Pharo-dev] Improving Ghost proxies

by Eliot Miranda

On Mon, Feb 15, 2016 at 2:15 AM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > Hello. > > I think it is important to get single proxy framework for Pharo. It should > provide required integration support for our tools. > > I want push Ghost in that direction. Debuggers and inspectors should work > correctly with proxies which based on Ghost. > > Now Ghost is stored in CAR team repository. Should we move it to Pharo? > > P.S. Ghost has one failed tests due to Spur problem 17536 > <https://pharo.fogbugz.com/f/cases/17536/VM-Crash-when-adding-an-iVar-to-a-s…> > . > Thanks Denis. I can reproduce this and am taking a look at this today. Best regards, > Denis > _,,,^..^,,,_ best, Eliot

Feb. 15, 2016

Re: [Pharo-dev] Problems? with MC browser?Process?

by Volkert

Cool. A new research field for testing: vdt - video driven testing .... sorry, could not resist ;-) i look forward to see the results .... registered for the course today. Volkert On 15.02.2016 18:23, Aliaksei Syrel wrote: > > Now I cannot work with a system in that state. > To produce a videos of 8 min I lost more than 3 hours. Not fun. > > > I see you take a lot of videos and you catch lots of bugs during > filming. Which means that video can be used as helpers to reproduce > bugs you catch. Why would not you publish those videos on mail list? > > On Mon, Feb 15, 2016 at 5:19 PM, stepharo <stepharo(a)free.fr > <mailto:stepharo@free.fr>> wrote: > > >> >> 2016-02-15 17:09 GMT+01:00 stepharo <stepharo(a)free.fr >> <mailto:stepharo@free.fr>>: >> >> Hi >> >> Today I got several times problems with the refresh of the >> screen (see attachement) >> and I could >> not produce the videos I wanted. >> I wonder if there is not a problem with the processes around >> MC browser. >> I do not see what else it would be. >> >> Now I cannot work so I'm stuck with the Mooc videos >> production so I will >> try to use older images but this is slowing me down. >> >> Stef >> >> >> Most probable this happens when multiple Morphic UI-processes are >> active. >> (caused by the debugger ?) > > Exactly the case. > But the debugger was closed because I was about to save the code > once implemented in the debugger. > Now I cannot work with a system in that state. > To produce a videos of 8 min I lost more than 3 hours. Not fun. > > Stef > >> > >

Feb. 15, 2016

Re: [Pharo-dev] Problems? with MC browser?Process?

by Aliaksei Syrel

> > Now I cannot work with a system in that state. > To produce a videos of 8 min I lost more than 3 hours. Not fun. I see you take a lot of videos and you catch lots of bugs during filming. Which means that video can be used as helpers to reproduce bugs you catch. Why would not you publish those videos on mail list? On Mon, Feb 15, 2016 at 5:19 PM, stepharo <stepharo(a)free.fr> wrote: > > > 2016-02-15 17:09 GMT+01:00 stepharo <stepharo(a)free.fr>: > >> Hi >> >> Today I got several times problems with the refresh of the screen (see >> attachement) >> and I could >> not produce the videos I wanted. >> I wonder if there is not a problem with the processes around MC browser. >> I do not see what else it would be. >> >> Now I cannot work so I'm stuck with the Mooc videos production so I will >> try to use older images but this is slowing me down. >> >> Stef >> > > Most probable this happens when multiple Morphic UI-processes are active. > (caused by the debugger ?) > > > Exactly the case. > But the debugger was closed because I was about to save the code once > implemented in the debugger. > Now I cannot work with a system in that state. > To produce a videos of 8 min I lost more than 3 hours. Not fun. > > Stef > > > > >

Feb. 15, 2016