February 2016 - Pharo-users

Re: [Pharo-users] Compiling Pharo to C++

by kilon.alios

"Are you sure C++ compilation is the best way ? There are a lot of flaws: Smalltalk needs to be restrictive or will be compiled to very slow code, you will loose debugging features, etc. I would try to integrate Pharo the same way lua is integrated in engines like Source2 from valve: you run your Unreal engine and have it call Pharo as a scripting language to code some features. In such case, one needs to make the VM embeddable in a C++ app, but on the vm-dev mailing list people can help you to do so. Then you need to define some APIs based on what exist in the interpreter proxy to easily call Pharo methods and share struct / objects from C++. Lastly, you may consider adding a few settings in FFI. By default the Smalltalk stack and the C stack are separated to to Smalltalk specific stack page handling. I guess with some annotation in Pharo API code called from C++ you could define the case where the stack could be shared or not. While compiling to C++, how to you plan your compiler to manage memory ? " I am not able to use Pharo as scripting language for Unreal I am afraid the easy way. I cannot use FFI for reason I describe later on. How I plan my compiler to manage memory , it wont, the compiler will expect very close to C++ syntax and features, it will do minimal work at its side so it will be up to the coder. "I think you totally underestimate the task. Yes, Pharo (Smalltalk) is a pretty simple language (by design), so given a parsed AST (thanks Opal) it will not be so hard to transpile into some other language. But Pharo (Smalltalk) is much, much more: you need a sophisticated runtime (basically the VM) and a large part of the core libraries. Many people have worked very hard for decades to give us what we have today, this represents a huge amount of developer years. " I think you overestimate what I am trying to do, I am not trying to recreate Pharo. There wont be a reimplementation of the VM , or the pharo libraries. This is a simple tool that take pharo-like and turns it to C++ making zero assumptions. That means it will be up to the coder to make sure that what it makes is proper C++. It wont be a magical tool that takes pharo code and turns it to C++ no matter what. There will be a ton of limitations on how you write the code to make sure it produces readable and proper C++ code. The advantage of using it over pure Pharo code ? NONE !!! and NOT RECOMMENDED !!! The advantage of using it over pure C++ code ? a) prototype in pure Pharo code using all the advantages and tools of Pharo IDE b) turn it to pharo like C++ code without having to rewrite your code from scratch in C++ "even for C, there is no compiler that takes Pharo code and compiles it to C - binary code. what slang and vmmaker is doing, is more like a preprocessor, generating C - source code. Compiling C to code is done by an ordinary c-compiler, not smalltalk." we call it "compiler" when we have a tool that take one input and turns it to another output. I am well aware that the compiling is done by ordinary C compiler to machine code. I did not mean that I want to compile pharo code to machine code, but pharo code to C++ code and to be precise Unreal friendly C++ code. "AST for pharo code is not used in any step generating vm code or FFI glue code AFAIK" The way I understand it is that AST is used to take pharo code and brake it down to individual part that is the feed into bytecode compiler and the vm then turns bytecode to machine JIT or interprets it to machine depending on the optimisation. Again I am not concerned with the binary part this is taking one source code and turning to another source code, so it has nothing to do with VM, FFI and machine code. "If Unity does not provide a c-binding, it is really difficult to interact with it through a FF-interface. C++ code uses (theoretically every c++ runtime its own) different naming scheme for functions. You can not load and call c++ code the same way as for C. " My engine is Unreal not Unity. Unreal assumes that you either write C++ code or using their own scripting language "Blueprint". Its not designed to be used as a library so making wrappers for Pharo via FFI is not possible. The main reason is that iOS allows only for static executables. -- View this message in context: http://forum.world.st/Compiling-Pharo-to-C-tp4877775p4877832.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Feb. 16, 2016

Re: [Pharo-users] Metacello reverse project dependency

by Werner Kassens

Hi Peter, in your second example, which has a new BaselineOfZar & of bar, BaselineOfZar-PeterUhnak.9 & BaselineOfBar-PeterUhnak.11 is fetched, which have the same versionnumber as in the 1st example, are you sure, that the new baselines of ex2 are loaded? werner On 02/16/2016 09:33 AM, Peter UhnÃ¡k wrote: > Unfortunately #includes: doesn't seem to matter and I still get errors. > > I tried to somehow summarize my findings with minimal code & results > here http://peteruhnak.com/metacello/ >

Feb. 16, 2016

Re: [Pharo-users] Compiling Pharo to C++

by Nicolai Hess

2016-02-16 9:52 GMT+01:00 kilon.alios <kilon.alios(a)gmail.com>: > So doing further research into Unreal , I have reached the conclusion that > the best way to integrate Pharo is make a compiler that takes Pharo code > and > compiles it to C++. > even for C, there is no compiler that takes Pharo code and compiles it to C - binary code. what slang and vmmaker is doing, is more like a preprocessor, generating C - source code. Compiling C to code is done by an ordinary c-compiler, not smalltalk. > > I have barely scratch such area when I was playing around with Pharo AST > nodes. Any advice / ideas how to accomplish this ? > AST for pharo code is not used in any step generating vm code or FFI glue code AFAIK > > Currently my goal is to start super simple, with compiling small pharo code > fragments to C++ code and then much later on even introduce extensions to > the pharo language to accommodate for C++ features like templates , > pointers > (FFI probably already cover this) , static types etc. Obviously if I can > keep the pharo syntax intact would be the ideal option. > The attractiveness of NativeBoost was, that it can generate (written in smalltalk) glue code for interacting with C-libraries. Call functions, provide (C-Type)-arguments and process return values. "Mirror" C-Structures in the pharo image side and generate accessors for objects living in the heap memory. And the new FFI can do this (differently) too. But this is again only for C-libraries. If Unity does not provide a c-binding, it is really difficult to interact with it through a FF-interface. C++ code uses (theoretically every c++ runtime its own) different naming scheme for functions. You can not load and call c++ code the same way as for C. > > > > -- > View this message in context: > http://forum.world.st/Compiling-Pharo-to-C-tp4877775.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. > >

Feb. 16, 2016

Re: [Pharo-users] Compiling Pharo to C++

by kilon.alios

Ok first my apology to people because I see that many have misunderstood my motivation. So let me be crystal clear. *This IS NOT an effort to make Pharo "super cool". "Look mom Pharo compiles to C++ magically, it runs blazzing fast, it can use C++ libraries with zero efforts" and similar none sense.* Unfortunately I cannot use Pharo as a scripting language that would be awesome but I afraid not possible. Compiling Unreal as a set of dlls is somewhat possible but the problem is that iOS does not allow dlls, iOS support is super crucial for me so that goes out of the window. Embeding Pharo in Unreal is no go either since pharo does not support this. I am not looking into bringing Pharo features to C++. For example I dont care about dynamic types because Unreal depends a lot on templates which are very dynamic, I dont care about Pharo GC because unreal has its own GC, I dont care about Pharo reflection because Unreal also gives C++ reflection features, I dont even care about Pharo live coding because Unreal also allow to hot reload on the fly when you recompile C++ code. WARNING !!! I am not saying that coding C++ in Unreal is better , more dynamic or more live than Pharo. Pharo is awesome. So my thinking is like this, I cannot avoid C++. So since I cannot avoid it then maybe I could try to make it more enjoyable by making it move closer to pharo syntax. That means my implementation will do a lot of nasty compromises, meaning manual memory managment, disallow dynamic types (though later on with the help of templates maybe I can make it more dynamic) , have to include headers etc. So in general write C++ code using something that is close to pharo syntax. My goal is not an ideal solution but a practical solution something that is relative simple to make for a single developer. It wont be a tool you guys will be using because it will be too tied to Unreal anyway and it will contain the nasty parts of C++. The idea will be similar to what Cython is for python, start prototyping in pure python with all the great features it has, then when code works as you expect start inserting static types, manual memory management etc to turn it to C++. I hope now I am much more clear what I aiming for -- View this message in context: http://forum.world.st/Compiling-Pharo-to-C-tp4877775p4877821.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Feb. 16, 2016

Re: [Pharo-users] Compiling Pharo to C++

by Sven Van Caekenberghe

I think you totally underestimate the task. Yes, Pharo (Smalltalk) is a pretty simple language (by design), so given a parsed AST (thanks Opal) it will not be so hard to transpile into some other language. But Pharo (Smalltalk) is much, much more: you need a sophisticated runtime (basically the VM) and a large part of the core libraries. Many people have worked very hard for decades to give us what we have today, this represents a huge amount of developer years. > On 16 Feb 2016, at 09:52, kilon.alios <kilon.alios(a)gmail.com> wrote: > > So doing further research into Unreal , I have reached the conclusion that > the best way to integrate Pharo is make a compiler that takes Pharo code and > compiles it to C++. > > I have barely scratch such area when I was playing around with Pharo AST > nodes. Any advice / ideas how to accomplish this ? > > Currently my goal is to start super simple, with compiling small pharo code > fragments to C++ code and then much later on even introduce extensions to > the pharo language to accommodate for C++ features like templates , pointers > (FFI probably already cover this) , static types etc. Obviously if I can > keep the pharo syntax intact would be the ideal option. > > > > -- > View this message in context: http://forum.world.st/Compiling-Pharo-to-C-tp4877775.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. >

Feb. 16, 2016

Re: [Pharo-users] Compiling Pharo to C++

by ClÃ©ment Bera

Are you sure C++ compilation is the best way ? There are a lot of flaws: Smalltalk needs to be restrictive or will be compiled to very slow code, you will loose debugging features, etc. I would try to integrate Pharo the same way lua is integrated in engines like Source2 from valve: you run your Unreal engine and have it call Pharo as a scripting language to code some features. In such case, one needs to make the VM embeddable in a C++ app, but on the vm-dev mailing list people can help you to do so. Then you need to define some APIs based on what exist in the interpreter proxy to easily call Pharo methods and share struct / objects from C++. Lastly, you may consider adding a few settings in FFI. By default the Smalltalk stack and the C stack are separated to to Smalltalk specific stack page handling. I guess with some annotation in Pharo API code called from C++ you could define the case where the stack could be shared or not. While compiling to C++, how to you plan your compiler to manage memory ? 2016-02-16 9:52 GMT+01:00 kilon.alios <kilon.alios(a)gmail.com>: > So doing further research into Unreal , I have reached the conclusion that > the best way to integrate Pharo is make a compiler that takes Pharo code > and > compiles it to C++. > > I have barely scratch such area when I was playing around with Pharo AST > nodes. Any advice / ideas how to accomplish this ? > > Currently my goal is to start super simple, with compiling small pharo code > fragments to C++ code and then much later on even introduce extensions to > the pharo language to accommodate for C++ features like templates , > pointers > (FFI probably already cover this) , static types etc. Obviously if I can > keep the pharo syntax intact would be the ideal option. > > > > -- > View this message in context: > http://forum.world.st/Compiling-Pharo-to-C-tp4877775.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. > >

Feb. 16, 2016

Compiling Pharo to C++

by kilon.alios

So doing further research into Unreal , I have reached the conclusion that the best way to integrate Pharo is make a compiler that takes Pharo code and compiles it to C++. I have barely scratch such area when I was playing around with Pharo AST nodes. Any advice / ideas how to accomplish this ? Currently my goal is to start super simple, with compiling small pharo code fragments to C++ code and then much later on even introduce extensions to the pharo language to accommodate for C++ features like templates , pointers (FFI probably already cover this) , static types etc. Obviously if I can keep the pharo syntax intact would be the ideal option. -- View this message in context: http://forum.world.st/Compiling-Pharo-to-C-tp4877775.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Feb. 16, 2016

Re: [Pharo-users] Bloc Space within a Space

by Alain Plantec

> > But how to drive it? e.g. simulate events ok, so we agree here. Bloc is done in that perspective. For now, you have to implement your own BlocSpace with an adequate #processEvents method, your can also implement your own main loop manager (polymorphic with BlMainLoopManager). > > > Pharo Smalltalk Users mailing list wrote >> but your code does not make sense regarding how Bloc is designed >> you can have one and only one **active** BlocSpace at a time. > > I understand from a Pharo perspective that one is only expected to have one > *real* space at a time which is receiving hardware events and such, but what > prevents us from simulating Spaces?! If there is a SpaceElement, and > Elements can be placed in the World, the principle of uniformity encourages > that a SpaceElement can be placed there like any other. yes, your right. I had that idea in mind. The only problem we have today is that the current implementation of damages repairing makes the assumption that a SpaceElement is at 0@0. (need to review BlSimpleRedrawer for that). > > > Pharo Smalltalk Users mailing list wrote >> imagine that a BlocSpace is a Pharo image. >> Making your code working would mean that we would be >> able to run a Pharo image within a Pharo image. > > Exactly, like the VM simulator! Sign me up! This was the level of uniformity > and explorability that I always wished in Morphic. :) ok, so not infeasible but we are not there today. Do not forget that Bloc is currently implemented by a very small team. lets do it step by step. Cheers Alain > > > > ----- > Cheers, > Sean > -- > View this message in context: http://forum.world.st/Bloc-Space-within-a-Space-tp4877474p4877733.html > Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com. >

Feb. 16, 2016

Re: [Pharo-users] Metacello reverse project dependency

by Peter UhnÃ¡k

Unfortunately #includes: doesn't seem to matter and I still get errors. I tried to somehow summarize my findings with minimal code & results here http://peteruhnak.com/metacello/ On Mon, Feb 15, 2016 at 11:31 PM, Werner Kassens <wkassens(a)libello.com> wrote: > Hi Peter, > have you looked at metacellos #includes: method? > from https://code.google.com/archive/p/metacello/wikis/APIReference.wiki : > "When 'Example-AddOn' is loaded, load 'Example-UI'" > includes: #('Example-UI' ); > werner > > On 02/13/2016 11:26 PM, Peter UhnÃ¡k wrote: > >> Hi, >> >> I have the following situation: >> >> I have ProjectMain, and ProjectPlugin, now normally the Plugin depends >> on Main, however I would like to load Plugin as part of Main. >> >> The problem is that I can't specify to load Plugin after Main has been >> already fully loaded, if I do something like this >> >> BaselineOfMain>>baseline: spec >> <baseline> >> spec >> for: #common >> do: >> [ spec >> baseline: 'Plugin' >> with: [ spec >> repository: '...'; >> loads: #('default' OR 'plugin') ]. >> spec >> package: 'Main' with: [ spec requires: #(more stuff) ]; >> spec group: 'default' with: #('Main'). >> specgroup: 'complete' with: #('default' 'Plugin') ] >> >> The BaselineOfPlugin has two groupsâ¦ "default", which loads also "Main", >> and "plugin", which doesn't. >> >> Now the problem is that there is no guaranteed way that Plugin will load >> after Main (in fact the should happen). If I load the plugin's #default, >> then Metacello ends up in an endless loop for some reason, and if I load >> #plugin, then it will load it before the Main which will break it >> (because Plugin has dependencies on main). >> >> So my question is: >> is it possible to reverse it somehow? >> Maybe add at the end `specpostLoadDoIt: #postLoadPlugins`, and then in >> that method manually load the plugins with Metacello/Gofer? Or is there >> a better way? >> >> Thanks, >> Peter >> > >

Feb. 16, 2016

Fwd: [squeak-dev] New Cog VMs available as per VMMaker.oscog-eem.1679/r3602

by John Pfersich

FYI. I noticed that Eliot didn't send to Pharo-users. Sent from my iPad Begin forwarded message: > From: Eliot Miranda <eliot.miranda(a)gmail.com> > Date: February 15, 2016 at 11:39:29 PST > To: Squeak Virtual Machine Development Discussion <vm-dev(a)lists.squeakfoundation.org> > Cc: "newspeaklanguage(a)googlegroups.com" <newspeaklanguage(a)googlegroups.com>, Discusses Development of Pharo <pharo-dev(a)lists.pharo.org>, The general-purpose Squeak developers list <squeak-dev(a)lists.squeakfoundation.org> > Subject: [squeak-dev] New Cog VMs available as per VMMaker.oscog-eem.1679/r3602 > Reply-To: The general-purpose Squeak developers list <squeak-dev(a)lists.squeakfoundation.org> > > Hi All, > > first of all, apologies for the long wait for new VMs. I know there are bugs that these VMs address and that it's been 5 months since VMs were last made is something I regret. The wait was due to my developing the 64-bit Spur Cog JIT VM which required surgery not just to the Cogit but also to Slang. I beg your forgiveness. > > Secondly, the Mac VMs are much changed, using John McIntosh's Cocoa support libraries. The old Carbon UI VMs are history. That means that Cog supports building on contemporary Mac OS releases. These VMs were built on 10.9; I like the 3D look much more than the 10.10 one ;-). The VMs are also signed, so installing them should be easier. > > Thirdly, these VMs include 64-bit JIT support. There is an updated squeak trunk image at http://www.mirandabanda.org/files/Cog/SpurImages for you to try. As yet the 64-bit Vms do not include an FFI plugin. I hope to rectify this soon. > > Finally I have included Pharo Mac VMs, not to displace the official Pharo VMs, but to test my build process. I hope that in very few months the svn repository will be moed to githup and that Esteban and I will build official VMs from the same repository. > > > > The VMs are here: http://www.mirandabanda.org/files/Cog/VM/VM.r3602 > > CogVM binaries as per VMMaker.oscog-eem.1679/r3602 > > Include 64-bit Spur VMs for Mac OS X x64 and Linux x64. N.B. As yet these VMs > lack FFI support. > > Move all Mac OS X VMs to the Cocoa libraries. Minimum supported version should > be 10.6.x. The Mac OS X VMs are signed and so should install properly without > having to disable the launch security checks. > > > Bug Fixes and Improvements: > > General: > Add vmParameter 20 to answer the utc microseconds at startup. > > > CoInterpreter: > Remove the atCache from the CoInterpreter. It's of little benefit given the > Cogit. > > ceSend: et al need to add looked up methods to the method cache, otherwise > methods only found through machine code sends will not be jitted. > > Fix slip in maybeSelectorOfMethod: which will improve back traces > including methods containing pragmas (including primitives). > > Modify secret primtiive 161, primitiveSetIdentityHash so that with 0 args it > answers whether an object has an identity hash. This is vacuously true for V3, > but is meaningful in Spur. > > > Cogit: > Fix /horrible/ bug with primitive error codes and fixups. The adjustment of the > initialPC to skip the primitive and error code, if any, was done after > generating fixups for backward branches, and hence these branches were bogus. > Object>>shallowCopy in Spur is subject to this, but amazingly enough it has not > surfaced before now. Fix this by adjusting the initialPC before scanMethod > scans for fixups. > > Fix bug in Spur machine-code at:put: on 32-bit bits objects; method failed to > fail for negative values. > > Fill unused portions of methods with the stop instruction, not nop. > > Explicitly initialize the code zone with stops. Explicitly clear the > reclaimed portion of the method zone after a compaction. > > Include the reclaimed portion of the method zone in the I-cache flush > after a compaction: no point in leaving it in cache since it isn't a > valid call/jump target. > > > Spur: > Maintain the survivor count on scavenge. > > Fix start up of images containing >= 16 segments. The old code assumed > numSegments < 16 and failed to grow the segment records, resulting in objects > in segments greater than 15 to not be swizzled, and a resulting crash. > > Spur Cogit: Support CompiledMethod indexing in genPrimitiveAt[Put]. Hence fix > the regression in at:[put:] whereby in Spur one could access the literals part > of a method as bytes. > Add a machine-code primitive for objectAt: to provide fast literal and header > access on Spur. > > Add 16-bit indexability to Spur. Generate slimmer code for > positive32BitValueOf: on 64-bit Spur. > > Fix a bug with out-of-place two-way become, which got the hash copy the wrong > way round. By default, the hash stays with the reference in two-way become. > > Fix what appears to be a compiler bug with in-place two-way-become by treating > the headers to be swapped so that remembered flags are swapped and hash bits > are swapped if required. This fixes a bug with in-place two-way become of > objects that are the same size on the heap. One object's contents would not > be updated correctly. > > Implement ephemeron queue primitive. Add an image flag that controls the > finalization support (StackInterpreter instVar newFinalization). If unset, > finalization is as currently expected; WeakArray finalizes all WeakArrays in > its registries on each finalization (which does not scale!). If set, then both > fired ephemerons and bereaved weak arrays are added to the finalization queue > and the finalization semaphore is signalled. In this regime the image is > expected to send mourn to each element of the queue, and weak arrays will send > themselves finalize in mourn, side-stepping the finalizeValues steps and > rendering WeakArray's FinalizationDependents and WeakRegistry obsolete. > The bit is set via > Smalltalk vmParameterAt: 48 > put: ((Smalltalk vmParameterAt: 48) bitOr: 64). > > > Slang: > Fix bad regression in type inference from VMMaker.oscog-eem.1587. > returnTypeForSend:in: can only default return types to sqInt for unknown > selectors. The rewrite in VMMaker.oscog-eem.1587 inadvertently defaulted the > return type of known methods whose return type was yet to be determined to > sqInt. The fix is to only default to sqInt if there is no known method for the > given selector. This fixes, for instance, the weird flipping of the type of > 32-bit Spur's headerWhileForwardingOf:,which should always be #sqLong. > > Fix bad bug in type inferrence. Avoid computing variable and return types > prematurely i.e. don't derive types from as-yet-untyped methods. > > Fix baaaad bug in node:typeCompatibleWith:inliningInto:in: which was looking > up the type of the formal parameter in inlining in the target method, not the > method being inlined! > > > Newspeak: > Fix bug in MNU for cogged dynamic super sends - should start looking for > #doesNotUnderstand: in the superclass not the receiver class. Unify MNU > lookup between absent and present receiver sends as a result. > > Further fix outer send MNU by not confusing lkupClass and lkupClassTag. > Push setting lkupClassTag down to clarify it is for new cache entries. > > > Plugins: > Fix the FilePlugin's unnecessary fullGC on Spur. > Include the file sync prim in the FilePrims plugin. > > ThreadedFFIPlugin: > - ExternalAddress now are taken correctly (nor as ByteArray or Alien, because > they are different beasts). > - allow pushing of pointers to any type (into a ByteArray, an ExternalAddress > or an Alien), to allow passing parameters style int*, float*, etc. > - allow reading in memory to allow read chunks of ByteArray > > Added ThreadedFFIPlugin>>#primitiveLoadSymbolFromModule (copy and adapt from > IA32ABIPlugin). > > Fix coercion problems on FFI plugin (IA32 & ARM) > - character were not correctly mapped back > - floats were not being well checked > - external addressed needs to be treated as aliens for the purpose of coercing > them as pointers (bad regression by Eliot) > > Alien Plugins: > Fix missing primitive failure for Alien typeAt: 0 [put:]. > Add word accessors to Alien. > > _,,,^..^,,,_ > best, Eliot >

Feb. 16, 2016