- Pharo-users - lists.pharo.org

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by Thierry Goubier

Le 29/02/2016 00:32, Ben Coman a Ã©crit : > On Sat, Feb 27, 2016 at 3:30 AM, Dimitris Chloupis > <kilon.alios(a)gmail.com> wrote: >> So now that AsmJit is gone , I had a new idea for a project and I wanted >> your opinion on this if you think it will be useful for the community, I >> call it WarpSpeed. >> >> I was thinking it would still be cool to have some kind of library that will >> help us boost the speed of pharo . So I was thinking about a C inliner >> which is far easier than an Assembly inliner (see AsmJIT) I also found >> TinyC >> >> http://bellard.org/tcc/ >> >> which is as you guessed a tiny c compiler 100kbs in size which i can >> include with a pharo image to allow me to dynamically compile inlined C code >> inside the pharo image so even pharo users without a C compiler installed >> can benefit from it . >> >> That means that if something does not perform as fast as you want you could >> mix C code with pharo code , I am thinking passing the C source code as a >> string or even having the C code in a separate file, and then dynamically >> call the TinyC compiler, compile it to shared library and load if via UFFI >> back to pharo. TinyC is very fast so compilation of the code should be as >> fast if not faster than compiling pharo code. >> >> Looks easy enough to do for a small project by me with the condition there >> are people really interested into something like this. TinyC is licensed as >> LGPL but I can make it fall back to other compilers as well if ones want to >> close the source. >> >> How it sounds as an idea ? > > I've thought about something similar myself, and also looked at tcc. > I think its a really interesting idea, but you need to treat it as an > *experiment* for learning. Whether its ultimately useful to the > community will depend on a lot of things in implementation (I see > Clement indicates some difficulties). > > But maybe tcc is not the right backend. [1] says "Note: I am no longer > working on TCC", the last release dates wehre 2009 & 2013, and [2] > indicates it tcc may be a dead end. > > What would be *really* interesting is, given we have a good > infrastructure to compile Smalltalk to bytecode, investigate some new > constructs to compile to SPIR-V [3]. The specification [4] and > execution/memory model [5] have recently been released. It would be > nice if someone more competent than myself could skim and advise on > the feasibility of such an idea. SPIR-V format is similar to the LLVM-IR and it is quite hard to generate for a language like Smalltalk. Typed operations, SSA, Phi nodes, etc... You get extremely high performance this way, but it is a lot of work. > This presentation [6] starting p38 says SPIRV presents an "opportunity > to unleash innovation: Domain Specific Languages..." which is > typically considered a strength of Smalltalk. p40 says "Compiler split > in two, Front end compiler emits SPIR-V portable binary offline (that > would be us), then SPIR-V is compiled to machine-specific binary by > driver online (that would be just passing the SPIR-V to the GPU > driver). [7] Yes, this is certainly where SPIR-V is interesting: all OpenCL devices would carry a SPIR-V back-end compiler. > Perhaps interesting to see what people are trying in some other languages... > * A bytecode translator like is being tried for Python [A], since SPIR-V is > * End-user tools for working with SPIR-V, disassembling, analysing, > debugging, optimising it, for Go[B] and Python[C]. > > The opportunity in producing tools for working with SPIR-V is you draw > people in for the tool, and they get a side-exposure to our > environment, (which of course they then fall in love with.) The opportunity is more like a compiler infrastructure you'll be writing. If well written, yes, it could be an interesting artefact (training, teaching compilation). Thierry > > [1] http://bellard.org/tcc/ > [2] http://www.landley.net/code/tinycc/ > [3] https://en.wikipedia.org/wiki/Standard_Portable_Intermediate_Representation > [4] https://www.khronos.org/registry/spir-v/specs/1.0/SPIRV.pdf > [5] https://www.khronos.org/registry/spir-v/specs/1.0/SPIR-V-execution-and-memo… > [6] > http://vulkan-tutorial.com/assets/Khronos-Vulkan-GDC-Mar15.pdf > [7] http://benedictgaster.org/wp-content/uploads/2012/08/building_700_languages… > > > [A] https://github.com/cheery/spirthon/blob/master/DEV.md > [B] https://github.com/andreas-jonsson/spirv > [C] https://github.com/kristerw/spirv-tools > >

Feb. 28, 2016

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by kilon.alios

TinyC has not similar perfomance to gcc and clang, its actually slower as Stephan already mentioned. Thats the price one has to pay for the much smaller compiler, it loses a lot of optimisations. Clement from what you saying its clear my idea would not work at all, the advantage of tinyC is that is 10 times faster than gcc and clang for compilation but the FFI overheads would kill performance indeed. And if the code becomes much larger it does not make sense to use a C inliner as it would be cleaner and easier to just keep the C code separate. Ben TinyC has been abandoned by the original author but it is now even more actively developed because several people have stepped in his place , its git repo can be found here http://repo.or.cz/w/tinycc.git its not near as actively developed as gcc or clang of course but its not inactive either. Anyway this is obviously a ton more work than I am willing to invest and with no real benefit, so thanks for opening my eyes. Back to something far more simple. -- View this message in context: http://forum.world.st/A-new-idea-for-a-project-WarpSpeed-a-C-inliner-tp4881… Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Feb. 28, 2016

SortFunctions

by stepharo

I enjoyed reading this. I do not know if we have this simple but cool lib for Pharo but we should. Stef Objology: TAG-SortFunctions Objology: TAG-SortFunctions Objology: TAG-SortFunctions

Feb. 28, 2016

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by Ben Coman

On Sat, Feb 27, 2016 at 3:30 AM, Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > So now that AsmJit is gone , I had a new idea for a project and I wanted > your opinion on this if you think it will be useful for the community, I > call it WarpSpeed. > > I was thinking it would still be cool to have some kind of library that will > help us boost the speed of pharo . So I was thinking about a C inliner > which is far easier than an Assembly inliner (see AsmJIT) I also found > TinyC > > http://bellard.org/tcc/ > > which is as you guessed a tiny c compiler 100kbs in size which i can > include with a pharo image to allow me to dynamically compile inlined C code > inside the pharo image so even pharo users without a C compiler installed > can benefit from it . > > That means that if something does not perform as fast as you want you could > mix C code with pharo code , I am thinking passing the C source code as a > string or even having the C code in a separate file, and then dynamically > call the TinyC compiler, compile it to shared library and load if via UFFI > back to pharo. TinyC is very fast so compilation of the code should be as > fast if not faster than compiling pharo code. > > Looks easy enough to do for a small project by me with the condition there > are people really interested into something like this. TinyC is licensed as > LGPL but I can make it fall back to other compilers as well if ones want to > close the source. > > How it sounds as an idea ? I've thought about something similar myself, and also looked at tcc. I think its a really interesting idea, but you need to treat it as an *experiment* for learning. Whether its ultimately useful to the community will depend on a lot of things in implementation (I see Clement indicates some difficulties). But maybe tcc is not the right backend. [1] says "Note: I am no longer working on TCC", the last release dates wehre 2009 & 2013, and [2] indicates it tcc may be a dead end. What would be *really* interesting is, given we have a good infrastructure to compile Smalltalk to bytecode, investigate some new constructs to compile to SPIR-V [3]. The specification [4] and execution/memory model [5] have recently been released. It would be nice if someone more competent than myself could skim and advise on the feasibility of such an idea. This presentation [6] starting p38 says SPIRV presents an "opportunity to unleash innovation: Domain Specific Languages..." which is typically considered a strength of Smalltalk. p40 says "Compiler split in two, Front end compiler emits SPIR-V portable binary offline (that would be us), then SPIR-V is compiled to machine-specific binary by driver online (that would be just passing the SPIR-V to the GPU driver). [7] Perhaps interesting to see what people are trying in some other languages... * A bytecode translator like is being tried for Python [A], since SPIR-V is * End-user tools for working with SPIR-V, disassembling, analysing, debugging, optimising it, for Go[B] and Python[C]. The opportunity in producing tools for working with SPIR-V is you draw people in for the tool, and they get a side-exposure to our environment, (which of course they then fall in love with.) [1] http://bellard.org/tcc/ [2] http://www.landley.net/code/tinycc/ [3] https://en.wikipedia.org/wiki/Standard_Portable_Intermediate_Representation [4] https://www.khronos.org/registry/spir-v/specs/1.0/SPIRV.pdf [5] https://www.khronos.org/registry/spir-v/specs/1.0/SPIR-V-execution-and-memo… [6] http://vulkan-tutorial.com/assets/Khronos-Vulkan-GDC-Mar15.pdf [7] http://benedictgaster.org/wp-content/uploads/2012/08/building_700_languages… [A] https://github.com/cheery/spirthon/blob/master/DEV.md [B] https://github.com/andreas-jonsson/spirv [C] https://github.com/kristerw/spirv-tools

Feb. 28, 2016

minimal image load config question

by Robert Kuszinger

Hello! I'm trying to load my app (config) into the minimal image from the command line. pharo min.image config http://smalltalkhub.com/mc/RobertKuszinger/GcUploader/main ConfigurationOfGiscomKkt --install=stable --username=<mylogin> --password=<mypwd> After a few seconds of working it stops like this (longer version at the end if this email): [....] OrderedCollection(Collection)>>do:displayingProgress:every: OrderedCollection(Collection)>>do:displayingProgress: *Got startup errors: * * MessageNotUnderstood: Unicode class>>isUnifiedKanji:* Does anyone have any idea *what to check*? Is there something missing? Minimal image is from the Pharo website. I've tested my config on a brand new Pharo4 image and it loads well... Should I preload another config into the image? Thanks -------------------------------- [kuszi@kuszidell pharo-minimal]$ pharo --version 3.9-7 #1 Thu Jan 7 00:37:02 CET 2016 gcc 4.6.3 [Production ITHB VM] NBCoInterpreter NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Jan 7 2016 NBCogit NativeBoost-CogPlugin-EstebanLorenzano.21 uuid: 4d9b9bdf-2dfa-4c0b-99eb-5b110dadc697 Jan 7 2016 https://github.com/pharo-project/pharo-vm.git Commit: bad6fa2684afcf1d7cdb485f11049a4a4290fbc8 Date: 2015-12-14 12:47:06 +0100 By: Esteban Lorenzano <estebanlm(a)gmail.com> Jenkins build #15027 Linux pharo-linux 3.2.0-31-generic-pae #50-Ubuntu SMP Fri Sep 7 16:39:45 UTC 2012 i686 i686 i386 GNU/Linux plugin path: /usr/share/pharo-vm/ [default: /usr/share/pharo-vm/] --------------------------------- MethodAddition>>writeSourceToLog MethodAddition>>createCompiledMethod MCMethodDefinition>>addMethodAdditionTo: [ :each | each addMethodAdditionTo: methodAdditions ] in MCPackageLoader>>basicLoadDefinitions in Block: [ :each | each addMethodAdditionTo: methodAddition...etc... [ :each | | newLabel | "Special handling for first and last element" (count = 0 or: [ count + 1 = size or: [ (Time millisecondsSince: lastUpdate) >= msecs ] ]) ifTrue: [ bar current: count. oldLabel = (newLabel := (labelBlock cull: each) ifNil: [ oldLabel ]) ifFalse: [ bar label: newLabel. ProgressNotification signal: '' extra: (oldLabel := newLabel) ]. lastUpdate := Time millisecondClockValue ]. aBlock value: each. count := count + 1 ] in [ :bar | labelBlock := aStringOrBlock isString ifTrue: [ bar label: aStringOrBlock. [ :dummyItem | aStringOrBlock ] ] ifFalse: [ aStringOrBlock ]. self do: [ :each | | newLabel | "Special handling for first and last element" (count = 0 or: [ count + 1 = size or: [ (Time millisecondsSince: lastUpdate) >= msecs ] ]) ifTrue: [ bar current: count. oldLabel = (newLabel := (labelBlock cull: each) ifNil: [ oldLabel ]) ifFalse: [ bar label: newLabel. ProgressNotification signal: '' extra: (oldLabel := newLabel) ]. lastUpdate := Time millisecondClockValue ]. aBlock value: each. count := count + 1 ] ] in OrderedCollection(Collection)>>do:displayingProgress:every: in Block: [ :each | ... OrderedCollection>>do: [ :bar | labelBlock := aStringOrBlock isString ifTrue: [ bar label: aStringOrBlock. [ :dummyItem | aStringOrBlock ] ] ifFalse: [ aStringOrBlock ]. self do: [ :each | | newLabel | "Special handling for first and last element" (count = 0 or: [ count + 1 = size or: [ (Time millisecondsSince: lastUpdate) >= msecs ] ]) ifTrue: [ bar current: count. oldLabel = (newLabel := (labelBlock cull: each) ifNil: [ oldLabel ]) ifFalse: [ bar label: newLabel. ProgressNotification signal: '' extra: (oldLabel := newLabel) ]. lastUpdate := Time millisecondClockValue ]. aBlock value: each. count := count + 1 ] ] in OrderedCollection(Collection)>>do:displayingProgress:every: in Block: [ :bar | ... BlockClosure>>cull: [ result := block cull: self ] in [ self prepareForRunning. [ result := block cull: self ] on: JobNotification do: [ :notification | notification handle: self ] ] in Job>>run in Block: [ result := block cull: self ] BlockClosure>>on:do: [ self prepareForRunning. [ result := block cull: self ] on: JobNotification do: [ :notification | notification handle: self ] ] in Job>>run in Block: [ ... BlockClosure>>ensure: Job>>run DummyUIManager(UIManager)>>displayProgress:from:to:during: ByteString(String)>>displayProgressFrom:to:during: OrderedCollection(Collection)>>do:displayingProgress:every: OrderedCollection(Collection)>>do:displayingProgress: Got startup errors: MessageNotUnderstood: Unicode class>>isUnifiedKanji:

Feb. 28, 2016

Re: [Pharo-users] Who is using Pharo Launcher and for what

by Stephan Eggermont

On 20-02-16 11:49, Damien Cassou wrote: > I don't need long paragraphs, just give me the keywords please. Today I triaged a bug in CodePanel by downloading image versions 50600 50590 50595 50597 and 50596 and installing NewUI from the catalog browser to find out that it was broken by a change in 50596. With PharoLauncher it took me about 5 minutes, just long enough to wonder if I should automate this further, and just short enough to not bother. Stephan

Feb. 28, 2016

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by Stephan Eggermont

On 28-02-16 10:17, ClÃ©ment Bera wrote: > assuming tinyC has performance similar to gcc > or llvm, you may get faster, It does not. On the website they compare to -O1 speed and then are already twice as slow Stephan

Feb. 28, 2016

Helpful info for Fuel data migration

by Hilaire

Hello guys, It will be very helpful to migrate fuel data from one image to another to have written somewhere on the Fuel page[1], the Fuel number version shipped with each image release. Thanks Hilaire [1] http://rmod.inria.fr/web/software/Fuel -- Dr. Geo http://drgeo.eu

Feb. 28, 2016

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by ClÃ©ment Bera

Hello, As Stephan Eggermont said, you need to make some measurements. There are a few big problems. In short, if the FFI call is going to do quite some computation (at least a few millisecond), assuming tinyC has performance similar to gcc or llvm, you may get faster, if the FFI call is going to do very little (calling Math.abs() for example as Pharo has no primitive for abs), the overhead of the call is going to be too big. Maybe you can make your runtime with C worth it, but for example it's clear that if you write tiny methods in C for reusability / modularity of your code and you bind each tiny method to the smalltalk runtime, using those tiny methods one by one is not going to be worth it. However, if you write a merge sort or another algorithm fully in C and do a single call to the algorithm to sort a collection, on large collection and large collections only, it's going to be worth it. In details, the main problems are: - the Smalltalk stack has specific constraints, like hand-written stack page management to support heavy stack manipulation (setting sender of a context from the image). Hence, a C stack cannot just grow inside the Smalltalk stack as in other languages directly integrated to C. The way FFI tackles this problem is by using the interpreter C stack to execute C code. This is problematic as the runtime needs to switch from assembly runtime (Smalltalk method generated from the jit) to C runtime for the FFI call and then switch back, which induces overhead in jitted code, i.e., performance critical code. Maybe you could change the way the jitted code activates C code by checking the stack size needed by the dynamic lib API and if there is enough room on the smalltalk stack, then call it on the Smalltalk stack, though it creates other problems (what if garbage collection happens from another native thread while the FFI call is performed in non blocking FFI ?). - you need to convert types between Smalltalk to C at each call inducing FFI call overhead or makes the C code aware of your data structures which you have to support for those Smalltalk data structure yourself and induces some overhead in the C runtime. In Smalltalk/X, the C runtime is directly integrated in the Smalltalk runtime. You can ask Jan Vrany how he does it. But I think he doesn't have our hand written stack management (I am not 100% sure). 2016-02-28 2:04 GMT+01:00 Stephan Eggermont <stephan(a)stack.nl>: > On 26/02/16 20:30, Dimitris Chloupis wrote: > >> I was thinking it would still be cool to have some kind of library that >> will help us boost the speed of pharo . So I was thinking about a C >> inliner which is far easier than an Assembly inliner (see AsmJIT) I also >> found TinyC >> > > The first thing to test is if TinyC is actually faster than Pharo. > > Stephan > > > >

Feb. 28, 2016

Re: [Pharo-users] A new idea for a project: WarpSpeed (a C inliner)

by Esteban Lorenzano

> On 28 Feb 2016, at 02:04, Stephan Eggermont <stephan(a)stack.nl> wrote: > > On 26/02/16 20:30, Dimitris Chloupis wrote: >> I was thinking it would still be cool to have some kind of library that >> will help us boost the speed of pharo . So I was thinking about a C >> inliner which is far easier than an Assembly inliner (see AsmJIT) I also >> found TinyC > > The first thing to test is if TinyC is actually faster than Pharo. yes! but it can be a nice addition to gain part of what is lost with the removal of ASMJIT (not that anyone was using it, but well... :P) > > Stephan > > >

Feb. 28, 2016