March 2017 - Pharo-dev - lists.pharo.org

Missing Zip64 support

by Pavel Krivanek

Hi, our current Compression package does not support Zip64 which is limiting us in usage of Metacello together with Git. When we use the Metacello for installing of a baseline like: Metacello new baseline: 'IDE'; repository: 'github://guillep/pharo-core'; load. then the Metacello downloads the corresponding archive from Github. However the standard Zip archives have count limit of 65535 members but the pharo-core repository and some others contain much more than this (pharo-core has 125021 files). The Zip64 extension is used for them during compression which we cannot read now. So if you will see Error: bad signature 16r... at position ..., then it has this reason. The workaround for now is to load projects from a local clone of such big repositories. Cheers, -- Pavel

March 15, 2017

[ANN] Collecting all Pharo related publications

by Marcus Denker

Hi, We want to create a list of all Pharo related papers. I have added this page: http://pharo.org/web/PharoPublications <http://pharo.org/web/PharoPublications> it is linked from http://pharo.org/documentation <http://pharo.org/documentation> Marcus

March 15, 2017

Re: [Pharo-dev] [Vm-dev] BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)

by Igor Stasenko

On 15 March 2017 at 11:53, Igor Stasenko <siguctua(a)gmail.com> wrote: > Here's my d) > implement callback functions in C, or in native form => no need for > entering the smalltalk execution => no risk of GC => nothing to worry about. > > I guess nobody will like it (and will be right, of course ;) , but it is > how it was originally done. I used NativeBoost to implement those callback > functions and they're won't cause any GC problems. > > That, of course, gave me solution in this concrete case, but not in > general.. i.e. : if you have another callback that cannot be implemented > na(t)ively, then > you facing similar problems, mainly: how to work around the problem, that > primitive(s) that using callbacks may capture state, that are subject of GC > activity. > > In general , then, i think such primitive should be (re)written in such > way , that it won't get puzzled by GC.. and addGCRoot(s), IMO then best > way, from general interfacing/implementation standpoint. > I would just add extra interface for using it especially in primitives, so > that it > 1) won't punish primitive writer with too much coding > 2) automatically handle primitive/callback nesting e.g. > primitive1 -> adds roots1 -> calls fn -> callback -> st code -> primitive2 > -> adds roots2 -> calls fn2 -> callback2 ... > > > something like this: > > static initialized once myprimooptable = [ a,b,c]. > vm pushPrimRoots: myooptable. > self do things primitive does. > vm popPrimRoots > > or, since we have green threading, then maybe better will be in this form: > > rootsId := static initialized once myprimooptable = [ a,b,c]. > vm pushPrimRoots: myooptable. > self do things primitive does. > vm popPrimRoots: rootsId. > > oops, i meant: static initialized once myprimooptable = [ a,b,c]. rootsId := vm pushPrimRoots: myooptable. self do things primitive does. vm popPrimRoots: rootsId. the idea is mainly, that VM answers you some key, using which you can later identify, what to release. > > > On 14 March 2017 at 18:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > >> >> >> >> On Tue, Mar 14, 2017 at 8:56 AM, Nicolai Hess <nicolaihess(a)gmail.com> >> wrote: >> >>> >>> >>> >>> 2017-03-14 16:46 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >>> >>>> >>>> Hi Esteban, Hi Igor, Hi All, >>>> >>>> On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <estebanlm(a)gmail.com >>>> > wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>> Iâm tumbling into an error in Pharo, because we use callbacks >>>>> intensively, in Athens(cairo)-to-World conversion in particular, and people >>>>> is sending always their crash reportsâ¦ we made the whole conversion a lot >>>>> more robust since problems started to arise, but now I hit a wall I cannot >>>>> solve: I think problem is in something in callbacks. >>>>> >>>>> And problem is showing very easy on 64bits (while in 32bits it takes >>>>> time and is more random). >>>>> >>>> >>>> I responded in the "image not opening" thread, but it's the same >>>> problem. I really want to hear what y'all think because I'll happily >>>> implement a fix, but I want to know which one y'all think is a good idea. >>>> Here's my reply (edits between [ & ] to add information): >>>> >>>> >>>> On Mon, Mar 13, 2017 at 9:11 PM, Eliot Miranda <eliot.miranda(a)gmail.com >>>> > wrote: >>>> >>>> I'm pretty confident [I know] this is to do with bugs in the Athens >>>> surface code which assumes that callbacks can be made in the existing >>>> copyBits and warpBits primitive. They can't do this safely because a GC >>>> (scavenge) can happen during a callback, which then causes chaos when the >>>> copyBits primitive tries to access objects that have been moved under its >>>> feet. >>>> >>>> I've done work to fix callbacks so that when there is a failure it is >>>> the copyBits primitive that fails, instead of apparently the callback >>>> return primitive. One of the apparent effects of this fix is to stop the >>>> screen opening up too small; another is getting the background colour >>>> right, and yet another is eliminating bogus pixels in the VGTigerDemo >>>> demo. But more work is required to fix the copyBits and warpBits >>>> primitives. There are a few approaches one might take: >>>> >>>> a) fixing the primitive so that it saves and restores oops around the >>>> callbacks using the external oop table [InterpreterProxy>>addGCRoot: & >>>> removeGCRoot:]. That's a pain but possible. [It's a pain because all the >>>> derived pointers (the start of the destForm, sourceForm, halftoneForm and >>>> colorMapTable) must be recomputed also, and of course most of the time the >>>> objects don't move; we only scavenge about once every 2 seconds in normal >>>> running] >>>> >>>> b) fixing the primitive so that it pins the objects it needs before >>>> ever invoking a callback [this is a pain because pinning an object causes >>>> it to be tenured to old space if it is in new space; objects can't be >>>> pinned in new space, so instead the pin operation forwards the new space >>>> object to an old space copy if required and answers its location in old >>>> space, so a putative withPinnedObjectsDo: operation for the copyBits >>>> primitive looks like >>>> withPinnedFormsDo: aBlock >>>> <inline: #always> >>>> self cppIf: SPURVM & false >>>> ifTrue: >>>> [| bitBltOopWasPinned destWasPinned sourceWasPinned halftoneWasPinned | >>>> (bitBltOopWasPinned := interpreterProxy isPinned: bitBltOop) ifFalse: >>>> [bitBltOop := interpreterProxy pinObject: bitBltOop]. >>>> (destWasPinned := interpreterProxy isPinned: destForm) ifFalse: >>>> [destForm := interpreterProxy pinObject: destForm]. >>>> (sourceWasPinned := interpreterProxy isPinned: sourceForm) ifFalse: >>>> [sourceForm := interpreterProxy pinObject: sourceForm]. >>>> (halftoneWasPinned := interpreterProxy isPinned: halftoneForm) ifFalse: >>>> [halftoneForm := interpreterProxy pinObject: halftoneForm]. >>>> aBlock value. >>>> bitBltOopWasPinned ifFalse: [interpreterProxy unpinObject: bitBltOop]. >>>> destWasPinned ifFalse: [interpreterProxy unpinObject: destForm]. >>>> sourceWasPinned ifFalse: [interpreterProxy unpinObject: sourceForm]. >>>> halftoneWasPinned ifFalse: [interpreterProxy unpinObject: halftoneForm]] >>>> ifFalse: [aBlock value] >>>> and tenuring objects to old space is not ideal because they are only >>>> collected by a full GC, so doing this would at least tenure the bitBltOop >>>> which is very likely to be in new space] >>>> >>>> c) fixing the primitive so that it uses the scavenge and fullGC >>>> counters in the VM to detect if a GC occurred during one of the callbacks >>>> and would fail the primitive [if it detected that a GC had occurred in any >>>> of the surface functions]. The primitive would then simply be retried. >>>> >>>> d) ? >>>> >>> >>> Wouldn't it be possible to just pause the GC (scavange) when entering a >>> primitive ? >>> >> >> I don't think so. There is a callback occurring. If the computation >> executed by the callback requires a GC the application will abort if a GC >> cannot be done. Right? This is the case here. >> >> >> I like c) as it's very lightweight, but it has issues. It is fine to use >>>> for callbacks *before* cop[yBits and warpBits move any bits (the >>>> lockSurface and querySurface functions). But it's potentially erroneous >>>> after the unlockSurface primitive. For example, a primitive which does an >>>> xor with the screen can't simply be retried as the first, falling pass, >>>> would have updated the destination bits but not displayed them via >>>> unlockSurface. But I think it could be arranged that no objects are >>>> accessed after unlockSurface, which should naturally be the last call in >>>> the primitive (or do I mean showSurface?). So the approach would be to >>>> check for GCs occurring during querySurface and lockSurface, failing if so, >>>> and then caching any and all state needed by unlockSurface and showSurface >>>> in local variables. This way no object state is accessed to make the >>>> unlockSurface and showSurface calls, and no bits are moved before the >>>> queryDurface and lockSurface calls. >>>> >>>> If we used a failure code such as #'object may move' then the >>>> primitives could answer this when a GC during callbacks is detected and >>>> then the primitive could be retried only when required. >>>> >>>> >>>> [Come on folks, please comment. I want to know which idea you like >>>> best. We could fix this quickly. But right now it feels like I'm talking >>>> to myself.] >>>> >>>> >>>>> Here is the easiest way to reproduce it (in mac): >>>>> >>>>> wget files.pharo.org/get-files/60/pharo64-mac-latest.zip >>>>> wget files.pharo.org/get-files/60/pharo64.zip >>>>> wget files.pharo.org/get-files/60/sources.zip >>>>> unzip pharo64-mac-latest.zip >>>>> unzip pharo64.zip >>>>> unzip sources.zip >>>>> ./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval >>>>> "VGTigerDemo runDemo" >>>>> >>>>> eventually (like 5-6 seconds after, if not immediately), you will have >>>>> a stack like this: >>>>> >>>>> SmallInteger(Object)>>primitiveFailed: >>>>> SmallInteger(Object)>>primitiveFailed >>>>> SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext: >>>>> GrafPort>>copyBits >>>>> GrafPort>>image:at:sourceRect:rule: >>>>> FormCanvas>>image:at:sourceRect:rule: >>>>> FormCanvas(Canvas)>>drawImage:at:sourceRect: >>>>> FormCanvas(Canvas)>>drawImage:at: >>>>> VGTigerDemo>>runDemo >>>>> VGTigerDemo class>>runDemo >>>>> UndefinedObject>>DoIt >>>>> OpalCompiler>>evaluate >>>>> OpalCompiler(AbstractCompiler)>>evaluate: >>>>> [ result := Smalltalk compiler evaluate: aStream. >>>>> self hasSessionChanged >>>>> ifFalse: [ self stdout >>>>> print: result; >>>>> lf ] ] in EvaluateCommandLineHandler>>evaluate: >>>>> in Block: [ result := Smalltalk compiler evaluate: aStream.... >>>>> BlockClosure>>on:do: >>>>> EvaluateCommandLineHandler>>evaluate: >>>>> EvaluateCommandLineHandler>>evaluateArguments >>>>> EvaluateCommandLineHandler>>activate >>>>> EvaluateCommandLineHandler class(CommandLineHandler >>>>> class)>>activateWith: >>>>> [ aCommandLinehandler activateWith: commandLine ] in >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>>> in Block: [ aCommandLinehandler activateWith: commandLine ] >>>>> BlockClosure>>on:do: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument: >>>>> [ self >>>>> handleArgument: >>>>> (self arguments >>>>> ifEmpty: [ '' ] >>>>> ifNotEmpty: [ :arguments | arguments first ]) >>>>> ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in >>>>> Block: [ self... >>>>> BlockClosure>>on:do: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activate >>>>> PharoCommandLineHandler>>activate >>>>> PharoCommandLineHandler class(CommandLineHandler class)>>activateWith: >>>>> [ super activateWith: aCommandLine ] in PharoCommandLineHandler >>>>> class>>activateWith: in Block: [ super activateWith: aCommandLine ] >>>>> >>>>> Any idea? >>>>> >>>>> thanks! >>>>> Esteban >>>>> >>>> >>>> >>>> >>>> -- >>>> _,,,^..^,,,_ >>>> best, Eliot >>>> >>>> >>> >>> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> > > > -- > Best regards, > Igor Stasenko. > -- Best regards, Igor Stasenko.

March 15, 2017

Re: [Pharo-dev] [Vm-dev] BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)

by Igor Stasenko

Here's my d) implement callback functions in C, or in native form => no need for entering the smalltalk execution => no risk of GC => nothing to worry about. I guess nobody will like it (and will be right, of course ;) , but it is how it was originally done. I used NativeBoost to implement those callback functions and they're won't cause any GC problems. That, of course, gave me solution in this concrete case, but not in general.. i.e. : if you have another callback that cannot be implemented na(t)ively, then you facing similar problems, mainly: how to work around the problem, that primitive(s) that using callbacks may capture state, that are subject of GC activity. In general , then, i think such primitive should be (re)written in such way , that it won't get puzzled by GC.. and addGCRoot(s), IMO then best way, from general interfacing/implementation standpoint. I would just add extra interface for using it especially in primitives, so that it 1) won't punish primitive writer with too much coding 2) automatically handle primitive/callback nesting e.g. primitive1 -> adds roots1 -> calls fn -> callback -> st code -> primitive2 -> adds roots2 -> calls fn2 -> callback2 ... something like this: static initialized once myprimooptable = [ a,b,c]. vm pushPrimRoots: myooptable. self do things primitive does. vm popPrimRoots or, since we have green threading, then maybe better will be in this form: rootsId := static initialized once myprimooptable = [ a,b,c]. vm pushPrimRoots: myooptable. self do things primitive does. vm popPrimRoots: rootsId. On 14 March 2017 at 18:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > On Tue, Mar 14, 2017 at 8:56 AM, Nicolai Hess <nicolaihess(a)gmail.com> > wrote: > >> >> >> >> 2017-03-14 16:46 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >> >>> >>> Hi Esteban, Hi Igor, Hi All, >>> >>> On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <estebanlm(a)gmail.com> >>> wrote: >>> >>>> >>>> Hi, >>>> >>>> Iâm tumbling into an error in Pharo, because we use callbacks >>>> intensively, in Athens(cairo)-to-World conversion in particular, and people >>>> is sending always their crash reportsâ¦ we made the whole conversion a lot >>>> more robust since problems started to arise, but now I hit a wall I cannot >>>> solve: I think problem is in something in callbacks. >>>> >>>> And problem is showing very easy on 64bits (while in 32bits it takes >>>> time and is more random). >>>> >>> >>> I responded in the "image not opening" thread, but it's the same >>> problem. I really want to hear what y'all think because I'll happily >>> implement a fix, but I want to know which one y'all think is a good idea. >>> Here's my reply (edits between [ & ] to add information): >>> >>> >>> On Mon, Mar 13, 2017 at 9:11 PM, Eliot Miranda <eliot.miranda(a)gmail.com> >>> wrote: >>> >>> I'm pretty confident [I know] this is to do with bugs in the Athens >>> surface code which assumes that callbacks can be made in the existing >>> copyBits and warpBits primitive. They can't do this safely because a GC >>> (scavenge) can happen during a callback, which then causes chaos when the >>> copyBits primitive tries to access objects that have been moved under its >>> feet. >>> >>> I've done work to fix callbacks so that when there is a failure it is >>> the copyBits primitive that fails, instead of apparently the callback >>> return primitive. One of the apparent effects of this fix is to stop the >>> screen opening up too small; another is getting the background colour >>> right, and yet another is eliminating bogus pixels in the VGTigerDemo >>> demo. But more work is required to fix the copyBits and warpBits >>> primitives. There are a few approaches one might take: >>> >>> a) fixing the primitive so that it saves and restores oops around the >>> callbacks using the external oop table [InterpreterProxy>>addGCRoot: & >>> removeGCRoot:]. That's a pain but possible. [It's a pain because all the >>> derived pointers (the start of the destForm, sourceForm, halftoneForm and >>> colorMapTable) must be recomputed also, and of course most of the time the >>> objects don't move; we only scavenge about once every 2 seconds in normal >>> running] >>> >>> b) fixing the primitive so that it pins the objects it needs before ever >>> invoking a callback [this is a pain because pinning an object causes it to >>> be tenured to old space if it is in new space; objects can't be pinned in >>> new space, so instead the pin operation forwards the new space object to an >>> old space copy if required and answers its location in old space, so a >>> putative withPinnedObjectsDo: operation for the copyBits primitive looks >>> like >>> withPinnedFormsDo: aBlock >>> <inline: #always> >>> self cppIf: SPURVM & false >>> ifTrue: >>> [| bitBltOopWasPinned destWasPinned sourceWasPinned halftoneWasPinned | >>> (bitBltOopWasPinned := interpreterProxy isPinned: bitBltOop) ifFalse: >>> [bitBltOop := interpreterProxy pinObject: bitBltOop]. >>> (destWasPinned := interpreterProxy isPinned: destForm) ifFalse: >>> [destForm := interpreterProxy pinObject: destForm]. >>> (sourceWasPinned := interpreterProxy isPinned: sourceForm) ifFalse: >>> [sourceForm := interpreterProxy pinObject: sourceForm]. >>> (halftoneWasPinned := interpreterProxy isPinned: halftoneForm) ifFalse: >>> [halftoneForm := interpreterProxy pinObject: halftoneForm]. >>> aBlock value. >>> bitBltOopWasPinned ifFalse: [interpreterProxy unpinObject: bitBltOop]. >>> destWasPinned ifFalse: [interpreterProxy unpinObject: destForm]. >>> sourceWasPinned ifFalse: [interpreterProxy unpinObject: sourceForm]. >>> halftoneWasPinned ifFalse: [interpreterProxy unpinObject: halftoneForm]] >>> ifFalse: [aBlock value] >>> and tenuring objects to old space is not ideal because they are only >>> collected by a full GC, so doing this would at least tenure the bitBltOop >>> which is very likely to be in new space] >>> >>> c) fixing the primitive so that it uses the scavenge and fullGC counters >>> in the VM to detect if a GC occurred during one of the callbacks and would >>> fail the primitive [if it detected that a GC had occurred in any of the >>> surface functions]. The primitive would then simply be retried. >>> >>> d) ? >>> >> >> Wouldn't it be possible to just pause the GC (scavange) when entering a >> primitive ? >> > > I don't think so. There is a callback occurring. If the computation > executed by the callback requires a GC the application will abort if a GC > cannot be done. Right? This is the case here. > > > I like c) as it's very lightweight, but it has issues. It is fine to use >>> for callbacks *before* cop[yBits and warpBits move any bits (the >>> lockSurface and querySurface functions). But it's potentially erroneous >>> after the unlockSurface primitive. For example, a primitive which does an >>> xor with the screen can't simply be retried as the first, falling pass, >>> would have updated the destination bits but not displayed them via >>> unlockSurface. But I think it could be arranged that no objects are >>> accessed after unlockSurface, which should naturally be the last call in >>> the primitive (or do I mean showSurface?). So the approach would be to >>> check for GCs occurring during querySurface and lockSurface, failing if so, >>> and then caching any and all state needed by unlockSurface and showSurface >>> in local variables. This way no object state is accessed to make the >>> unlockSurface and showSurface calls, and no bits are moved before the >>> queryDurface and lockSurface calls. >>> >>> If we used a failure code such as #'object may move' then the primitives >>> could answer this when a GC during callbacks is detected and then the >>> primitive could be retried only when required. >>> >>> >>> [Come on folks, please comment. I want to know which idea you like >>> best. We could fix this quickly. But right now it feels like I'm talking >>> to myself.] >>> >>> >>>> Here is the easiest way to reproduce it (in mac): >>>> >>>> wget files.pharo.org/get-files/60/pharo64-mac-latest.zip >>>> wget files.pharo.org/get-files/60/pharo64.zip >>>> wget files.pharo.org/get-files/60/sources.zip >>>> unzip pharo64-mac-latest.zip >>>> unzip pharo64.zip >>>> unzip sources.zip >>>> ./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval >>>> "VGTigerDemo runDemo" >>>> >>>> eventually (like 5-6 seconds after, if not immediately), you will have >>>> a stack like this: >>>> >>>> SmallInteger(Object)>>primitiveFailed: >>>> SmallInteger(Object)>>primitiveFailed >>>> SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext: >>>> GrafPort>>copyBits >>>> GrafPort>>image:at:sourceRect:rule: >>>> FormCanvas>>image:at:sourceRect:rule: >>>> FormCanvas(Canvas)>>drawImage:at:sourceRect: >>>> FormCanvas(Canvas)>>drawImage:at: >>>> VGTigerDemo>>runDemo >>>> VGTigerDemo class>>runDemo >>>> UndefinedObject>>DoIt >>>> OpalCompiler>>evaluate >>>> OpalCompiler(AbstractCompiler)>>evaluate: >>>> [ result := Smalltalk compiler evaluate: aStream. >>>> self hasSessionChanged >>>> ifFalse: [ self stdout >>>> print: result; >>>> lf ] ] in EvaluateCommandLineHandler>>evaluate: >>>> in Block: [ result := Smalltalk compiler evaluate: aStream.... >>>> BlockClosure>>on:do: >>>> EvaluateCommandLineHandler>>evaluate: >>>> EvaluateCommandLineHandler>>evaluateArguments >>>> EvaluateCommandLineHandler>>activate >>>> EvaluateCommandLineHandler class(CommandLineHandler >>>> class)>>activateWith: >>>> [ aCommandLinehandler activateWith: commandLine ] in >>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>> in Block: [ aCommandLinehandler activateWith: commandLine ] >>>> BlockClosure>>on:do: >>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand >>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument: >>>> [ self >>>> handleArgument: >>>> (self arguments >>>> ifEmpty: [ '' ] >>>> ifNotEmpty: [ :arguments | arguments first ]) ] >>>> in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in >>>> Block: [ self... >>>> BlockClosure>>on:do: >>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activate >>>> PharoCommandLineHandler>>activate >>>> PharoCommandLineHandler class(CommandLineHandler class)>>activateWith: >>>> [ super activateWith: aCommandLine ] in PharoCommandLineHandler >>>> class>>activateWith: in Block: [ super activateWith: aCommandLine ] >>>> >>>> Any idea? >>>> >>>> thanks! >>>> Esteban >>> >>> >>> >>> >>> -- >>> _,,,^..^,,,_ >>> best, Eliot >>> >>> >> >> > > > -- > _,,,^..^,,,_ > best, Eliot > > -- Best regards, Igor Stasenko.

March 15, 2017

Re: [Pharo-dev] PharoSpur32Vm

by Nicolai Hess

2017-03-15 9:22 GMT+01:00 philippe.back(a)highoctane.be < philippe.back(a)gmail.com>: > I made my own build here. > Not up to date with latest stuff but should work for the build process. > > https://ci.appveyor.com/project/philippeback/pharo-vm > > It uses my forked repo and provided you set your own bintray env vars for > API keys will publish there. > > Check all of the output of env vars and where/which in the appveyor > console to see what gets used when when it comes to compilers and so on as > there were various compiler versions involved at one point. > > Third party cache part is also worth checking. > > Still not able to build on my local box at the moment due to some tools > discrepancies happening. > > My build artifacts are embarking too much at this point but allow you ro > get the release, debug, and assert vms for windows. This helps when > debugging as backtraces and so on are much more meaningful and one can use > gdb more effectively to understand what is going on. > > Just keep the dlls and exes and you'll be fine. > Ah good, thank you. > > HTH > > Phil > > Le 15 mars 2017 08:58, "Nicolai Hess" <nicolaihess(a)gmail.com> a Ã©crit : > >> >> >> 2017-03-14 22:22 GMT+01:00 Nicolas Cellier <nicolas.cellier.aka.nice@gmai >> l.com>: >> >>> >>> >>> 2017-03-14 9:30 GMT+01:00 Nicolas Cellier <nicolas.cellier.aka.nice@gmai >>> l.com>: >>> >>>> >>>> >>>> 2017-03-14 8:58 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>> >>>>> >>>>> >>>>> 2017-03-11 10:01 GMT+01:00 Nicolas Cellier < >>>>> nicolas.cellier.aka.nice(a)gmail.com>: >>>>> >>>>>> Hi Nicolai, >>>>>> >>>>>> If you look at appveyor.yml configuration on >>>>>> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/. >>>>>> appveyor.yml, you will see that the pharo brand is not yet in the >>>>>> matrix. >>>>>> >>>>>> It's because builds are based on cygwin, but as I understood, some >>>>>> library used by some plugin required by pharo refuse to compile with >>>>>> cygwin. They appear to work with mingw32. >>>>>> Cygwin provides headers for legacy directx, some distribution of >>>>>> mingw32 did in the past, but it does not seem the case anymore. >>>>>> Using the directx headers from Microsoft SDK is a problem. They are >>>>>> not redistributable and can't be found anymore on the net (too old). We >>>>>> cannot seriously base the builds on something so fragile (both technically >>>>>> and legally) in the long term. >>>>>> >>>>>> Also, the 64 bits VM does only work with clang, and we don't have >>>>>> anything available as a 64bits Microsoft SDK... So pharo has to fix that >>>>>> too. >>>>>> >>>>>> In the interim, you should look at https://github.com/pharo-proje >>>>>> ct/pharo-vm/blob/master/.appveyor.yml and follow the scripts there. >>>>>> >>>>> >>>>> Ok, thank you. >>>>> >>>>> >>>> >>>> I gave it a shot on sunday, because it was particularly rainy in >>>> Nantes, and I almost succeeded in compiling all the dependencies with >>>> cygwin. >>>> Well, I mean with autotools cmake libtool pkg-config and I surely >>>> forget a few other niceties that some not so well informed programmers >>>> committed with the faith that it would make their life "easier". It >>>> certainly does not make mine simpler... >>>> Almost, because gcc 5.4.0 failed to compile cairo with ssize_t: it >>>> seems that the workaround of Igor does not work anymore. >>>> ssize_t, WTF??? >>>> Maybe I'll be able to fix it tonight. Or tomorrow. In which case I'll >>>> publish the branch to see how far appveyor goes. >>>> >>>> >>>> >>> >>> So I solved the ssize_t problem by removing the hack from Igor which is >>> not necessary anymore... >>> But got another problem soon after while building the tests... >>> There are trailing lines generated at end of >>> tests/cairo-test-constructors.c that make the compilation fail: >>> >>> i686-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I.. -I. -I./pdiff >>> -I../boilerplate -I../util/cairo-missing -I../util/cairo-script -I../src >>> -I../src -D_REENTRANT -I/cygdrive/y/Smalltalk/opensm >>> alltalk-vm/.thirdparty-cache/windows/i386/include/pixman-1 >>> -I/cygdrive/y/Smalltalk/opensmalltalk-vm/.thirdparty-cache/w >>> indows/i386/include/libpng16 -Wall -Wextra -Wmissing-declarations >>> -Werror-implicit-function-declaration -Wpointer-arith -Wwrite-strings >>> -Wsign-compare -Wpacked -Wswitch-enum -Wmissing-format-attribute >>> -Wvolatile-register-var -Wstrict-aliasing=2 -Winit-self >>> -Wunsafe-loop-optimizations -Wno-missing-field-initializers >>> -Wno-unused-parameter -Wno-attributes -Wno-long-long -Winline >>> -fno-strict-aliasing -fno-common -Wp,-D_FORTIFY_SOURCE=2 >>> -Wno-unused-but-set-variable -D_REENTRANT -m32 >>> -static-libgcc -static-libstdc++ -I/cygdrive/y/Smalltalk/opensm >>> alltalk-vm/.thirdparty-cache/windows/i386/include -march=pentium4 -c -o >>> cairo_test_suite-cairo-test-constructors.o `test -f >>> 'cairo-test-constructors.c' || echo './'`cairo-test-constructors.c >>> cairo-test-constructors.c:1118:1: attention : la dÃ©finition de donnÃ©es >>> n'a pas de type ni de classe de stockage >>> oning (); >>> ^ >>> cairo-test-constructors.c:1118:1: attention : type defaults to âintâ in >>> declaration of âoningâ [-Wimplicit-int] >>> cairo-test-constructors.c:1119:5: attention : la dÃ©finition de donnÃ©es >>> n'a pas de type ni de classe de stockage >>> _register_ft_show_glyphs_table (); >>> ^ >>> >>> And the file looks like it has obviously been overwritten, but not >>> truncated !!! >>> >>> ... >>> extern void _register_multi_page (void); >>> extern void _register_fallback_resolution (void); >>> >>> void >>> _cairo_test_runner_register_tests (void) >>> { >>> _register_a1_bug (); >>> _register_a1_clip_paint (); >>> ... >>> _register_multi_page (); >>> _register_fallback_resolution (); >>> } >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> *oning (); _register_ft_show_glyphs_table (); >>> _register_ft_text_vertical_layout_type1 (); >>> _register_ft_text_vertical_layout_type3 (); >>> _register_ft_text_antialias_none (); _register_ps_eps (); >>> _register_ps_features (); _register_ps_surface_source (); >>> _register_svg_surface (); _register_svg_clip (); >>> _register_svg_surface_source (); _register_multi_page (); >>> _register_fallback_resolution ();}* >>> >>> This file is generated by a shell script test/make-cairo-test-construct >>> ors.sh >>> I can't find any reference of the bug, and upgrade to version 1.14.8 >>> does not solve the issue. >>> So it will wait until tomorrow... >>> >> >> >> I got the build for windows with mingw nearly working. (it can not build >> some plugins, like SqueakSSL for windows, because the used wincrypt.h is >> different in the mingw distrubtion). >> >> I still have the problem, that there seems to be a preprocessing step , >> that should put the vm-version (and source timestamp) in the >> *sqSCCSVersion.h* >> I got this working by running .travis_build.sh (with the options for >> arch/flavor/platform) But how is this done normally when you build a vm >> locally? >> And how is this done for the pharo-vm we currently use? >> >> >> >>> >>> Nicolas >>> >>> >>>>>> I hope that Esteban will find time to resolve all these problems and >>>>>> have pharo brand back on opensmalltalk-vm. I guess that any form of help is >>>>>> welcome. >>>>>> >>>>>> Nicolas >>>>>> >>>>>> >>>>>> 2017-03-11 8:33 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>> >>>>>>> I still have problems building a vm on windows. >>>>>>> can you give me some hints how to start ? >>>>>>> I cloned the recent pharo-vm project, >>>>>>> in opensmalltalk-vm\build.win32x86\pharo.cog.spur\ >>>>>>> run >>>>>>> mvm >>>>>>> >>>>>>> But I got a couple of problems (mingw-32 compiler commands not >>>>>>> found, I had to adjust the include path for finding directx header, missing >>>>>>> variable replacement for git-versions). >>>>>>> So I may miss some important steps. >>>>>>> Is there a repository where I can clone the mingw environment used >>>>>>> to build the win32-pharo-vm, the environment used on the build-server ? >>>>>>> >>>>>>> Thanks >>>>>>> Nicolai >>>>>>> >>>>>>> 2017-02-04 1:50 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2017-02-04 1:44 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2017-01-23 8:59 GMT+01:00 Esteban Lorenzano <estebanlm(a)gmail.com>: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 22 Jan 2017, at 13:19, Nicolai Hess <nicolaihess(a)gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2017-01-22 10:21 GMT+01:00 ClÃ©ment Bera <bera.clement(a)gmail.com>: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I believe they're built from* https://github.com/OpenSmalltalk/vm >>>>>>>>>>> <https://github.com/OpenSmalltalk/vm>* using travis and >>>>>>>>>>> appveyor. On the gitbhub readme there are relevant links. All built >>>>>>>>>>> artifacts are also kept on bintray for history. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Thank you! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> no, they arenât :) >>>>>>>>>> instead, they are built here: https://github.com/pharo >>>>>>>>>> -project/pharo-vm >>>>>>>>>> >>>>>>>>>> (README still not updated) >>>>>>>>>> >>>>>>>>> >>>>>>>>> what did changed ? I am not able to build the vm on windows >>>>>>>>> anymore (something wrong with generating the generator.image, I'll now >>>>>>>>> reset my local pharo-vm build directory and see if it works afterwards). >>>>>>>>> >>>>>>>> >>>>>>>> see attached the stderr log : >>>>>>>> >>>>>>>> 'Errors in script loaded from u:\github\pharo-vm\scripts\Loa >>>>>>>> dVMMaker.st' >>>>>>>> [31mMessageNotUnderstood: receiver of "default:" is nil >>>>>>>> [0mUndefinedObject(Object)>>doesNotUnderstand: #default: >>>>>>>> BaseSoundSystem class>>initialize >>>>>>>> MCMethodDefinition>>postloadOver: >>>>>>>> >>>>>>>> .... >>>>>>>> >>>>>>>> >>>>>>>>> Are we still working with branch spur-64, or are we back on master >>>>>>>>> ? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> Esteban >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Sun, Jan 22, 2017 at 9:25 AM, Nicolai Hess < >>>>>>>>>>> nicolaihess(a)gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> Where are the latest Pharo-spur-vms (32bit) are built? >>>>>>>>>>>> I don't see them on the build server, only the buildresults at >>>>>>>>>>>> http://files.pharo.org/vm/pharo-spur32/linux/ >>>>>>>>>>>> >>>>>>>>>>>> The latest builds on the buildserver are from the last year >>>>>>>>>>>> only. >>>>>>>>>>>> >>>>>>>>>>>> nicolai >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>

March 15, 2017

Re: [Pharo-dev] Type Inference tools

by Peter Uhnak

Moose ( http://www.moosetechnology.org/ ) optionally uses RoelTyper when importing Smalltalk code for analysis. Lukas will need a good type inferencer as he is currently working on a better autocompetion. Peter On Tue, Mar 14, 2017 at 03:38:51PM -0700, Gus wrote: > Hi, > I am working on a project of type inference, and i have the following doubt > Are there currently any tools in pharo that use this information (perhaps in > any library)? > For example: a better autocomplete where you can chose different types, or > tool that show you the type of a variable. > I've already downloaded Roel Typer but i couldn't find any of those tools. > Any help would be appreciated. > Thanks > > > > -- > View this message in context: http://forum.world.st/Type-Inference-tools-tp4938677.html > Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >

March 15, 2017

Re: [Pharo-dev] PharoSpur32Vm

by philippe.back＠highoctane.be

I made my own build here. Not up to date with latest stuff but should work for the build process. https://ci.appveyor.com/project/philippeback/pharo-vm It uses my forked repo and provided you set your own bintray env vars for API keys will publish there. Check all of the output of env vars and where/which in the appveyor console to see what gets used when when it comes to compilers and so on as there were various compiler versions involved at one point. Third party cache part is also worth checking. Still not able to build on my local box at the moment due to some tools discrepancies happening. My build artifacts are embarking too much at this point but allow you ro get the release, debug, and assert vms for windows. This helps when debugging as backtraces and so on are much more meaningful and one can use gdb more effectively to understand what is going on. Just keep the dlls and exes and you'll be fine. HTH Phil Le 15 mars 2017 08:58, "Nicolai Hess" <nicolaihess(a)gmail.com> a Ã©crit : > > > 2017-03-14 22:22 GMT+01:00 Nicolas Cellier <nicolas.cellier.aka.nice@ > gmail.com>: > >> >> >> 2017-03-14 9:30 GMT+01:00 Nicolas Cellier <nicolas.cellier.aka.nice@gmai >> l.com>: >> >>> >>> >>> 2017-03-14 8:58 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>> >>>> >>>> >>>> 2017-03-11 10:01 GMT+01:00 Nicolas Cellier < >>>> nicolas.cellier.aka.nice(a)gmail.com>: >>>> >>>>> Hi Nicolai, >>>>> >>>>> If you look at appveyor.yml configuration on >>>>> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/. >>>>> appveyor.yml, you will see that the pharo brand is not yet in the >>>>> matrix. >>>>> >>>>> It's because builds are based on cygwin, but as I understood, some >>>>> library used by some plugin required by pharo refuse to compile with >>>>> cygwin. They appear to work with mingw32. >>>>> Cygwin provides headers for legacy directx, some distribution of >>>>> mingw32 did in the past, but it does not seem the case anymore. >>>>> Using the directx headers from Microsoft SDK is a problem. They are >>>>> not redistributable and can't be found anymore on the net (too old). We >>>>> cannot seriously base the builds on something so fragile (both technically >>>>> and legally) in the long term. >>>>> >>>>> Also, the 64 bits VM does only work with clang, and we don't have >>>>> anything available as a 64bits Microsoft SDK... So pharo has to fix that >>>>> too. >>>>> >>>>> In the interim, you should look at https://github.com/pharo-proje >>>>> ct/pharo-vm/blob/master/.appveyor.yml and follow the scripts there. >>>>> >>>> >>>> Ok, thank you. >>>> >>>> >>> >>> I gave it a shot on sunday, because it was particularly rainy in Nantes, >>> and I almost succeeded in compiling all the dependencies with cygwin. >>> Well, I mean with autotools cmake libtool pkg-config and I surely forget >>> a few other niceties that some not so well informed programmers committed >>> with the faith that it would make their life "easier". It certainly does >>> not make mine simpler... >>> Almost, because gcc 5.4.0 failed to compile cairo with ssize_t: it seems >>> that the workaround of Igor does not work anymore. >>> ssize_t, WTF??? >>> Maybe I'll be able to fix it tonight. Or tomorrow. In which case I'll >>> publish the branch to see how far appveyor goes. >>> >>> >>> >> >> So I solved the ssize_t problem by removing the hack from Igor which is >> not necessary anymore... >> But got another problem soon after while building the tests... >> There are trailing lines generated at end of >> tests/cairo-test-constructors.c that make the compilation fail: >> >> i686-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I.. -I. -I./pdiff >> -I../boilerplate -I../util/cairo-missing -I../util/cairo-script -I../src >> -I../src -D_REENTRANT -I/cygdrive/y/Smalltalk/opensm >> alltalk-vm/.thirdparty-cache/windows/i386/include/pixman-1 >> -I/cygdrive/y/Smalltalk/opensmalltalk-vm/.thirdparty-cache/ >> windows/i386/include/libpng16 -Wall -Wextra -Wmissing-declarations >> -Werror-implicit-function-declaration -Wpointer-arith -Wwrite-strings >> -Wsign-compare -Wpacked -Wswitch-enum -Wmissing-format-attribute >> -Wvolatile-register-var -Wstrict-aliasing=2 -Winit-self >> -Wunsafe-loop-optimizations -Wno-missing-field-initializers >> -Wno-unused-parameter -Wno-attributes -Wno-long-long -Winline >> -fno-strict-aliasing -fno-common -Wp,-D_FORTIFY_SOURCE=2 >> -Wno-unused-but-set-variable -D_REENTRANT -m32 >> -static-libgcc -static-libstdc++ -I/cygdrive/y/Smalltalk/opensm >> alltalk-vm/.thirdparty-cache/windows/i386/include -march=pentium4 -c -o >> cairo_test_suite-cairo-test-constructors.o `test -f >> 'cairo-test-constructors.c' || echo './'`cairo-test-constructors.c >> cairo-test-constructors.c:1118:1: attention : la dÃ©finition de donnÃ©es >> n'a pas de type ni de classe de stockage >> oning (); >> ^ >> cairo-test-constructors.c:1118:1: attention : type defaults to âintâ in >> declaration of âoningâ [-Wimplicit-int] >> cairo-test-constructors.c:1119:5: attention : la dÃ©finition de donnÃ©es >> n'a pas de type ni de classe de stockage >> _register_ft_show_glyphs_table (); >> ^ >> >> And the file looks like it has obviously been overwritten, but not >> truncated !!! >> >> ... >> extern void _register_multi_page (void); >> extern void _register_fallback_resolution (void); >> >> void >> _cairo_test_runner_register_tests (void) >> { >> _register_a1_bug (); >> _register_a1_clip_paint (); >> ... >> _register_multi_page (); >> _register_fallback_resolution (); >> } >> >> >> >> >> >> >> >> >> >> >> >> >> >> *oning (); _register_ft_show_glyphs_table (); >> _register_ft_text_vertical_layout_type1 (); >> _register_ft_text_vertical_layout_type3 (); >> _register_ft_text_antialias_none (); _register_ps_eps (); >> _register_ps_features (); _register_ps_surface_source (); >> _register_svg_surface (); _register_svg_clip (); >> _register_svg_surface_source (); _register_multi_page (); >> _register_fallback_resolution ();}* >> >> This file is generated by a shell script test/make-cairo-test-construct >> ors.sh >> I can't find any reference of the bug, and upgrade to version 1.14.8 does >> not solve the issue. >> So it will wait until tomorrow... >> > > > I got the build for windows with mingw nearly working. (it can not build > some plugins, like SqueakSSL for windows, because the used wincrypt.h is > different in the mingw distrubtion). > > I still have the problem, that there seems to be a preprocessing step , > that should put the vm-version (and source timestamp) in the > *sqSCCSVersion.h* > I got this working by running .travis_build.sh (with the options for > arch/flavor/platform) But how is this done normally when you build a vm > locally? > And how is this done for the pharo-vm we currently use? > > > >> >> Nicolas >> >> >>>>> I hope that Esteban will find time to resolve all these problems and >>>>> have pharo brand back on opensmalltalk-vm. I guess that any form of help is >>>>> welcome. >>>>> >>>>> Nicolas >>>>> >>>>> >>>>> 2017-03-11 8:33 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>> >>>>>> I still have problems building a vm on windows. >>>>>> can you give me some hints how to start ? >>>>>> I cloned the recent pharo-vm project, >>>>>> in opensmalltalk-vm\build.win32x86\pharo.cog.spur\ >>>>>> run >>>>>> mvm >>>>>> >>>>>> But I got a couple of problems (mingw-32 compiler commands not found, >>>>>> I had to adjust the include path for finding directx header, missing >>>>>> variable replacement for git-versions). >>>>>> So I may miss some important steps. >>>>>> Is there a repository where I can clone the mingw environment used to >>>>>> build the win32-pharo-vm, the environment used on the build-server ? >>>>>> >>>>>> Thanks >>>>>> Nicolai >>>>>> >>>>>> 2017-02-04 1:50 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>> >>>>>>> >>>>>>> >>>>>>> 2017-02-04 1:44 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2017-01-23 8:59 GMT+01:00 Esteban Lorenzano <estebanlm(a)gmail.com>: >>>>>>>> >>>>>>>>> >>>>>>>>> On 22 Jan 2017, at 13:19, Nicolai Hess <nicolaihess(a)gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> 2017-01-22 10:21 GMT+01:00 ClÃ©ment Bera <bera.clement(a)gmail.com>: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> I believe they're built from* https://github.com/OpenSmalltalk/vm >>>>>>>>>> <https://github.com/OpenSmalltalk/vm>* using travis and >>>>>>>>>> appveyor. On the gitbhub readme there are relevant links. All built >>>>>>>>>> artifacts are also kept on bintray for history. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Thank you! >>>>>>>>> >>>>>>>>> >>>>>>>>> no, they arenât :) >>>>>>>>> instead, they are built here: https://github.com/pharo >>>>>>>>> -project/pharo-vm >>>>>>>>> >>>>>>>>> (README still not updated) >>>>>>>>> >>>>>>>> >>>>>>>> what did changed ? I am not able to build the vm on windows anymore >>>>>>>> (something wrong with generating the generator.image, I'll now reset my >>>>>>>> local pharo-vm build directory and see if it works afterwards). >>>>>>>> >>>>>>> >>>>>>> see attached the stderr log : >>>>>>> >>>>>>> 'Errors in script loaded from u:\github\pharo-vm\scripts\Loa >>>>>>> dVMMaker.st' >>>>>>> [31mMessageNotUnderstood: receiver of "default:" is nil >>>>>>> [0mUndefinedObject(Object)>>doesNotUnderstand: #default: >>>>>>> BaseSoundSystem class>>initialize >>>>>>> MCMethodDefinition>>postloadOver: >>>>>>> >>>>>>> .... >>>>>>> >>>>>>> >>>>>>>> Are we still working with branch spur-64, or are we back on master ? >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> Esteban >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Sun, Jan 22, 2017 at 9:25 AM, Nicolai Hess < >>>>>>>>>> nicolaihess(a)gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Where are the latest Pharo-spur-vms (32bit) are built? >>>>>>>>>>> I don't see them on the build server, only the buildresults at >>>>>>>>>>> http://files.pharo.org/vm/pharo-spur32/linux/ >>>>>>>>>>> >>>>>>>>>>> The latest builds on the buildserver are from the last year only. >>>>>>>>>>> >>>>>>>>>>> nicolai >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >

March 15, 2017

Re: [Pharo-dev] PharoSpur32Vm

by Nicolai Hess

2017-03-14 22:22 GMT+01:00 Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com>: > > > 2017-03-14 9:30 GMT+01:00 Nicolas Cellier <nicolas.cellier.aka.nice@ > gmail.com>: > >> >> >> 2017-03-14 8:58 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >> >>> >>> >>> 2017-03-11 10:01 GMT+01:00 Nicolas Cellier < >>> nicolas.cellier.aka.nice(a)gmail.com>: >>> >>>> Hi Nicolai, >>>> >>>> If you look at appveyor.yml configuration on >>>> https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/. >>>> appveyor.yml, you will see that the pharo brand is not yet in the >>>> matrix. >>>> >>>> It's because builds are based on cygwin, but as I understood, some >>>> library used by some plugin required by pharo refuse to compile with >>>> cygwin. They appear to work with mingw32. >>>> Cygwin provides headers for legacy directx, some distribution of >>>> mingw32 did in the past, but it does not seem the case anymore. >>>> Using the directx headers from Microsoft SDK is a problem. They are not >>>> redistributable and can't be found anymore on the net (too old). We cannot >>>> seriously base the builds on something so fragile (both technically and >>>> legally) in the long term. >>>> >>>> Also, the 64 bits VM does only work with clang, and we don't have >>>> anything available as a 64bits Microsoft SDK... So pharo has to fix that >>>> too. >>>> >>>> In the interim, you should look at https://github.com/pharo-proje >>>> ct/pharo-vm/blob/master/.appveyor.yml and follow the scripts there. >>>> >>> >>> Ok, thank you. >>> >>> >> >> I gave it a shot on sunday, because it was particularly rainy in Nantes, >> and I almost succeeded in compiling all the dependencies with cygwin. >> Well, I mean with autotools cmake libtool pkg-config and I surely forget >> a few other niceties that some not so well informed programmers committed >> with the faith that it would make their life "easier". It certainly does >> not make mine simpler... >> Almost, because gcc 5.4.0 failed to compile cairo with ssize_t: it seems >> that the workaround of Igor does not work anymore. >> ssize_t, WTF??? >> Maybe I'll be able to fix it tonight. Or tomorrow. In which case I'll >> publish the branch to see how far appveyor goes. >> >> >> > > So I solved the ssize_t problem by removing the hack from Igor which is > not necessary anymore... > But got another problem soon after while building the tests... > There are trailing lines generated at end of tests/cairo-test-constructors.c > that make the compilation fail: > > i686-w64-mingw32-gcc -DHAVE_CONFIG_H -I. -I.. -I. -I./pdiff > -I../boilerplate -I../util/cairo-missing -I../util/cairo-script -I../src > -I../src -D_REENTRANT -I/cygdrive/y/Smalltalk/ > opensmalltalk-vm/.thirdparty-cache/windows/i386/include/pixman-1 > -I/cygdrive/y/Smalltalk/opensmalltalk-vm/.thirdparty- > cache/windows/i386/include/libpng16 -Wall -Wextra > -Wmissing-declarations -Werror-implicit-function-declaration > -Wpointer-arith -Wwrite-strings -Wsign-compare -Wpacked -Wswitch-enum > -Wmissing-format-attribute -Wvolatile-register-var -Wstrict-aliasing=2 > -Winit-self -Wunsafe-loop-optimizations -Wno-missing-field-initializers > -Wno-unused-parameter -Wno-attributes -Wno-long-long -Winline > -fno-strict-aliasing -fno-common -Wp,-D_FORTIFY_SOURCE=2 > -Wno-unused-but-set-variable -D_REENTRANT -m32 > -static-libgcc -static-libstdc++ -I/cygdrive/y/Smalltalk/ > opensmalltalk-vm/.thirdparty-cache/windows/i386/include -march=pentium4 > -c -o cairo_test_suite-cairo-test-constructors.o `test -f > 'cairo-test-constructors.c' || echo './'`cairo-test-constructors.c > cairo-test-constructors.c:1118:1: attention : la dÃ©finition de donnÃ©es > n'a pas de type ni de classe de stockage > oning (); > ^ > cairo-test-constructors.c:1118:1: attention : type defaults to âintâ in > declaration of âoningâ [-Wimplicit-int] > cairo-test-constructors.c:1119:5: attention : la dÃ©finition de donnÃ©es > n'a pas de type ni de classe de stockage > _register_ft_show_glyphs_table (); > ^ > > And the file looks like it has obviously been overwritten, but not > truncated !!! > > ... > extern void _register_multi_page (void); > extern void _register_fallback_resolution (void); > > void > _cairo_test_runner_register_tests (void) > { > _register_a1_bug (); > _register_a1_clip_paint (); > ... > _register_multi_page (); > _register_fallback_resolution (); > } > > > > > > > > > > > > > > *oning (); _register_ft_show_glyphs_table (); > _register_ft_text_vertical_layout_type1 (); > _register_ft_text_vertical_layout_type3 (); > _register_ft_text_antialias_none (); _register_ps_eps (); > _register_ps_features (); _register_ps_surface_source (); > _register_svg_surface (); _register_svg_clip (); > _register_svg_surface_source (); _register_multi_page (); > _register_fallback_resolution ();}* > > This file is generated by a shell script test/make-cairo-test- > constructors.sh > I can't find any reference of the bug, and upgrade to version 1.14.8 does > not solve the issue. > So it will wait until tomorrow... > I got the build for windows with mingw nearly working. (it can not build some plugins, like SqueakSSL for windows, because the used wincrypt.h is different in the mingw distrubtion). I still have the problem, that there seems to be a preprocessing step , that should put the vm-version (and source timestamp) in the *sqSCCSVersion.h* I got this working by running .travis_build.sh (with the options for arch/flavor/platform) But how is this done normally when you build a vm locally? And how is this done for the pharo-vm we currently use? > > Nicolas > > >>>> I hope that Esteban will find time to resolve all these problems and >>>> have pharo brand back on opensmalltalk-vm. I guess that any form of help is >>>> welcome. >>>> >>>> Nicolas >>>> >>>> >>>> 2017-03-11 8:33 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>> >>>>> I still have problems building a vm on windows. >>>>> can you give me some hints how to start ? >>>>> I cloned the recent pharo-vm project, >>>>> in opensmalltalk-vm\build.win32x86\pharo.cog.spur\ >>>>> run >>>>> mvm >>>>> >>>>> But I got a couple of problems (mingw-32 compiler commands not found, >>>>> I had to adjust the include path for finding directx header, missing >>>>> variable replacement for git-versions). >>>>> So I may miss some important steps. >>>>> Is there a repository where I can clone the mingw environment used to >>>>> build the win32-pharo-vm, the environment used on the build-server ? >>>>> >>>>> Thanks >>>>> Nicolai >>>>> >>>>> 2017-02-04 1:50 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>> >>>>>> >>>>>> >>>>>> 2017-02-04 1:44 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: >>>>>> >>>>>>> >>>>>>> >>>>>>> 2017-01-23 8:59 GMT+01:00 Esteban Lorenzano <estebanlm(a)gmail.com>: >>>>>>> >>>>>>>> >>>>>>>> On 22 Jan 2017, at 13:19, Nicolai Hess <nicolaihess(a)gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> 2017-01-22 10:21 GMT+01:00 ClÃ©ment Bera <bera.clement(a)gmail.com>: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I believe they're built from* https://github.com/OpenSmalltalk/vm >>>>>>>>> <https://github.com/OpenSmalltalk/vm>* using travis and appveyor. >>>>>>>>> On the gitbhub readme there are relevant links. All built artifacts are >>>>>>>>> also kept on bintray for history. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> Thank you! >>>>>>>> >>>>>>>> >>>>>>>> no, they arenât :) >>>>>>>> instead, they are built here: https://github.com/pharo >>>>>>>> -project/pharo-vm >>>>>>>> >>>>>>>> (README still not updated) >>>>>>>> >>>>>>> >>>>>>> what did changed ? I am not able to build the vm on windows anymore >>>>>>> (something wrong with generating the generator.image, I'll now reset my >>>>>>> local pharo-vm build directory and see if it works afterwards). >>>>>>> >>>>>> >>>>>> see attached the stderr log : >>>>>> >>>>>> 'Errors in script loaded from u:\github\pharo-vm\scripts\Loa >>>>>> dVMMaker.st' >>>>>> [31mMessageNotUnderstood: receiver of "default:" is nil >>>>>> [0mUndefinedObject(Object)>>doesNotUnderstand: #default: >>>>>> BaseSoundSystem class>>initialize >>>>>> MCMethodDefinition>>postloadOver: >>>>>> >>>>>> .... >>>>>> >>>>>> >>>>>>> Are we still working with branch spur-64, or are we back on master ? >>>>>>> >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> Esteban >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> On Sun, Jan 22, 2017 at 9:25 AM, Nicolai Hess < >>>>>>>>> nicolaihess(a)gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Where are the latest Pharo-spur-vms (32bit) are built? >>>>>>>>>> I don't see them on the build server, only the buildresults at >>>>>>>>>> http://files.pharo.org/vm/pharo-spur32/linux/ >>>>>>>>>> >>>>>>>>>> The latest builds on the buildserver are from the last year only. >>>>>>>>>> >>>>>>>>>> nicolai >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >

March 15, 2017

Re: [Pharo-dev] [Vm-dev] BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)

by phil＠highoctane.be

Le 14 mars 2017 17:34, "Eliot Miranda" <eliot.miranda(a)gmail.com> a Ã©crit : On Tue, Mar 14, 2017 at 8:56 AM, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > > > 2017-03-14 16:46 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > >> >> Hi Esteban, Hi Igor, Hi All, >> >> On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <estebanlm(a)gmail.com> >> wrote: >> >>> >>> Hi, >>> >>> Iâm tumbling into an error in Pharo, because we use callbacks >>> intensively, in Athens(cairo)-to-World conversion in particular, and people >>> is sending always their crash reportsâ¦ we made the whole conversion a lot >>> more robust since problems started to arise, but now I hit a wall I cannot >>> solve: I think problem is in something in callbacks. >>> >>> And problem is showing very easy on 64bits (while in 32bits it takes >>> time and is more random). >>> >> >> I responded in the "image not opening" thread, but it's the same >> problem. I really want to hear what y'all think because I'll happily >> implement a fix, but I want to know which one y'all think is a good idea. >> Here's my reply (edits between [ & ] to add information): >> >> >> On Mon, Mar 13, 2017 at 9:11 PM, Eliot Miranda <eliot.miranda(a)gmail.com> >> wrote: >> >> I'm pretty confident [I know] this is to do with bugs in the Athens >> surface code which assumes that callbacks can be made in the existing >> copyBits and warpBits primitive. They can't do this safely because a GC >> (scavenge) can happen during a callback, which then causes chaos when the >> copyBits primitive tries to access objects that have been moved under its >> feet. >> >> I've done work to fix callbacks so that when there is a failure it is the >> copyBits primitive that fails, instead of apparently the callback return >> primitive. One of the apparent effects of this fix is to stop the screen >> opening up too small; another is getting the background colour right, and >> yet another is eliminating bogus pixels in the VGTigerDemo demo. But more >> work is required to fix the copyBits and warpBits primitives. There are a >> few approaches one might take: >> >> a) fixing the primitive so that it saves and restores oops around the >> callbacks using the external oop table [InterpreterProxy>>addGCRoot: & >> removeGCRoot:]. That's a pain but possible. [It's a pain because all the >> derived pointers (the start of the destForm, sourceForm, halftoneForm and >> colorMapTable) must be recomputed also, and of course most of the time the >> objects don't move; we only scavenge about once every 2 seconds in normal >> running] >> >> b) fixing the primitive so that it pins the objects it needs before ever >> invoking a callback [this is a pain because pinning an object causes it to >> be tenured to old space if it is in new space; objects can't be pinned in >> new space, so instead the pin operation forwards the new space object to an >> old space copy if required and answers its location in old space, so a >> putative withPinnedObjectsDo: operation for the copyBits primitive looks >> like >> withPinnedFormsDo: aBlock >> <inline: #always> >> self cppIf: SPURVM & false >> ifTrue: >> [| bitBltOopWasPinned destWasPinned sourceWasPinned halftoneWasPinned | >> (bitBltOopWasPinned := interpreterProxy isPinned: bitBltOop) ifFalse: >> [bitBltOop := interpreterProxy pinObject: bitBltOop]. >> (destWasPinned := interpreterProxy isPinned: destForm) ifFalse: >> [destForm := interpreterProxy pinObject: destForm]. >> (sourceWasPinned := interpreterProxy isPinned: sourceForm) ifFalse: >> [sourceForm := interpreterProxy pinObject: sourceForm]. >> (halftoneWasPinned := interpreterProxy isPinned: halftoneForm) ifFalse: >> [halftoneForm := interpreterProxy pinObject: halftoneForm]. >> aBlock value. >> bitBltOopWasPinned ifFalse: [interpreterProxy unpinObject: bitBltOop]. >> destWasPinned ifFalse: [interpreterProxy unpinObject: destForm]. >> sourceWasPinned ifFalse: [interpreterProxy unpinObject: sourceForm]. >> halftoneWasPinned ifFalse: [interpreterProxy unpinObject: halftoneForm]] >> ifFalse: [aBlock value] >> and tenuring objects to old space is not ideal because they are only >> collected by a full GC, so doing this would at least tenure the bitBltOop >> which is very likely to be in new space] >> >> c) fixing the primitive so that it uses the scavenge and fullGC counters >> in the VM to detect if a GC occurred during one of the callbacks and would >> fail the primitive [if it detected that a GC had occurred in any of the >> surface functions]. The primitive would then simply be retried. >> >> d) ? >> > > Wouldn't it be possible to just pause the GC (scavange) when entering a > primitive ? > I don't think so. There is a callback occurring. If the computation executed by the callback requires a GC the application will abort if a GC cannot be done. Right? This is the case here. I like c) as it's very lightweight, but it has issues. It is fine to use >> for callbacks *before* cop[yBits and warpBits move any bits (the >> lockSurface and querySurface functions). But it's potentially erroneous >> after the unlockSurface primitive. For example, a primitive which does an >> xor with the screen can't simply be retried as the first, falling pass, >> would have updated the destination bits but not displayed them via >> unlockSurface. But I think it could be arranged that no objects are >> accessed after unlockSurface, which should naturally be the last call in >> the primitive (or do I mean showSurface?). So the approach would be to >> check for GCs occurring during querySurface and lockSurface, failing if so, >> and then caching any and all state needed by unlockSurface and showSurface >> in local variables. This way no object state is accessed to make the >> unlockSurface and showSurface calls, and no bits are moved before the >> queryDurface and lockSurface calls. >> >> If we used a failure code such as #'object may move' then the primitives >> could answer this when a GC during callbacks is detected and then the >> primitive could be retried only when required. >> >> >> [Come on folks, please comment. I want to know which idea you like >> best. We could fix this quickly. But right now it feels like I'm talking >> to myself.] >> > There is also the stacked callbacks story. So a callback is triggered from a callback etc. I think that pinning the objects one knows shouldn't move is the best "control freak" version. At least when something fails one knows why. Not callback related but the FT2Handle bug comes to mind... Maybe this moves such objects to old space today. But it doesn't mean that this is the end of it GC wise. Why not have a special space for pinned objects? After all they aren't old nor new. Some external code needs them so, do not touch unless so told. Ending up in half baked state like in the mentioned Display example is really bad for having confidencd in the system. With UFFI and callbacks we have a very powerful mechanism to take in lots of interesting features. As a library wrapping developer I want to know if the thing bombs because of me or because of the external library and exclude the VM code from that. Because I want to trust the VM and GC to do the right thing and not a "mostly works a lot of the times but sometimes I'll just blow in your face mate" version. If there are leaks I want to see pinned objects space grow. And then look into that with tools. And get it fixed. This concern is really a key to our future. That is in a sense what made nodejs so valued. Let's not make it crappy and focused on the Athens/Form story only. Let's solve it for good. Our future selves are going to thank us. Phil >> >>> Here is the easiest way to reproduce it (in mac): >>> >>> wget files.pharo.org/get-files/60/pharo64-mac-latest.zip >>> wget files.pharo.org/get-files/60/pharo64.zip >>> wget files.pharo.org/get-files/60/sources.zip >>> unzip pharo64-mac-latest.zip >>> unzip pharo64.zip >>> unzip sources.zip >>> ./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval >>> "VGTigerDemo runDemo" >>> >>> eventually (like 5-6 seconds after, if not immediately), you will have a >>> stack like this: >>> >>> SmallInteger(Object)>>primitiveFailed: >>> SmallInteger(Object)>>primitiveFailed >>> SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext: >>> GrafPort>>copyBits >>> GrafPort>>image:at:sourceRect:rule: >>> FormCanvas>>image:at:sourceRect:rule: >>> FormCanvas(Canvas)>>drawImage:at:sourceRect: >>> FormCanvas(Canvas)>>drawImage:at: >>> VGTigerDemo>>runDemo >>> VGTigerDemo class>>runDemo >>> UndefinedObject>>DoIt >>> OpalCompiler>>evaluate >>> OpalCompiler(AbstractCompiler)>>evaluate: >>> [ result := Smalltalk compiler evaluate: aStream. >>> self hasSessionChanged >>> ifFalse: [ self stdout >>> print: result; >>> lf ] ] in EvaluateCommandLineHandler>>evaluate: >>> in Block: [ result := Smalltalk compiler evaluate: aStream.... >>> BlockClosure>>on:do: >>> EvaluateCommandLineHandler>>evaluate: >>> EvaluateCommandLineHandler>>evaluateArguments >>> EvaluateCommandLineHandler>>activate >>> EvaluateCommandLineHandler class(CommandLineHandler class)>>activateWith: >>> [ aCommandLinehandler activateWith: commandLine ] in >>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>> in Block: [ aCommandLinehandler activateWith: commandLine ] >>> BlockClosure>>on:do: >>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand >>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument: >>> [ self >>> handleArgument: >>> (self arguments >>> ifEmpty: [ '' ] >>> ifNotEmpty: [ :arguments | arguments first ]) ] >>> in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in Block: >>> [ self... >>> BlockClosure>>on:do: >>> PharoCommandLineHandler(BasicCommandLineHandler)>>activate >>> PharoCommandLineHandler>>activate >>> PharoCommandLineHandler class(CommandLineHandler class)>>activateWith: >>> [ super activateWith: aCommandLine ] in PharoCommandLineHandler >>> class>>activateWith: in Block: [ super activateWith: aCommandLine ] >>> >>> Any idea? >>> >>> thanks! >>> Esteban >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> > > -- _,,,^..^,,,_ best, Eliot

March 15, 2017

Re: [Pharo-dev] [Moose-dev] Too frequent crashes :-(

by Ronie Salgado

Hi Alex, I gave the following script: Smalltalk compiler evaluate: ' > | surface | > surface := AthensCairoSurface extent: 640@480. > surface drawDuring: [ :canvas | > surface clear: Color blue. > ]. > > surface asForm asMorph openInWindow > ' > Notice that I am wrapping the script in a string that I am passing to the compiler, so that the workspace cannot retain any reference to the surface. Best regards, Ronie 2017-03-14 16:12 GMT-03:00 Alexandre Bergel <alexandre.bergel(a)me.com>: > Hi Ronie, > > Below you said: > > The problem with asForm, is that the form returned can be used after the > cairo surface which holds the pixels is garbage collected. > > > I cannot reproduce this case. You gave a script: > > surface := AthensCairoSurface extent: 640@480. > surface drawDuring: [ :canvas | > surface clear: Color blue. > ]. > > surface asForm asMorph openInWindow > > > I see a blue window. > > Cheers, > Alexandre > > > How about changing AthensCairoSurface >> asForm into the following > definition?: > asForm > > "create a form and copy an image data there" > | form | > self checkSession. > > self flush. > form := Form extent: (self width@self height) depth: 32. > form unhibernate. > LibC memCopy: self getDataPtr to: form bits size: self width*self > height*4. > ^ form > > This involves a whole copy, but it removes completely the dependency on > the surface plugin. > > If we want to keep using the surface plugin with Cairo, the old definition > of asForm still has a bug, which can be reproduced with the following > snippet: > > Smalltalk compiler evaluate: ' > | surface | > surface := AthensCairoSurface extent: 640@480. > surface drawDuring: [ :canvas | > surface clear: Color blue. > ]. > > surface asForm asMorph openInWindow > ' > > The problem with asForm, is that the form returned can be used after the > cairo surface which holds the pixels is garbage collected. > > >

March 15, 2017