November 2018 - Pharo-dev - lists.pharo.org

[Pharo 7.0-dev] Build #1387: 22665-NECContextTesttestReceiverTempVar-is-failing-in-64-bit-image

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1387 was: FAILURE. The Pull Request #1979 was integrated: "22665-NECContextTesttestReceiverTempVar-is-failing-in-64-bit-image" Pull request url: https://github.com/pharo-project/pharo/pull/1979 Issue Url: https://pharo.fogbugz.com/f/cases/22665 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

[Pharo 7.0-dev] Build #1386: 22664-many-Opal-tests-are-failing-on-64-bit-image

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1386 was: FAILURE. The Pull Request #1982 was integrated: "22664-many-Opal-tests-are-failing-on-64-bit-image" Pull request url: https://github.com/pharo-project/pharo/pull/1982 Issue Url: https://pharo.fogbugz.com/f/cases/22664 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

[Pharo 7.0-dev] Build #1385: 22662-IntervalTest-is-failing-on-64-bit-image

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1385 was: FAILURE. The Pull Request #1978 was integrated: "22662-IntervalTest-is-failing-on-64-bit-image" Pull request url: https://github.com/pharo-project/pharo/pull/1978 Issue Url: https://pharo.fogbugz.com/f/cases/22662 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

[Pharo 7.0-dev] Build #1384: 22667-operation-support-for-Global-var-read-and-more-tests

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1384 was: FAILURE. The Pull Request #1981 was integrated: "22667-operation-support-for-Global-var-read-and-more-tests" Pull request url: https://github.com/pharo-project/pharo/pull/1981 Issue Url: https://pharo.fogbugz.com/f/cases/22667 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

[Pharo 7.0-dev] Build #1383: 22666-add-refications-for-operation-for-message-and-method

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1383 was: SUCCESS. The Pull Request #1976 was integrated: "22666-add-refications-for-operation-for-message-and-method" Pull request url: https://github.com/pharo-project/pharo/pull/1976 Issue Url: https://pharo.fogbugz.com/f/cases/22666 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

[Pharo 7.0-dev] Build #1382: 22558-FFICalloutresolveType-ignores-class-side-definition (v2)

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #1382 was: FAILURE. The Pull Request #1899 was integrated: "22558-FFICalloutresolveType-ignores-class-side-definition (v2)" Pull request url: https://github.com/pharo-project/pharo/pull/1899 Issue Url: https://pharo.fogbugz.com/f/cases/22558 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 13, 2018

Pharo 7 speed compared to VW (was: The STON Specification, Cog speed, and namespaces/modules)

by Shaping

Does anyone know whether Pharo 7 is as fast as VW 7.10 or later? Do we have recent comparative benchmarks? The comparison should ignore potential GPU-based improvement in my algo; that will happen later. The test should involve some math, file streaming, and the parsing that entailsâan ordinary mix of macrobenchmarks. The comparison should be based on both Pharo 7 and VW each running a single Smalltalk Process (one time-slice of one OS thread in one OS process). I need Pharo 7 speed to be comparable or better to justify the port. Pharo is definitely looking and working better. Iâve spend more time with it last few weeks than during the previous decade. Thanks to everyone for the effort and improvements. Shaping From: Shaping [mailto:shaping@uurda.org] Sent: Wednesday, November 7, 2018 00:41 To: 'Pharo Development List' <pharo-dev(a)lists.pharo.org> Subject: RE: [Pharo-dev] [ANN] The STON Specification, Cog speed, and namespaces/modules Hi Eliot. Pharo (& Squeak & Cuis) Float subclass BoxedFloat64 maps exactly to VW's Double. In 64-bit SmallFloat64 maps exactly to SmallDouble. But I wonder whether there is any issue here. STON would use the print strings for (PSC) Float / (VW) Double, and so deseerialization on Pharo would automatically produce the right class. Going in the other direction might need some help. APF needs support in PSC before one can port, but are representable as suitably-sized word arrays. There is no support for __float128 anywhere in the VM (e.g. not even in the FFI) on PSC as yet. I see Pharoâs WordArray. Iâll work on an APF for Pharo, as time permits. Iâm using APFs in VW in the 300-bit range, and want to reduce the needed precision to 64 bits, to save space and time on large (5 million+) scalar time-series, both on the heap and during BOSSing (25 m save-time now). The problem is not so much an issue for the JulianDayNumber (JDN)-precision, which is adequate in this app at 14 to 15 digits (even though my JDN class subclasses APF, for now). Other calculations need the more extreme precision. I think I can make 128-bit floats work, and would really like to see a small, fast, boxed 128-bit float implementation in Pharo or VW. The APFs are big and slow. Where in the queue of planned improvements to Pharo does such a task lie? I suspect itâs not a very popular item. Broadening the issue somewhat, Iâm trying to find as many good reasons as possible to justify the work needed to port all my VW stuff to Pharo. Iâve seen the references to Cogâs speed and coming speed-up. Are there recent (say, in the last year) benchmarks comparing VW and Pharo? Any details here would be very much appreciated. Having no namespaces in Pharo is, I think, the biggest impediment. I prefer not to prefix class names, but there may be fewer name-collisions than I suppose--maybe none. Still, I need to know how VW and Pharo classes map in order to place overrides and extensions correctly. Besides the mentioned float-class mappings is there a reference on this? Object allSubclasses in Pharo 7 64-bit, produces 14946 classes. Pharo is a little bigger than it used to be. I suppose I donât need to check all unloaded packages because all classes in each of those will have the same unique prefix. Is that correct? Or, I could just load every package I can find, before I check names. But that little experiment has never gone well in the past. Is the Pharo-with-namespaces issue dead or merely suspended, awaiting a more fitting idea than what VW currently offers? Shaping On Tue, Nov 6, 2018 at 12:56 AM Shaping <shaping(a)uurda.org <mailto:shaping@uurda.org> > wrote: STON is able to serialise Pharoâs Floats, what do you mean by double ? Floating-point numbers in IEEE 64-bit format, 14 or 15 significant digits, with a range between =10^307 and 10^307. Additionally, I recently asked Sven if t would be possible to store ScaledDecimals (I think it implements what you call ArbitraryPrecisionFloats) without loss of precision. Iâm referring to VWâs Double (and SmallDouble in 64-bit engines/images). APFs are binary representations that can be arbitrarily large, using Integers (LargePositiveIntegers for example) to model the bits of the mantissa. Before, because STON extends JSON, it was storing all kind of numbers either as float or integer. Now, thanks to Sven, STON stores ScaledDecimals correctly (without loss of precision through serialisation as float, what was done before). But I do not know if this change is integrated in recent images yet. â¦.. Number subclass: #Float instanceVariableNames: '' classVariableNames: 'E Epsilon Halfpi Infinity Ln10 Ln2 MaxVal MaxValLn MinValLogBase2 NaN NegativeInfinity NegativeZero Pi RadiansPerDegree Sqrt2 ThreePi Twopi' poolDictionaries: '' category: 'Kernel-Numbers' My instances represent IEEE-754 floating-point double-precision numbers. They have about 16 digits of accuracy and their range is between plus and minus 10^307. Some valid examples are: 8.0 13.3 0.3 2.5e6 1.27e-30 1.27e-31 -12.987654e12 â¦. I see that Pharoâs Float is VWâs Double. So then I just need to be able to serialize APF. â¦. FIFloatType subclass: #FFIFloat128 instanceVariableNames: '' classVariableNames: '' poolDictionaries: '' category: 'UnifiedFFI-Types' I'm a 128bits (cuadruple precision) float. It is usually not used, but some compiler modes support it (__float128 in gcc) THIS IS NOT YET SUPPORTED â¦. The class above is also from the Pharo 7 image. This is the largest of the c-type FFIFloats. Any Float classes of this size and larger for the Smalltalk heap on 64-bit Pharo? Cheers, Shaping Le 6 nov. 2018 Ã 06:58, Shaping <shaping(a)uurda.org <mailto:shaping@uurda.org> > a Ã©crit : (Having domain problems recently. Please excuse this posting if you have seen it twice. I've not seen it appear yet on the list.) Can STON be extended to handle Doubles and ArbitraryPrecisionFloats? Shaping -----Original Message----- From: Pharo-dev [mailto:pharo-dev-bounces@lists.pharo.org] On Behalf Of David T. Lewis Sent: Wednesday, October 31, 2018 14:58 To: Pharo Development List <pharo-dev(a)lists.pharo.org <mailto:pharo-dev@lists.pharo.org> > Subject: Re: [Pharo-dev] [ANN] The STON Specification This is very clear and well written. Dave Hi, Since there can never be enough documentation I finally took some time to write a more formal description of STON as a data format. https://github.com/svenvc/ston/blob/master/ston-spec.md The idea is to let this stabilise a bit and to then update the two other documents describing STON, where necessary: https://github.com/svenvc/ston/blob/master/ston-paper.md https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuc cessfulBuild/artifact/book-result/STON/STON.html Also, the latest changes in STON have to make their way to the Pharo image as well. https://github.com/svenvc/ston All feedback is welcome. Sven -- Sven Van Caekenberghe Proudly supporting Pharo http://pharo.org http://association.pharo.org http://consortium.pharo.org -- _,,,^..^,,,_ best, Eliot

Nov. 13, 2018

Re: [Pharo-dev] Anyone else seen crashes like these ?

by Eliot Miranda

Hi Ben, On Mon, Nov 12, 2018 at 8:51 AM Ben Coman <btc(a)openinworld.com> wrote: > > > On Mon, 12 Nov 2018 at 21:37, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> Hi, >> >> I run Pharo 7 64-bit on a macOS laptop, where the images are kept running >> across sleep/wake cycles. >> > > Just to confirm, you mean your laptop sleep/wake cycle. > On my Window 10 laptop I do the same quite often with no problems. > > >> For many weeks, it often happens that an image crashes before/after such >> a sleep/wakeup (not all the time, just regularly). >> >> Here is a crash dump from today (fresh image/vm from WE, nothing special >> loaded). >> >> Related to scheduling ? Event handling ? > > > In trying to understand the last few moments, is the recent primitives > list strongly ordered in time?... > Most recent primitives > signal > nowTick > primSignal:atUTCMicroseconds: > wait > millisecondClockValue > @ > actualScreenSize > Yes, it's a log of the most recent 256 named primitives. Note that they're irrelevant in this case. The crash is always rooted in vmIOProcessEvents. i.e. the VM is responding to some input event, and it calls pumpRunLoop to do (Objective-C) [NSRunLoop mainRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantPast], which in Smalltalk would be written NSRunLoop mainRunLoop runMode: NSDefaultRunLoopMode beforeDate: NSDate distantPast. And somewhere within this a display update occurs which crashes, presumably because we're using stale data that should have been invalidated oil sleep and refreshed on wake. > > > Starting at #actualScreenSize, instrumenting the code by adding... > Transcript crShow: thisContext sender printString. > indicates the following call chain... > DisplayScreen-class>>actualScreenSize > MorphicUIManager>>checkForNewDisplaySize > DisplayScreen class>>checkForNewScreenSize > WorldState>>doOneCycleNowFor: > WorldState>>doOneCycleFor: > WorldMorph>>doOneCycle > WorldMorph class>>doOneCycle > MorphicUIManager>>spawnNewProcess > > In detail from a static analysis of the code... > > WorldState>>doOneCycleNowFor: > > DisplayScreen class>>checkForNewScreenSize > > MorphicUIManager>>checkForNewDisplaySize > > DisplayScreen-class>>actualScreenSize > PRIMITIVE > < MorphicUIManager>>checkForNewDisplaySize > < DisplayScreen class>>checkForNewScreenSize > < WorldState>>doOneCycleNowFor > < WorldState>>doOneCycleFor: > < WorldMorph>>doOneCycle > < WorldMorph class>>doOneCycle > < MorphicUIManager>>spawnNewProcess > > WorldMorph class>>doOneCycle > > WorldMorph>>doOneCycle > > WorldState>>doOneCycleNowFor: > > WorldState>>interCyclePause: > > Time>>millisecondClockValue PRIMITIVE > < WorldState>>interCyclePause: > > Delay>>schedule > > DelaySemaphoreScheduler>>schedule: > > Semaphore>>wait > PRIMITIVE "readyToSchedule variable" > > Semaphore>>signal > PRIMITIVE "timingSemaphore variable" NOT RECORDED > > maybe context change causes this to be recorded later in "Most > recent primitives" > > > DelayMicrosecondTicker>>waitForUserSignalled:orExpired: waking up from > Semaphore>>wait > the primSignal:atUTCMicroseconds: > PRIMITIVE immediately before the #wait seems to have been logged now in > "Most recent primitives" > > > DelaySemaphoreScheduler>>scheduleAtTimingPriority > > Semaphore>>signal > PRIMITIVE "readyToSchedule variable" seems to have not been recorded in > "Most recent primitives" > > > DelayBasicScheduler>>scheduleAtTimingPriority > < > DelaySemaphoreScheduler>>scheduleAtTimingPriority > < > DelayBasicScheduler>>runBackendLoopAtTimingPriority > > DelayMicrosecondTicker>>nowTick > PRIMITIVE > > Delay>>timingPrioritySignalExpired > > Semaphore>>signal > PRIMITIVE "delaySemaphore variable" CRASH > > Smalltalk stack dump: > 0x7ffee21d7138 M Delay>timingPrioritySignalExpired 0x113e3b138: a(n) > Delay > 0x7ffee21d7170 M [] in > DelaySemaphoreScheduler(DelayBasicScheduler)>runBackendLoopAtTimingPriority > 0x11453b8c0: a(n) DelaySemaphoreScheduler > 0x1169f51c8 s BlockClosure>ensure: > 0x1169f5828 s > DelaySemaphoreScheduler(DelayBasicScheduler)>runBackendLoopAtTimingPriority > 0x1169f5c38 s [] in > DelaySemaphoreScheduler(DelayBasicScheduler)>startTimerEventLoopPriority: > 0x1169fae10 s [] in BlockClosure>newProcess > > The only reason I can think that last semaphore call would be a problem is > if variable delaySemaphore was not a Semaphore, > but the C stack backtrace doesn't look like it crashes in the Semaphore > handling. > Its perhaps co-incidental with delay scheduling code is invoked fairly > often, that its the current process when this crash occurs. > > > To hep isolate the failure point, I would considering copying #nowTIck to > a multiple series like... > #nowTick1 > #nowTick2 > #nowTick3 > etc so that all of the sender sites can be made distinct. > And maybe the same for #signal senders within the delay scheduling code. > > One side-curiousity is I can't see where the primitive #@ is being called > from. > Does it indicate some loop as its trying to determine > > To me the "vmIOProcessEvents" in the C backtrace with a fairly deep stack > into the AppKit framework > indicates its related to event handling. > > cheers -ben > -- _,,,^..^,,,_ best, Eliot

Nov. 13, 2018

Re: [Pharo-dev] Anyone else seen crashes like these ?

by Eliot Miranda

Hi Sven, On Mon, Nov 12, 2018 at 5:37 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Hi, > > I run Pharo 7 64-bit on a macOS laptop, where the images are kept running > across sleep/wake cycles. > > For many weeks, it often happens that an image crashes before/after such a > sleep/wakeup (not all the time, just regularly). > > Here is a crash dump from today (fresh image/vm from WE, nothing special > loaded). > > Related to scheduling ? Event handling ? > The stack trace points to OpenGL rendering: 2 Pharo sigsegv + 155 3 libsystem_platform.dylib _sigtramp + 29 4 ??? 0x0 + 0 5 AppleIntelKBLGraphicsGLDriver glrUpdateTexture + 8979 6 libGPUSupportMercury.dylib gpusLoadCurrentTextures + 591 7 AppleIntelKBLGraphicsGLDriver updateDispatch + 663 8 GLEngine gleDoDrawDispatchCore + 629 9 GLEngine glDrawArraysInstanced_STD_Exec + 264 10 GLEngine glDrawArrays_UnpackThread + 40 11 GLEngine gleCmdProcessor + 77 12 libdispatch.dylib _dispatch_client_callout + 8 13 libdispatch.dylib _dispatch_lane_barrier_sync_invoke_and_complete + 60 14 GLEngine glFlush_ExecThread + 15 15 Pharo -[sqSqueakOSXOpenGLView drawRect:flush:] + 301 16 Pharo -[sqSqueakOSXOpenGLView drawRect:] + 43 17 AppKit _NSViewDrawRect + 66 18 AppKit -[NSView _drawRect:clip:] + 1745 19 AppKit -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] + 1400 20 AppKit -[NSView _recursiveDisplayAllDirtyWithLockFocus:visRect:] + 2622 21 AppKit -[NSView _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] + 938 22 AppKit -[NSThemeFrame _recursiveDisplayRectIfNeededIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:topView:] + 65 23 AppKit -[NSView _oldDisplayRectIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:] + 1887 24 AppKit -[NSView _displayRectIgnoringOpacity:isVisibleRect:rectIsVisibleRectForView:] + 253 25 AppKit -[NSView displayIfNeeded] + 1300 26 AppKit -[NSWindow displayIfNeeded] + 283 27 AppKit __NSWindowGetDisplayCycleObserverForDisplay_block_invoke + 722 28 AppKit NSDisplayCycleObserverInvoke + 170 29 AppKit NSDisplayCycleFlush + 1073 30 QuartzCore _ZN2CA11Transaction19run_commit_handlersE18CATransactionPhase + 49 31 QuartzCore _ZN2CA11Transaction6commitEv + 186 32 AppKit __65+[CATransaction(NSCATransaction) NS_setFlushesWithDisplayRefresh]_block_invoke + 274 33 CoreFoundation __CFRUNLOOP_IS_CALLING_OUT_TO_AN_OBSERVER_CALLBACK_FUNCTION__ + 23 34 CoreFoundation __CFRunLoopDoObservers + 452 35 CoreFoundation __CFRunLoopRun + 1166 36 CoreFoundation CFRunLoopRunSpecific + 463 37 Foundation -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 280 38 Pharo -[sqSqueakMainApplication(events) pumpRunLoop] + 103 39 Pharo -[sqSqueakOSXApplication(events) pumpRunLoop] + 43 40 Pharo vmIOProcessEvents + 255 and this is core VM code, so likely an pensmalltalk-vm macOS platform bug, hence best discussed on vm-dev. Can you describe exactly the steps you take to reproduce the crash? _,,,^..^,,,_ best, Eliot

Nov. 13, 2018

Re: [Pharo-dev] Anyone else seen crashes like these ?

by Ben Coman

On Tue, 13 Nov 2018 at 02:18, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Yes I mean laptop sleep/wake. > Thanks for looking into this. > But I think the first step is to see if this is repeatable for someone > else. > It is probably some graphics primitive rather than scheduling (I am > guessing here). > Seems so. Different Smalltalk stack. Very similar C backtrace. I've compared the two here... https://docs.google.com/spreadsheets/d/1pa8Ehwp9b0cW7NaSlElqvPQ5lzSbieike_h… cheers -ben

Nov. 12, 2018