January 2018 - Pharo-dev - lists.pharo.org

Pharo 64bit on Ubuntu (was Re: failing tests in Pharo 7)

by Alistair Grant

Hi Ben, On 7 January 2018 at 05:22, Ben Coman <btc(a)openinworld.com> wrote: > Ubuntu 16.04 64bit > Build information: > Pharo-7.0+alpha.build.409.sha.bb4eaaf976e3fb148b33b6d87598022b77329768 (64 Bit) > Virtual Machine: Pharo64/lib/pharo/5.0-201712211450/pharo I've mostly ignored 64 bit Pharo because I can't get Iceberg stable enough. Since you're also using Ubuntu 16.04, could I ask you to try (obviously somewhere it is OK to crash the vm :-)): | pharoRepository | Iceberg enableMetacelloIntegration: true. Metacello new repository: 'github://akgrant43/pharo-akgosprocess/mc'; baseline: 'AKGOSProcess'; load. pharoRepository := IceRepository registry detect: [ :each | each name = 'pharo-akgosprocess' ]. pharoRepository pull. And let me know the result. I get: pharo: malloc.c:2394: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed. /snap/pharo7/x1/usr/bin/pharo: line 43: 21304 Aborted (core dumped) $pharoexe $origargs Thanks! Alistair

Jan. 7, 2018

Re: [Pharo-dev] 1.0 largeIdentityHash fails on 64-bit Linux

by Max Leske

That seems to fix it. SmallFloat64 needs to implement #basicIdentityHash with primitive 171. Thanks Eliot. On 7 January 2018 at 06:19:18, Eliot Miranda (eliot.miranda(a)gmail.com) wrote: Hi Ben, On Sat, Jan 6, 2018 at 8:47 PM, Ben Coman <btc(a)openinworld.com> wrote: > With Linux 64 bit image > TestRunner running FLBinaryFileStreamSerializationTest > produces error "PrimitiveFailed: primitive #basicIdentityHash in > SmallFloat64 failed" > I suspect the SmallFloat64 basicIdentityHash method is missing or using the wrong primitive number. Here's what happens in Squeak: 1.0 class -> 1.0 identityHash hex -> ((1.0 class whichClassIncludesSelector: #identityHash) >> #identityHash) primitive SmallFloat64->'16r7F0000000000000'->171 And primitive 171 is the primitiveImmediateAsInteger primitive that converts SmallIntegers to themselves, Characters to their integer codes, and SmallFloat64 to an integer representing their bits. e.g. #(-0.0 0.0 -1.0 1.0) collect: [:n| n -> n identityHash hex -> (n identityHash bitAnd: 2 << 64 - 1) hex] {-0.0->'-16r1000000000000000'->'16r1F000000000000000' . 0.0->'16r0'->'16r0' . -1.0->'-16r810000000000000'->'16r1F7F0000000000000' . 1.0->'16r7F0000000000000'->'16r7F0000000000000'} SmallFloat64>>identityHash "Answer an integer unique to the receiver." <primitive: 171> ^self primitiveFailed HTH > The same error occurs evaluating... > 1.0 largeIdentityHash > which works fine on 32-bit. > > https://pharo.fogbugz.com/f/cases/20936/64bit-primitiveFailure-for-1-0- > largeIdentityHash > > Build information: > Pharo-7.0+alpha.build.409.sha.bb4eaaf976e3fb148b33b6d87598022b77329768 > (64 Bit) > CoInterpreter VMMaker.oscog-eem.2295 Dec 21 2017 > StackToRegisterMappingCogit VMMaker.oscog-eem.2297 Dec 21 2017 > VM: 201712211450 > Unix built on Dec 21 2017 14:53:19 Compiler: 4.8.4 > > cheers -ben > -- _,,,^..^,,,_ best, Eliot

Jan. 7, 2018

Re: [Pharo-dev] failing tests in Pharo 7

by Sven Van Caekenberghe

> On 7 Jan 2018, at 10:30, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > Hi Sven. > > On 6 January 2018 at 22:55, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> >>> On 6 Jan 2018, at 20:08, Alistair Grant <akgrant0710(a)gmail.com> wrote: >>> >>> Hi Ben, >>> >>> On 6 January 2018 at 12:09, Ben Coman <btc(a)openinworld.com> wrote: >>>> It great having a CI system to check all our tests a passing, >>>> but I thought I'd go old school and run them from the TestRunner GUI >>>> and found in a fresh latest Pharo 7 there are "128 failures, 63 errors" >>>> >>>> This one is interesting... >>>> ArrayTest>>test0FixtureAsSetForIdentityMultiplinessTest >>>> | anElement res | >>>> self elementsCopyNonIdenticalWithoutEqualElements. >>>> anElement := self elementsCopyNonIdenticalWithoutEqualElements anyOne. >>>> self deny: anElement copy == anElement. "<<<<FAILS" >>>> >>>> where anElement ==> SmallFloat64 (1.5) >>>> so the deny fails. >>> >>> Which platform and image? >>> >>> Pharo 7.0 >>> Build information: >>> Pharo-7.0+alpha.build.410.sha.aa40385cd6fee69bb51d4a3afcacfb5f91eed206 >>> (32 Bit) >>> Ubuntu 16.04 >>> >>> 14915 run, 14859 passes, 31 skipped, 50 expected failures, 5 failures, >>> 1 errors, 0 unexpected passes >>> Failures: >>> ReleaseTest>>#testMethodsWithUnboundGlobals >>> TonelWriterTest>>#testWriteSnapshot >>> ReleaseTest>>#testUnpackagedClasses >>> ReleaseTest>>#testNoEmptyPackages >>> ReleaseTest>>#testThatAllMethodsArePackaged >>> >>> Errors: >>> ZnClientTests>>#testRedirect >> >> Hmm, strange. Does it failed each time ? >> >> It does not fail for me (macOS). Furthermore, all Zinc tests are green https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/ > > It's failing quite rarely. I think we just got (un)lucky this time. > #testRedirect connects to http://zn.stfx.eu, so it could well be an > intermittent internet connection issue. > > What would we lose by restructuring the tests to use an internal > server? My initial reaction is that an increase in test reliability > would outweigh any loss of testing. And we could create manually run > tests for anything that can't be covered by the internal server tests. I know, but most (functional) tests *ARE* internal (are running against a server created just for the test). The remaining are there because HTTP *IS* a network communication protocol where it is important to test against servers 'out there in the wild'. It would be too limiting to exclude the networking (with its connectivity and timing issues) and the fact that there are hundreds of server and TLS/HTTPS implementations out there. Splitting the tests in two group might be a first step. > Cheers, > Alistair > > > > > >>> Cheers, >>> Alistair >>> >>> >>> >>>> It seems that the test setUp is wrong since >>>> #elementsCopyNonIdenticalWithoutEqualElements >>>> does not return a collection (of elements for which copy is not identical ) >>>> without equal elements:" >>>> >>>> This seems like a permanent error rather than a transient one. >>>> How does this slip through CI testing? >>>> and what can be done about cleaning this up? >>>> >>>> cheers -ben >>>> >>>> P.S. With the loss of method version info is impossible to guess when this >>>> may have been introduced, to guess which Issue it was related to, to guess >>>> how I might approach fixing this myself.

Jan. 7, 2018

PrimitiveOSError class name

by Alistair Grant

Hi Eliot, I'm just preparing a fogbugz issue and PR to add PrimitiveOSError to Pharo and was wondering about the class name. As you say in the class comments, errorName is typically set to #'operating system error' (I haven't seen anything else), which makes the class name and errorName somewhat redundant. What do you think of renaming the class to PrimitiveError, which will make it a bit more general and extensible in future? Thanks, Alistair

Jan. 7, 2018

Re: [Pharo-dev] failing tests in Pharo 7

by Alistair Grant

Hi Sven. On 6 January 2018 at 22:55, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > >> On 6 Jan 2018, at 20:08, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> Hi Ben, >> >> On 6 January 2018 at 12:09, Ben Coman <btc(a)openinworld.com> wrote: >>> It great having a CI system to check all our tests a passing, >>> but I thought I'd go old school and run them from the TestRunner GUI >>> and found in a fresh latest Pharo 7 there are "128 failures, 63 errors" >>> >>> This one is interesting... >>> ArrayTest>>test0FixtureAsSetForIdentityMultiplinessTest >>> | anElement res | >>> self elementsCopyNonIdenticalWithoutEqualElements. >>> anElement := self elementsCopyNonIdenticalWithoutEqualElements anyOne. >>> self deny: anElement copy == anElement. "<<<<FAILS" >>> >>> where anElement ==> SmallFloat64 (1.5) >>> so the deny fails. >> >> Which platform and image? >> >> Pharo 7.0 >> Build information: >> Pharo-7.0+alpha.build.410.sha.aa40385cd6fee69bb51d4a3afcacfb5f91eed206 >> (32 Bit) >> Ubuntu 16.04 >> >> 14915 run, 14859 passes, 31 skipped, 50 expected failures, 5 failures, >> 1 errors, 0 unexpected passes >> Failures: >> ReleaseTest>>#testMethodsWithUnboundGlobals >> TonelWriterTest>>#testWriteSnapshot >> ReleaseTest>>#testUnpackagedClasses >> ReleaseTest>>#testNoEmptyPackages >> ReleaseTest>>#testThatAllMethodsArePackaged >> >> Errors: >> ZnClientTests>>#testRedirect > > Hmm, strange. Does it failed each time ? > > It does not fail for me (macOS). Furthermore, all Zinc tests are green https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/ It's failing quite rarely. I think we just got (un)lucky this time. #testRedirect connects to http://zn.stfx.eu, so it could well be an intermittent internet connection issue. What would we lose by restructuring the tests to use an internal server? My initial reaction is that an increase in test reliability would outweigh any loss of testing. And we could create manually run tests for anything that can't be covered by the internal server tests. Cheers, Alistair >> Cheers, >> Alistair >> >> >> >>> It seems that the test setUp is wrong since >>> #elementsCopyNonIdenticalWithoutEqualElements >>> does not return a collection (of elements for which copy is not identical ) >>> without equal elements:" >>> >>> This seems like a permanent error rather than a transient one. >>> How does this slip through CI testing? >>> and what can be done about cleaning this up? >>> >>> cheers -ben >>> >>> P.S. With the loss of method version info is impossible to guess when this >>> may have been introduced, to guess which Issue it was related to, to guess >>> how I might approach fixing this myself. > >

Jan. 7, 2018

Re: [Pharo-dev] 1.0 largeIdentityHash fails on 64-bit Linux

by Eliot Miranda

Hi Ben, On Sat, Jan 6, 2018 at 8:47 PM, Ben Coman <btc(a)openinworld.com> wrote: > With Linux 64 bit image > TestRunner running FLBinaryFileStreamSerializationTest > produces error "PrimitiveFailed: primitive #basicIdentityHash in > SmallFloat64 failed" > I suspect the SmallFloat64 basicIdentityHash method is missing or using the wrong primitive number. Here's what happens in Squeak: 1.0 class -> 1.0 identityHash hex -> ((1.0 class whichClassIncludesSelector: #identityHash) >> #identityHash) primitive SmallFloat64->'16r7F0000000000000'->171 And primitive 171 is the primitiveImmediateAsInteger primitive that converts SmallIntegers to themselves, Characters to their integer codes, and SmallFloat64 to an integer representing their bits. e.g. #(-0.0 0.0 -1.0 1.0) collect: [:n| n -> n identityHash hex -> (n identityHash bitAnd: 2 << 64 - 1) hex] {-0.0->'-16r1000000000000000'->'16r1F000000000000000' . 0.0->'16r0'->'16r0' . -1.0->'-16r810000000000000'->'16r1F7F0000000000000' . 1.0->'16r7F0000000000000'->'16r7F0000000000000'} SmallFloat64>>identityHash "Answer an integer unique to the receiver." <primitive: 171> ^self primitiveFailed HTH > The same error occurs evaluating... > 1.0 largeIdentityHash > which works fine on 32-bit. > > https://pharo.fogbugz.com/f/cases/20936/64bit-primitiveFailure-for-1-0- > largeIdentityHash > > Build information: > Pharo-7.0+alpha.build.409.sha.bb4eaaf976e3fb148b33b6d87598022b77329768 > (64 Bit) > CoInterpreter VMMaker.oscog-eem.2295 Dec 21 2017 > StackToRegisterMappingCogit VMMaker.oscog-eem.2297 Dec 21 2017 > VM: 201712211450 > Unix built on Dec 21 2017 14:53:19 Compiler: 4.8.4 > > cheers -ben > -- _,,,^..^,,,_ best, Eliot

Jan. 7, 2018

1.0 largeIdentityHash fails on 64-bit Linux

by Ben Coman

With Linux 64 bit image TestRunner running FLBinaryFileStreamSerializationTest produces error "PrimitiveFailed: primitive #basicIdentityHash in SmallFloat64 failed" The same error occurs evaluating... 1.0 largeIdentityHash which works fine on 32-bit. https://pharo.fogbugz.com/f/cases/20936/64bit-primitiveFailure-for-1-0-larg… Build information: Pharo-7.0+alpha.build.409.sha.bb4eaaf976e3fb148b33b6d87598022b77329768 (64 Bit) CoInterpreter VMMaker.oscog-eem.2295 Dec 21 2017 StackToRegisterMappingCogit VMMaker.oscog-eem.2297 Dec 21 2017 VM: 201712211450 Unix built on Dec 21 2017 14:53:19 Compiler: 4.8.4 cheers -ben

Jan. 7, 2018

Re: [Pharo-dev] failing tests in Pharo 7

by Ben Coman

On 7 January 2018 at 03:08, Alistair Grant <akgrant0710(a)gmail.com> wrote: > Hi Ben, > > On 6 January 2018 at 12:09, Ben Coman <btc(a)openinworld.com> wrote: > > It great having a CI system to check all our tests a passing, > > but I thought I'd go old school and run them from the TestRunner GUI > > and found in a fresh latest Pharo 7 there are "128 failures, 63 errors" > > > > This one is interesting... > > ArrayTest>>test0FixtureAsSetForIdentityMultiplinessTest > > | anElement res | > > self elementsCopyNonIdenticalWithoutEqualElements. > > anElement := self elementsCopyNonIdenticalWithoutEqualElements anyOne. > > self deny: anElement copy == anElement. "<<<<FAILS" > > > > where anElement ==> SmallFloat64 (1.5) > > so the deny fails. > > Which platform and image? > I guess its not a platform issue, and the SmallFloat64 was sufficient hint, but sorry I should be more explicit. Ubuntu 16.04 64bit Build information: Pharo-7.0+alpha.build.409.sha.bb4eaaf976e3fb148b33b6d87598022b77329768 (64 Bit) Virtual Machine: Pharo64/lib/pharo/5.0-201712211450/pharo The root cause seems to be... in 32-bit Image... x := 1.0. x copy == x. ==> false in 64-bit Image... x := 1.0. x copy == x. ==> true which I presume has always been the case and is expected, but such floats are no longer are suitable for tests like... test0FixtureAsSetForIdentityMultiplinessTest "a collection (of elements for which copy is not identical ) [...]" If someone can suggest the preferred element type to substitute for the floats I can do the leg work to push this through. https://pharo.fogbugz.com/f/cases/20935/floats-no-longer-suitable-elements-… btw I'm curious, how are these steps interrelated in the CI build? a. build 32-bit image b. build 64-bit image c. test 32-bit image d. test 64-bit image > Pharo 7.0 > Build information: > Pharo-7.0+alpha.build.410.sha.aa40385cd6fee69bb51d4a3afcacfb5f91eed206 > (32 Bit) > Ubuntu 16.04 > > 14915 run, 14859 passes, 31 skipped, 50 expected failures, 5 failures, > 1 errors, 0 unexpected passes > Failures: > ReleaseTest>>#testMethodsWithUnboundGlobals > TonelWriterTest>>#testWriteSnapshot > ReleaseTest>>#testUnpackagedClasses > ReleaseTest>>#testNoEmptyPackages > ReleaseTest>>#testThatAllMethodsArePackaged > I get about the same in 32-bit. I'm too worried about the release tests, although being totally clear would be really nice. > Errors: > ZnClientTests>>#testRedirect > On 7 January 2018 at 05:55, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hmm, strange. Does it failed each time ? > > It does not fail for me (macOS). Furthermore, all Zinc tests are green > https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/ ZnClientTests>>#testRedirect doesn't error for me, but ZnHTTPSTests>>#testGForceInria failed yesterday and today five out of five tries. The error is TestTookTooMuchTime I see that test does... (client := ZnClient new) get: 'http://gforge.inria.fr/frs/?group_id=1299'. and that url opens okay in Chrome web browser, so I don't know what to make of it. In Pharo-7.0.0-alpha.build.410.sha.aa40385.arch.32bit I also get a bunch of errors for SystemDependenciesTest that seem to have the same root cause... KeyNotFound: key #ManifestManifestResourcesTests not found in SystemDictionary cheers -ben

Jan. 7, 2018

Re: [Pharo-dev] failing tests in Pharo 7

by Sven Van Caekenberghe

> On 6 Jan 2018, at 20:08, Alistair Grant <akgrant0710(a)gmail.com> wrote: > > Hi Ben, > > On 6 January 2018 at 12:09, Ben Coman <btc(a)openinworld.com> wrote: >> It great having a CI system to check all our tests a passing, >> but I thought I'd go old school and run them from the TestRunner GUI >> and found in a fresh latest Pharo 7 there are "128 failures, 63 errors" >> >> This one is interesting... >> ArrayTest>>test0FixtureAsSetForIdentityMultiplinessTest >> | anElement res | >> self elementsCopyNonIdenticalWithoutEqualElements. >> anElement := self elementsCopyNonIdenticalWithoutEqualElements anyOne. >> self deny: anElement copy == anElement. "<<<<FAILS" >> >> where anElement ==> SmallFloat64 (1.5) >> so the deny fails. > > Which platform and image? > > Pharo 7.0 > Build information: > Pharo-7.0+alpha.build.410.sha.aa40385cd6fee69bb51d4a3afcacfb5f91eed206 > (32 Bit) > Ubuntu 16.04 > > 14915 run, 14859 passes, 31 skipped, 50 expected failures, 5 failures, > 1 errors, 0 unexpected passes > Failures: > ReleaseTest>>#testMethodsWithUnboundGlobals > TonelWriterTest>>#testWriteSnapshot > ReleaseTest>>#testUnpackagedClasses > ReleaseTest>>#testNoEmptyPackages > ReleaseTest>>#testThatAllMethodsArePackaged > > Errors: > ZnClientTests>>#testRedirect Hmm, strange. Does it failed each time ? It does not fail for me (macOS). Furthermore, all Zinc tests are green https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/ > Cheers, > Alistair > > > >> It seems that the test setUp is wrong since >> #elementsCopyNonIdenticalWithoutEqualElements >> does not return a collection (of elements for which copy is not identical ) >> without equal elements:" >> >> This seems like a permanent error rather than a transient one. >> How does this slip through CI testing? >> and what can be done about cleaning this up? >> >> cheers -ben >> >> P.S. With the loss of method version info is impossible to guess when this >> may have been introduced, to guess which Issue it was related to, to guess >> how I might approach fixing this myself.

Jan. 6, 2018

Re: [Pharo-dev] (Morph>>#asDraggableMorph) isPrimitive

by ClÃ©ment Bera

On Sat, Jan 6, 2018 at 8:07 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Denis, Hi Norbert, > > On Sat, Jan 6, 2018 at 6:30 AM, Denis Kudriashov <dionisiydk(a)gmail.com> > wrote: > >> Hi. >> >> I guess "quick" methods are compiled as special primitives. For example >> accessors are primitives too: >> >> >> (Point >> #x) isPrimitive >> >> >> There is another method #isRealPrimitive which checks for explicit >> primitive call. >> > > That's right. The scheme is as follows: > > Numbered primitives used to be 9 bits fields and primitive numbers 1 > through 255 were for conventional primitives, and primitive numbers 256 > through 512 were for quick primitives, such that 256 through 263 pushed > specific constants and 264 through 512 pushed the inst vars, 263 pushing > inst var 0, 264 pushing inst var 1, etc. Here's the relevant section of > the VM's primitive table: > > "Quick Push Const Methods" > (256 nil) "primitivePushSelf" > (257 nil) "primitivePushTrue" > (258 nil) "primitivePushFalse" > (259 nil) "primitivePushNil" > (260 nil) "primitivePushMinusOne" > (261 nil) "primitivePushZero" > (262 nil) "primitivePushOne" > (263 nil) "primitivePushTwo" > > "Quick Push Inst Var Methods" > (264 519 nil) "primitiveLoadInstVar" > > Now we have 16 bits of primitive index. The scheme up to 32767 is the > same; 1 through 32767 are available for conventional primitives, with 256 > through 512 reserved for quick primitives. Numbers 32768 through 65535 are > reserved for use "inline primitives". We divide the 32k values from 32767 > on up into four 8k groups. Sista has the first 8k. Lowcode has the second > 8k, and the remaining two groups of 8k are unassigned, reserved for future > use. > > > Now that SistaV1 and Spur support up to 64k inst vars one could argue that > we should extend the range of numbers available to quick primitives. But > in the JIT there's no difference between the code generated for a quick > primitive and the code generated for a conventional > pshInstVar:/pushConstant:, methodReturnTop. IQuicjk primitive serve to > improve performance only in the interpreter. So given that we have no > classes defining more than 210 inst vars (in my image) I don't think its a > pressing need :-) > > I think classes with more than 255 inst var will always be uncommon so we don't need to improve the interpreter performance for those cases. In the JIT getters are optimised anyway. > >> 2018-01-06 13:24 GMT+01:00 Norbert Hartl <norbert(a)hartl.name>: >> >>> Can anyone explain why >>> >>> (Morph>>#asDraggableMorph) isPrimitive. >>> >>> returns true? >>> >>> Norbert >>> >> > _,,,^..^,,,_ > best, Eliot > -- ClÃ©ment BÃ©ra Pharo consortium engineer https://clementbera.wordpress.com/ BÃ¢timent B 40, avenue Halley 59650 Villeneuve d'Ascq

Jan. 6, 2018