- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [NB] Undocumented NativeBoost options

by Igor Stasenko

On 15 November 2014 12:37, Max Leske <maxleske(a)gmail.com> wrote: > I got a bit frustrated and ended up searching the source for > â[\w]opt[[:upper]]â to find all NativeBoost options. Some of those options > are in fact undocumented: > > optMayGC > - tells that during calling external function one may trigger GC. Therefore, if that option is on, the code generator will use a gate function for call, to safeguard against possible relocation of compiled method, which contains machine code (and thus could return from external function to wrong place). > optProxyLabels > - used for debugging purposes, to see where the proxy function (and what exactly function) get called in generated code. Does not affects anything in resulting machine code. > optCdecl > - use cdecl calling convention. This option are exposed through different public API and not used directly > optReturnNullAsNil > - for pointer return types and ExternalObject, coerce null return value to nil. > optStdcall > - same as for optCdecl, except it is stdcall convention. > optNonMovable - tells the code generator that the generated code will never move (relocate) in memory. Used for generating code and placing it in external memory > optReserveCallgateSpace > - not used optDebug > - halt at the end of code generation, to see what is generated. For debugging purposes. > optNoAlignment > - ignore default stack alignment when generating a call to external function. Can be used, if you absolutely sure that subject function doesn't requires stack alignment. Thus one can avoid emitting extra code for stack alignment => saving some CPU cycles. > optEmitCall > - private. optNoCleanup > - do not emit code for cleaning stack after calling a function according to calling convention (not all conventions require stack cleanup, cdecl does). Basically, leaving stack pointer untouched after return from external call. optCheckFailOnEveryArgument > - check for the primitive failure after each argument coercion. By default the check is generated only at single place, right before performing an external function call, after all arguments are coerced and pushed on stack. Useful for debugging, when it is hard to figure, which exactly argument fails the coercion. > optStringOrNull > - allow passing nil for arguments of String type, effectively will push null pointer as argument WinUnicode > - Controls the coercion of Windows string types to be either single-byte or multi-byte. Related to types like LPCTSTR etc. See NBTString class and its sole class-side method. -- Best regards, Igor Stasenko.

Nov. 16, 2014

Re: [Pharo-dev] [NB] NativeBoost and optCoerceNilToNull

by Igor Stasenko

On 16 November 2014 19:11, Max Leske <maxleske(a)gmail.com> wrote: > > On 15.11.2014, at 22:28, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 15 November 2014 15:53, Max Leske <maxleske(a)gmail.com> wrote: > >> Once again Iâm struggling with NativeBoost. Iâm trying to use >> optCoerceNilToNull with this function: >> >> ^self call: #(LGitReturnCodeEnum git_commit_create( >> LGitId * theCommitId, >> LGitRepository repo, >> String update_ref, >> LGitSignature * theAuthor, >> LGitSignature * theCommitter, >> String message_encoding, >> String theMessage, >> LGitTree theTree, >> int parent_count, >> LGitCommit * parentsPointer)) options: #( optCoerceNilToNull ) >> >> The last argument can be NULL, per documentation, (no parents). >> LGitCommit is a subclass of NBExternalObject and apparently >> NBExternalObjctType>>pushAsPointer: does *not* use the optCoerceNilToNull >> option (the super implementation in NBExternalType>>pushAsPointer: however >> does). The upshot of course is that the call fails with âan instance of >> LGitCommit expectedâ. >> >> My questions: >> - The documentation says: >> >> >> "#optCoerceNilToNull" >> "passing nil as a pointer-type argument, converts it to C NULL â >> >> but apparently this doesnât apply to every pointer type. Should the >> documentation be updated? >> - Why does NBExternlObjectType not use optCoerceNilToNull? I donât want >> to use tricks to pass null if I donât have to. >> > > perhaps because it is not there? :) > So, lets add it. > > > Yay! :) I have zero assembly skillsâ¦ Iâve tried the following but then get > an âundefined label done18â error: > > NBExternalObject>>pushAsPointer: gen > "push a pointer to handle value" > | asm proxy oop notNil done | > proxy := gen proxy. > asm := gen asm. > > <<<<<<<<<<<<<< new > > done := asm uniqueLabelName: 'doneâ. > > >>>>>>>>>>>>>>> new > > oop := gen reserveTemp. > loader emitLoad: gen to: oop. > > <<<<<<<<<<<<<< new > > "handle nils, if we care" > gen optCoerceNilToNull > ifTrue: [ > notNil := asm uniqueLabelName: 'notNil'. > proxy nilObject. > asm cmp: asm EAX with: oop; > jne: notNil; > xor: asm EAX with: asm EAX; > jmp: done; > label: notNil ] > ifFalse: [ > "we can skip class verification, if loader loads receiver, since nothing > to fear there" > loader isReceiver ifFalse: [ > self verifyClassOf: oop is: objectClass generator: gen. > ] ]. > > >>>>>>>>>>>>>>> new > proxy fetchPointer: (self handleIvarIndex) ofObject: oop. "handle ivar" > proxy varBytesFirstFieldOf: asm EAX. "handle value ptr" > asm push: asm EAX. > gen releaseTemps: 1. > > > please make an issue, so we can fix & integrate it. > > - Related: there are instances where even non-pointer arguments are >> allowed to be NULL. Why does optCoerceNilToNull only work for pointer types? >> >> > like what? Suppose the argument type is a structure, e.g > "struct foo bar", that is non-pointer type. What in this case the NULL > value for the structure? > > > Ok, I think I understand. Are you saying that, since NULL is a macro, you > have no way of telling during runtime how NULL is defined? For a pointer it > is simply 0 or (void *) 0, but for a non pointer it depends on the context. > Is that about right? > > I am saying that there is only 'NULL pointer' defined, i never heard anything about 'NULL Float', or 'NULL struct', or 'NULL union'. This is why there's only nil->NULL coercion provided. > > >> Igor! Help! :) >> >> Cheers, >> Max >> >> > > > -- > Best regards, > Igor Stasenko. > > > -- Best regards, Igor Stasenko.

Nov. 16, 2014

Re: [Pharo-dev] [NB] NativeBoost and optCoerceNilToNull

by Max Leske

> On 15.11.2014, at 22:28, Igor Stasenko <siguctua(a)gmail.com> wrote: > > > > On 15 November 2014 15:53, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: > Once again Iâm struggling with NativeBoost. Iâm trying to use optCoerceNilToNull with this function: > > ^self call: #(LGitReturnCodeEnum git_commit_create( > LGitId * theCommitId, > LGitRepository repo, > String update_ref, > LGitSignature * theAuthor, > LGitSignature * theCommitter, > String message_encoding, > String theMessage, > LGitTree theTree, > int parent_count, > LGitCommit * parentsPointer)) options: #( optCoerceNilToNull ) > > The last argument can be NULL, per documentation, (no parents). > LGitCommit is a subclass of NBExternalObject and apparently NBExternalObjctType>>pushAsPointer: does *not* use the optCoerceNilToNull option (the super implementation in NBExternalType>>pushAsPointer: however does). The upshot of course is that the call fails with âan instance of LGitCommit expectedâ. > > My questions: > - The documentation says: > > "#optCoerceNilToNull" > "passing nil as a pointer-type argument, converts it to C NULL â > > but apparently this doesnât apply to every pointer type. Should the documentation be updated? > - Why does NBExternlObjectType not use optCoerceNilToNull? I donât want to use tricks to pass null if I donât have to. > > perhaps because it is not there? :) > So, lets add it. Yay! :) I have zero assembly skillsâ¦ Iâve tried the following but then get an âundefined label done18â error: NBExternalObject>>pushAsPointer: gen "push a pointer to handle value" | asm proxy oop notNil done | proxy := gen proxy. asm := gen asm. <<<<<<<<<<<<<< new done := asm uniqueLabelName: 'doneâ. >>>>>>>>>>>>>>> new oop := gen reserveTemp. loader emitLoad: gen to: oop. <<<<<<<<<<<<<< new "handle nils, if we care" gen optCoerceNilToNull ifTrue: [ notNil := asm uniqueLabelName: 'notNil'. proxy nilObject. asm cmp: asm EAX with: oop; jne: notNil; xor: asm EAX with: asm EAX; jmp: done; label: notNil ] ifFalse: [ "we can skip class verification, if loader loads receiver, since nothing to fear there" loader isReceiver ifFalse: [ self verifyClassOf: oop is: objectClass generator: gen. ] ]. >>>>>>>>>>>>>>> new proxy fetchPointer: (self handleIvarIndex) ofObject: oop. "handle ivar" proxy varBytesFirstFieldOf: asm EAX. "handle value ptr" asm push: asm EAX. gen releaseTemps: 1. > > - Related: there are instances where even non-pointer arguments are allowed to be NULL. Why does optCoerceNilToNull only work for pointer types? > > > like what? Suppose the argument type is a structure, e.g > "struct foo bar", that is non-pointer type. What in this case the NULL value for the structure? Ok, I think I understand. Are you saying that, since NULL is a macro, you have no way of telling during runtime how NULL is defined? For a pointer it is simply 0 or (void *) 0, but for a non pointer it depends on the context. Is that about right? > > Igor! Help! :) > > Cheers, > Max > > > > > -- > Best regards, > Igor Stasenko.

Nov. 16, 2014

Re: [Pharo-dev] [smalltalk-research] Re: Julia vs Pharo

by S Krish

Great read: Picking some elements from Julia might take lesser time.. I pick on at a first look: 1. Express: http://bogumilkaminski.pl/files/julia_express.pdf Quick deep dive in 1 hour..! Also: http://forio.com/labs/julia-studio/tutorials/advanced/1/ .. cool collection of tutorials following it just teaches everything about Julia gripping one like a fiction novel.. 2. Data Frame: http://www.r-tutor.com/r-introduction/data-frame 3. Single package.. : all of these, perhaps auto pulls in four packages.. "reading a delimited file / xls into an array, reading a delimited file/ xls into a DataFrame and accessing databases using ODBC." readdlm, readcsv: read from file.. writedlm, writecsv: read from file.. 4) Readability of a whole file: Remember Peter Deutsch's comment over a dinner: Smalltalk does not have readability over the whole class in one go.. Now if we can also have a display in our Nautilus browser of the whole class in one read only browse mode with all methods as in Julia: // all variables classVariables = classVar1 , classVar2... instanceVariables = instVar1 , instVar2 .. // all class side.. class function init() .... end //all instance side function inst1() ..... end ** Easy interface with C / Fortran / Python.. pyplot for instance.. and JavaCall with in proc JVM.. On Sun, Nov 16, 2014 at 9:12 PM, Serge Stinckwich < serge.stinckwich(a)gmail.com> wrote: > On Sun, Nov 16, 2014 at 1:45 AM, Alexandre Bergel > <alexandre.bergel(a)me.com> wrote: > > Hi Serge! > > > > Thanks for sharing this. > > > > I indeed believe that scientific computing will play a big role in Pharo > adoption by a large public > > The problem is that you need to invest at least 5 years of work before > having some impact in this domain. > > -- > Serge Stinckwich > UCBN & UMI UMMISCO 209 (IRD/UPMC) > Help fight Ebola by joining the Computing for Ebola Challenge > http://bit.ly/1oEdBag > >

Nov. 16, 2014

Reproducible VM crash with excessive garbage collection

by Max Leske

Hi. I may have discovered a bug in the PharoVM (stable and latest), which may have been around for at least two years (I had a very similiar situation about two years ago with Fuel). I have a test that fails when a particular method contains â3 timesRepeat: [ Smalltalk garbageCollect ]â. In order to be sure that itâ not the #timesRepeat message thatâs causing the crash, I replaced that with multiple âSmalltalk garbageCollectâ sends. The test Iâm talking about ObsoleteTest>>testFixObsoleteSharedPools. To make the test crash the VM, simply add a couple of garbageCollects at the end of SmalltalkImage>>fixObsoleteSharedPools. I tried to come up with a little script to demonstrate the issue but I can only reproduce the problem with Nautilus (which makes me suspect that the problem is related to Nautilus). Steps to reproduce: 1. get a fresh image and the latest vm: curl get.pharo.org/40+vmLastest <http://get.pharo.org/40+vmLastest> | bash && ./pharo-ui Pharo.image 2. open the class browser and navigate to SmalltalkImage>>fixObsoleteSharedPools 3. Append ten âself garbageCollectâ statements to the method. 4. navigate to ObsoleteTest>>testFixObsoleteSharedPools and click on the test runner icon to the left of the method name. I can reproduce this with the Mac VM (OS X 10.10) as well as with the linux VM (Ubuntu 12.04.1). Iâve opened an issue for this: https://pharo.fogbugz.com/f/cases/14478/Reproducible-VM-crash-with-excessiv… <https://pharo.fogbugz.com/f/cases/14478/Reproducible-VM-crash-with-excessiv…> Cheers, Max

Nov. 16, 2014

Re: [Pharo-dev] [pharo-project/pharo-core] 0635d1: 40371

by Ben Coman

Marcus Denker wrote: > >> On 16 Nov 2014, at 16:47, Marcus Denker <marcus.denker(a)inria.fr >> <mailto:marcus.denker@inria.fr>> wrote: >> >>> >>> On 16 Nov 2014, at 16:31, Marcus Denker <marcus.denker(a)inria.fr >>> <mailto:marcus.denker@inria.fr>> wrote: >>> >>> >>>>> Log Message: >>>>> ----------- >>>>> 40371 >>>>> 14261 Delay refactoring (part 1) - move scheduler from class-side >>>>> to instance-side of its own class >>>>> https://pharo.fogbugz.com/f/cases/14261 >>>>> http://files.pharo.org/image/40/40371.zip >>>> >>>> Thanks very much for that, but it picked up the .4 slice >>>> rather than the .9 slice. Probably due to my changing the issue >>>> title half way through (sorry). >>>> >>> Ok, I will do a manual integration of .9.. good that it does not need >>> a roll-back. >>> >> >> The result is that two tests are failing now in 372: >> >> KernelTests.Processes.SemaphoreTest.testWaitAndWaitTimeoutTogether >> KernelTests.Processes.SemaphoreTest.testWaitTimeoutMSecs >> > > And the step 3 is failing. > > https://ci.inria.fr/pharo/job/Pharo-4.0-Update-Step-3-Minimal/223/ > > I give up for today. > > I think we should revert both 372 and 371â¦ but I will not do that today > as this is sunday. > (lesson learned: never ingrate if you canât spend 3 hoursâ¦) > > Marcus I can confirm those failing tests, in both slice.4 and slice.9, so strange they didn't chow up in the Validator report. Now if its less work, you can revert to the original code by doing "Delay newCodeEnabled: false", and both tests will pass. It late here, so I'll work on fixing those tests tomorrow. Thanks for your efforts. cheers -ben

Nov. 16, 2014

Re: [Pharo-dev] [pharo-project/pharo-core] 0635d1: 40371

by Marcus Denker

> On 16 Nov 2014, at 16:47, Marcus Denker <marcus.denker(a)inria.fr> wrote: > >> >> On 16 Nov 2014, at 16:31, Marcus Denker <marcus.denker(a)inria.fr> wrote: >> >> >>>> Log Message: >>>> ----------- >>>> 40371 >>>> 14261 Delay refactoring (part 1) - move scheduler from class-side to instance-side of its own class >>>> https://pharo.fogbugz.com/f/cases/14261 >>>> http://files.pharo.org/image/40/40371.zip >>> >>> Thanks very much for that, but it picked up the .4 slice >>> rather than the .9 slice. Probably due to my changing the issue title half way through (sorry). >>> >> Ok, I will do a manual integration of .9.. good that it does not need a roll-back. >> > > The result is that two tests are failing now in 372: > > KernelTests.Processes.SemaphoreTest.testWaitAndWaitTimeoutTogether > KernelTests.Processes.SemaphoreTest.testWaitTimeoutMSecs > And the step 3 is failing. https://ci.inria.fr/pharo/job/Pharo-4.0-Update-Step-3-Minimal/223/ I give up for today. I think we should revert both 372 and 371â¦ but I will not do that today as this is sunday. (lesson learned: never ingrate if you canât spend 3 hoursâ¦) Marcus

Nov. 16, 2014

Re: [Pharo-dev] [pharo-project/pharo-core] 0635d1: 40371

by Marcus Denker

> On 16 Nov 2014, at 16:31, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > >>> Log Message: >>> ----------- >>> 40371 >>> 14261 Delay refactoring (part 1) - move scheduler from class-side to instance-side of its own class >>> https://pharo.fogbugz.com/f/cases/14261 >>> http://files.pharo.org/image/40/40371.zip >> >> Thanks very much for that, but it picked up the .4 slice >> rather than the .9 slice. Probably due to my changing the issue title half way through (sorry). >> > Ok, I will do a manual integration of .9.. good that it does not need a roll-back. > The result is that two tests are failing now in 372: KernelTests.Processes.SemaphoreTest.testWaitAndWaitTimeoutTogether KernelTests.Processes.SemaphoreTest.testWaitTimeoutMSecs Marcus

Nov. 16, 2014

Re: [Pharo-dev] [smalltalk-research] Re: Julia vs Pharo

by Serge Stinckwich

On Sun, Nov 16, 2014 at 1:45 AM, Alexandre Bergel <alexandre.bergel(a)me.com> wrote: > Hi Serge! > > Thanks for sharing this. > > I indeed believe that scientific computing will play a big role in Pharo adoption by a large public The problem is that you need to invest at least 5 years of work before having some impact in this domain. -- Serge Stinckwich UCBN & UMI UMMISCO 209 (IRD/UPMC) Help fight Ebola by joining the Computing for Ebola Challenge http://bit.ly/1oEdBag

Nov. 16, 2014

Re: [Pharo-dev] [pharo-project/pharo-core] 0635d1: 40371

by Marcus Denker

>> Log Message: >> ----------- >> 40371 >> 14261 Delay refactoring (part 1) - move scheduler from class-side to instance-side of its own class >> https://pharo.fogbugz.com/f/cases/14261 >> http://files.pharo.org/image/40/40371.zip > > Thanks very much for that, but it picked up the .4 slice > rather than the .9 slice. Probably due to my changing the issue title half way through (sorry). > Ok, I will do a manual integration of .9.. good that it does not need a roll-back. Marcus

Nov. 16, 2014