November 2017 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Why 2017 sizeInMemory = 0?

by Marcus Denker

> > #sizeInMemory only takes into account object header size and the size of the object > itself. > > sizeInMemory is also ignorant of details such as the padding of objects to 64-bits in both 32-bit and 64-bit Spur, the overflow header when there are more than 254 slots, etc. siseInMemory should really be implemented like this: > Thanks, I added an issue tracker entry: https://pharo.fogbugz.com/f/cases/20663/improve-sizeInMemory <https://pharo.fogbugz.com/f/cases/20663/improve-sizeInMemory> Marcus

Nov. 9, 2017

Re: [Pharo-dev] You can cheat in FFI as long as you don't get caught...

by Igor Stasenko

On 9 November 2017 at 12:03, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > yeah, but this is because LGitDiff, LGitRepository and LGitTree (and many > others) are defined as FFIExternalObjects while they should be > FFIOpaqueObjects. Then, the problem is that external objects are *already* > pointers (they have arity 1). > > hmm but the external objects were already designed with thought that it holding some opaque handle to some external object. it does not neccessary should be a pointer .. it could be some handle, that doesn't points directly to some memory location (like in windows api etc). they only difference is only in what function(s) expecting, e.g. if it takes a handle value - you should pass it by value (the handle itself), and when it asks for pointer to the handle (to fill a return value , for instance) - you passing a pointer to it. > this are remaining of old nativeboost and we created the opaque objects to > not need this âcheatingââ¦ just, I wanted to change them but is a lot of > work and I didnât have the time. > > I agree it stinks. > > Esteban > > On 8 Nov 2017, at 19:06, Nicolas Cellier <nicolas.cellier.aka.nice@ > gmail.com> wrote: > > I was trying to inquire about my latest vm crash in libgit2 > > https://pharo.fogbugz.com/f/cases/20655/vm-crash-in- > libgit-git_tree_free-throw-suspiscion-on-potential-double-free-problem > > when I found this bizarre prototype: > > ^ self > callUnchecked: > #(LGitReturnCodeEnum git_diff_tree_to_tree #(LGitDiff * diff > , LGitRepository repo , LGitTree old_tree , LGitTree new_tree , > LGitDiffOptions * opts)) > options: #() > > While the libgit2 signature differs in indirection levels: > https://libgit2.github.com/libgit2/#v0.19.0/group/diff/ > git_diff_tree_to_tree > > int git_diff_tree_to_tree(git_diff_list **diff, git_repository *repo, > git_tree *old_tree, git_tree *new_tree, const git_diff_options *opts); > > The fact that it does not differs for opts, made me think of a bug... > But no. > > opts is allocated on external c heap with: > > callUnchecked: #(LGitReturnCodeEnum git_diff_init_options(LGitDiffOptions > * self, LGitOptionsVersionsEnum version)) > > int git_diff_init_options(git_diff_options *opts, unsigned int version); > > What you see this time is that signatures match... > So the LGitDiffOptions handle will be an ExternalAddress that will really > point on a git_diff_options > In other words, the handle is a git_diff_options *... > IOW, our LGitDiffOptions object points to an external it_diff_options so > it is what it promises to be. > > For the other structures not so. > We are lying on the level of indirection, so our LGitTree handle are not > really a git_tree *. They are a git_tree **. (an ExternalAddress on a > pointer). > > As long as we lie consistently, it's OK, because it avoids too much > packToArity: unpackToArity: dance. > > That's the "beauty" of C pointers. > We can cast them to whatever or pretend they are handle_for_anything * in > the intermediate level, as long as original producer and final consumer > agree on what it really points to. > > But from code quality perspective, it really stinks. Anyone like me > opening a page to get the exact signature of the function will be > scratching head and loose precious time. Especially when tracking vm crash > down. > > I'm not sure how well it's documented, I presume it's a well known > conscious hack from the original developpers, but such practice really > ought to be discussed here. > > > > > -- Best regards, Igor Stasenko.

Nov. 9, 2017

Re: [Pharo-dev] You can cheat in FFI as long as you don't get caught...

by Igor Stasenko

Yes. Signatures of C functions often tells little about correct use of arguments, when there are pointers. And true, without checking the docs & use example, you cannot really make it work correctly, if you will rely only on signature when designing the call to such function(s). It is, in fact, orthogonal to FFI .. it is the way it works in C.. But what is good from FFI part, is that you can design a convenient interface to API that having such functions, in a way, that your users don't need to scratch their heads and stumble upon the code, asking , - hey.. wtf. That's why i said many times, that yes, it is cool to have automatic FFI wrapper generators, but they will never be as good as well thought and designed manually written interfaces to libraries, providing features of a library without burden of strange, bizarre and inconvenient way to pass data between user and library. On 9 November 2017 at 00:06, Nicolas Cellier < nicolas.cellier.aka.nice(a)gmail.com> wrote: > I was trying to inquire about my latest vm crash in libgit2 > > https://pharo.fogbugz.com/f/cases/20655/vm-crash-in- > libgit-git_tree_free-throw-suspiscion-on-potential-double-free-problem > > when I found this bizarre prototype: > > ^ self > callUnchecked: > #(LGitReturnCodeEnum git_diff_tree_to_tree #(LGitDiff * diff > , LGitRepository repo , LGitTree old_tree , LGitTree new_tree , > LGitDiffOptions * opts)) > options: #() > > While the libgit2 signature differs in indirection levels: > https://libgit2.github.com/libgit2/#v0.19.0/group/diff/ > git_diff_tree_to_tree > > int git_diff_tree_to_tree(git_diff_list **diff, git_repository *repo, > git_tree *old_tree, git_tree *new_tree, const git_diff_options *opts); > > The fact that it does not differs for opts, made me think of a bug... > But no. > > opts is allocated on external c heap with: > > callUnchecked: #(LGitReturnCodeEnum git_diff_init_options(LGitDiffOptions > * self, LGitOptionsVersionsEnum version)) > > int git_diff_init_options(git_diff_options *opts, unsigned int version); > > What you see this time is that signatures match... > So the LGitDiffOptions handle will be an ExternalAddress that will really > point on a git_diff_options > In other words, the handle is a git_diff_options *... > IOW, our LGitDiffOptions object points to an external it_diff_options so > it is what it promises to be. > > For the other structures not so. > We are lying on the level of indirection, so our LGitTree handle are not > really a git_tree *. They are a git_tree **. (an ExternalAddress on a > pointer). > > As long as we lie consistently, it's OK, because it avoids too much > packToArity: unpackToArity: dance. > > That's the "beauty" of C pointers. > We can cast them to whatever or pretend they are handle_for_anything * in > the intermediate level, as long as original producer and final consumer > agree on what it really points to. > > But from code quality perspective, it really stinks. Anyone like me > opening a page to get the exact signature of the function will be > scratching head and loose precious time. Especially when tracking vm crash > down. > > I'm not sure how well it's documented, I presume it's a well known > conscious hack from the original developpers, but such practice really > ought to be discussed here. > > > > -- Best regards, Igor Stasenko.

Nov. 9, 2017

[Pharo 7.0-dev] Build #271: 20657-Command-B-in-playground-does-not-work-the-save-way-as-printit

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #271 was: SUCCESS. The Pull Request #450 was integrated: "20657-Command-B-in-playground-does-not-work-the-save-way-as-printit" Pull request url: https://github.com/pharo-project/pharo/pull/450 Issue Url: https://pharo.fogbugz.com/f/cases/20657 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 9, 2017

[Pharo 7.0-dev] Build #270: 20656-Nautilus-comment-panel-got-broken-by-asLayoutFrame-cleans

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #270 was: SUCCESS. The Pull Request #449 was integrated: "20656-Nautilus-comment-panel-got-broken-by-asLayoutFrame-cleans" Pull request url: https://github.com/pharo-project/pharo/pull/449 Issue Url: https://pharo.fogbugz.com/f/cases/20656 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

Nov. 9, 2017

Re: [Pharo-dev] You can cheat in FFI as long as you don't get caught...

by Esteban Lorenzano

yeah, but this is because LGitDiff, LGitRepository and LGitTree (and many others) are defined as FFIExternalObjects while they should be FFIOpaqueObjects. Then, the problem is that external objects are *already* pointers (they have arity 1). this are remaining of old nativeboost and we created the opaque objects to not need this âcheatingââ¦ just, I wanted to change them but is a lot of work and I didnât have the time. I agree it stinks. Esteban > On 8 Nov 2017, at 19:06, Nicolas Cellier <nicolas.cellier.aka.nice(a)gmail.com <mailto:nicolas.cellier.aka.nice@gmail.com>> wrote: > > I was trying to inquire about my latest vm crash in libgit2 > > https://pharo.fogbugz.com/f/cases/20655/vm-crash-in-libgit-git_tree_free-th… <https://pharo.fogbugz.com/f/cases/20655/vm-crash-in-libgit-git_tree_free-th…> > > when I found this bizarre prototype: > > ^ self > callUnchecked: > #(LGitReturnCodeEnum git_diff_tree_to_tree #(LGitDiff * diff , LGitRepository repo , LGitTree old_tree , LGitTree new_tree , LGitDiffOptions * opts)) > options: #() > > While the libgit2 signature differs in indirection levels: > https://libgit2.github.com/libgit2/#v0.19.0/group/diff/git_diff_tree_to_tree <https://libgit2.github.com/libgit2/#v0.19.0/group/diff/git_diff_tree_to_tree> > > int git_diff_tree_to_tree(git_diff_list **diff, git_repository *repo, git_tree *old_tree, git_tree *new_tree, const git_diff_options *opts); > > The fact that it does not differs for opts, made me think of a bug... > But no. > > opts is allocated on external c heap with: > > callUnchecked: #(LGitReturnCodeEnum git_diff_init_options(LGitDiffOptions * self, LGitOptionsVersionsEnum version)) > > int git_diff_init_options(git_diff_options *opts, unsigned int version); > > What you see this time is that signatures match... > So the LGitDiffOptions handle will be an ExternalAddress that will really point on a git_diff_options > In other words, the handle is a git_diff_options *... > IOW, our LGitDiffOptions object points to an external it_diff_options so it is what it promises to be. > > For the other structures not so. > We are lying on the level of indirection, so our LGitTree handle are not really a git_tree *. They are a git_tree **. (an ExternalAddress on a pointer). > > As long as we lie consistently, it's OK, because it avoids too much packToArity: unpackToArity: dance. > > That's the "beauty" of C pointers. > We can cast them to whatever or pretend they are handle_for_anything * in the intermediate level, as long as original producer and final consumer agree on what it really points to. > > But from code quality perspective, it really stinks. Anyone like me opening a page to get the exact signature of the function will be scratching head and loose precious time. Especially when tracking vm crash down. > > I'm not sure how well it's documented, I presume it's a well known conscious hack from the original developpers, but such practice really ought to be discussed here. > > >

Nov. 9, 2017

Re: [Pharo-dev] [IMPORTANT] Is there a bug in Tonel with category:

by Denis Kudriashov

2017-11-08 21:29 GMT+01:00 Stephane Ducasse <stepharo.self(a)gmail.com>: > Sven > > I understand but it is SUPER strange to save the code in a format that > does not respect the syntax of the language. > I think that we should write a parser for the class definition or save > category as strings. > > Then should not we replace STON based class definition with actual smalltalk based definition? Tonel could parse class definition in similar way like it parses methods. > I do not want to have to explain to people. Yes and no this is not the > pharo syntax. > > Sorry I'm fed up. > > Stef > > On Mon, Nov 6, 2017 at 10:25 PM, Sven Van Caekenberghe <sven(a)stfx.eu> > wrote: > > > > > >> On 6 Nov 2017, at 20:52, Stephane Ducasse <stepharo.self(a)gmail.com> > wrote: > >> > >> Esteban told me that this is because he uses STON for the class > definition. > >> Now a simple question may be we could use a string instead of a symbol > >> because this is strange > >> to have a class definitino that does not respect Pharo syntax. > > > > It might be a bit confusing at first sight, but it is correct, IMHO. > > > > It is STON syntax, not Pharo. > > > > Yes, STON is a bit more liberal with Symbols than normal Smalltalk, but > it is totally consistent with itself. > > > > STON fromString: (STON toString: #'My Strange Symbol'). > > STON fromString: (STON toString: #'foo-bar'). > > > > If a Symbol consists only of characters in ' > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_./' then > the Symbol does not need to be quoted in STON. > > > > I see no problem there. > > > >> Stef > >> > >> On Mon, Nov 6, 2017 at 8:15 PM, Dale Henrichs > >> <dale.henrichs(a)gemtalksystems.com> wrote: > >>> > >>> > >>> On 11/06/2017 08:23 AM, Sven Van Caekenberghe wrote: > >>>> > >>>> > >>>>> On 6 Nov 2017, at 17:13, Dale Henrichs < > dale.henrichs(a)gemtalksystems.com> > >>>>> wrote: > >>>>> > >>>>> > >>>>> > >>>>> On 11/6/17 7:07 AM, Sven Van Caekenberghe wrote: > >>>>>>> > >>>>>>> On 6 Nov 2017, at 15:43, Dale Henrichs > >>>>>>> <dale.henrichs(a)gemtalksystems.com> wrote: > >>>>>>> > >>>>>>> of course with Pharo's implementation of Symbol it is not > practical to > >>>>>>> use asString nor type checks - things that are not necessary in > other > >>>>>>> Smalltalk implementations > >>>>>> > >>>>>> How so ? > >>>>>> > >>>>>> What is the problem with Symbol>>#asString ? > >>>>> > >>>>> I am not going to go to every field in the api that is supposed to > be a > >>>>> String and add asString. There are too many places to worry about > ... I > >>>>> would prefer that Pharo be ANSI compliant:) > >>>>> > >>>>> It's not just Metacello. > >>>>> > >>>>> It's an annoying issue that has to be dealt with every time a Pharo > >>>>> application is ported to another dialect of Smalltalk and an annoying > >>>>> barrier for folks running on other dialects to move their > application to > >>>>> Pharo - in this case the bugs that are introduced by Pharo's > behavior with > >>>>> respect to Symbols can be very hard to diagnose -- > >>>>> > >>>>> Making things harder to share code between dialects is a bad thing > for > >>>>> Smalltalk overall -- just another reason for non-Smalltalk > programmers to > >>>>> question the whether they should use Smalltalk or not... > >>>>> > >>>>> And I don't need to hear about how Pharo is not Smalltalk:) > >>>>> > >>>>> Dale > >>>> > >>>> So there is nothing 'wrong', you just want Pharo to remain the same as > >>>> every other non-changing Smalltalk out there. > >>> > >>> Did I say that? > >>> > >>> I support the direction that Pharo is going, but I reserve the right to > >>> disagree with some of the details. > >>> > >>> This is just one detail ... nothing more nothing less ... For those of > us > >>> that work in multiple dialects, it IS annoying and I take an > opportunity > >>> every year or so to remind you guys of the things that I find > annoying, just > >>> to keep you guys honest:) > >>> > >>>> > >>>> From one perspective you are right, it makes some cross platform > porting > >>>> in either direction harder. Seaside has many rules to help > portability. Not > >>>> mixing Strings and Symbol is probably one of them. > >>> > >>> ... and as I mentioned, this problem can be one of the more annoying > issues > >>> to track down, when a developer is not careful ... Honestly there are > two > >>> sides to the issue ... when developers use Symbols in tests to drive > an API > >>> that is supposed to use Strings (this happens the most often) things > break > >>> pretty quickly and the tests can be fixed pretty easily ... but when > the > >>> code itself is written with mixed Symbols and Strings, the tests might > >>> actually pass after the port, and the bugs will only show up in subtle > cases > >>> ... I've hit a handful of these over the years and they are hard to > track > >>> down... > >>>> > >>>> > >>>> But you know very well that Pharo was started so that we would be > able to > >>>> make changes, in any area or aspect of the system, without the burden > of > >>>> backwards or cross platform compatibility, even if some of these > changes are > >>>> taste based. > >>> > >>> Agree with your statement -- most of the changes that Pharo has made > have > >>> not been difficult to accommodate, but Symbol/String is at a > fundamental > >>> level and I'm not sure that it would be "illegal" to make this > accommodation > >>> --- I AM pretty certain that it would cause some short term pain, but > >>> probably no more pain (and likely less pain) than is caused by trying > to > >>> move an application to a new version of Pharo:) > >>>> > >>>> And I happen to like the ability to mix and match Strings and Symbols > (we > >>>> discussed about this before). > >>>> > >>> I won't argue with taste, it's is simply the portability for this > particular > >>> problem that I am highlighting ... > >>> > >>> Dale > >>> > >> > > > > > >

Nov. 9, 2017

Re: [Pharo-dev] Is fileout on the change sorter working in Pharo 70?

by Guillermo Polito

On Thu, Nov 9, 2017 at 10:35 AM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > Hi > > 2017-11-08 21:53 GMT+01:00 Stephane Ducasse <stepharo.self(a)gmail.com>: > >> It looks like the problem is here but I cannot find it. >> >> CodeExporter>> writeSourceCodeFrom: aStream baseName: baseName isSt: >> stOrCsFlag >> >> | extension fileName outputStream | >> self halt. >> extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. >> fileName := baseName, extension. >> fileName := FileSystem disk checkName: fileName fixErrors: true. >> outputStream := (File named: fileName) writeStream. >> (ZnCrPortableWriteStream on: (ZnCharacterWriteStream >> on: outputStream >> encoding: 'utf8')) nextPutAll: aStream contents. >> >> outputStream close. >> >> self inform: 'Filed out to: ', String cr, fileName. > > > Try rewrite code to FileSystem. I think we should never use File directly: > > extension := stOrCsFlag ifTrue: ['st'] ifFalse: ['cs']. > FileLocator home / baseName , extension writeStreamDo: [:s | > > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > > on: s > > encoding: 'utf8')) nextPutAll: aStream contents]. > > We should choose what to use instead of #home in that case. > I just discussed a point with Denis, just replicating it here. We should not **blindly** rewrite everything to use FileSystem. Because that may break dependencies in the system. For example, CodeExporter is used by Monticello. Should monticello depend on FileSystem or not? I'm not saying monticello should or should not depend on it. Just saying that we should ask ourselves the question :) > > I wondering does File is actually used by FileSystem? > It will ;) > Should not we remove it from system? > No! File is exactly the low level File representation, that FileSystem should use. > > >> >> On Wed, Nov 8, 2017 at 9:38 PM, Stephane Ducasse >> <stepharo.self(a)gmail.com> wrote: >> > Ah thanks we should fix this. >> > >> > On Wed, Nov 8, 2017 at 9:31 PM, Christian Kellermann >> > <ckeen(a)pestilenz.org> wrote: >> >> * Stephane Ducasse <stepharo.self(a)gmail.com> [171108 21:29]: >> >>> Hi >> >>> >> >>> I'm trying to rescue some of my work (that I cannot save in MC nor in >> >>> Iceberg because the system >> >>> does not see it - super funny). >> >>> >> >>> I can browse the change using the changesorter when I do fileout >> >>> the system tells me that the file it saved but I cannot find it on my >> disc. >> >>> Apparently none of the fileout is working in the changesorter. :( >> >> >> >> I have successfully filed out code with epicea but not changesorter. >> >> >> >> HTH, >> >> >> >> Christian >> >> >> >> -- >> >> May you be peaceful, may you live in safety, may you be free from >> >> suffering, and may you live with ease. >> >> >> >> > -- Guille Polito Research Engineer Centre de Recherche en Informatique, Signal et Automatique de Lille CRIStAL - UMR 9189 French National Center for Scientific Research - *http://www.cnrs.fr <http://www.cnrs.fr>* *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

Nov. 9, 2017

Re: [Pharo-dev] Is fileout on the change sorter working in Pharo 70?

by Denis Kudriashov

Hi 2017-11-08 21:53 GMT+01:00 Stephane Ducasse <stepharo.self(a)gmail.com>: > It looks like the problem is here but I cannot find it. > > CodeExporter>> writeSourceCodeFrom: aStream baseName: baseName isSt: > stOrCsFlag > > | extension fileName outputStream | > self halt. > extension := stOrCsFlag ifTrue: ['.st'] ifFalse: ['.cs']. > fileName := baseName, extension. > fileName := FileSystem disk checkName: fileName fixErrors: true. > outputStream := (File named: fileName) writeStream. > (ZnCrPortableWriteStream on: (ZnCharacterWriteStream > on: outputStream > encoding: 'utf8')) nextPutAll: aStream contents. > > outputStream close. > > self inform: 'Filed out to: ', String cr, fileName. Try rewrite code to FileSystem. I think we should never use File directly: extension := stOrCsFlag ifTrue: ['st'] ifFalse: ['cs']. FileLocator home / baseName , extension writeStreamDo: [:s | (ZnCrPortableWriteStream on: (ZnCharacterWriteStream on: s encoding: 'utf8')) nextPutAll: aStream contents]. We should choose what to use instead of #home in that case. I wondering does File is actually used by FileSystem? Should not we remove it from system? > > On Wed, Nov 8, 2017 at 9:38 PM, Stephane Ducasse > <stepharo.self(a)gmail.com> wrote: > > Ah thanks we should fix this. > > > > On Wed, Nov 8, 2017 at 9:31 PM, Christian Kellermann > > <ckeen(a)pestilenz.org> wrote: > >> * Stephane Ducasse <stepharo.self(a)gmail.com> [171108 21:29]: > >>> Hi > >>> > >>> I'm trying to rescue some of my work (that I cannot save in MC nor in > >>> Iceberg because the system > >>> does not see it - super funny). > >>> > >>> I can browse the change using the changesorter when I do fileout > >>> the system tells me that the file it saved but I cannot find it on my > disc. > >>> Apparently none of the fileout is working in the changesorter. :( > >> > >> I have successfully filed out code with epicea but not changesorter. > >> > >> HTH, > >> > >> Christian > >> > >> -- > >> May you be peaceful, may you live in safety, may you be free from > >> suffering, and may you live with ease. > >> > >

Nov. 9, 2017

Re: [Pharo-dev] [IMPORTANT] Is there a bug in Tonel with category:

by Sven Van Caekenberghe

> On 9 Nov 2017, at 00:52, Martin McClure <martin(a)hand2mouse.com> wrote: > > On 11/08/2017 01:15 PM, Sven Van Caekenberghe wrote: >> What is the opinion of the original Tonel designers ? > > Well, Esteban did the largest part of the work, but I guess I am *one* of the original designers, in that various design proposals were floated back and forth, starting with one of mine and ending with Tonel. > > My intent is that Tonel, as a declarative cross-platform format, should use strings exclusively, never Symbols. I *think* I discussed this with Esteban during ESUG, and I *think* he agreed, so it seems likely that the current behavior with class definitions is unintentional. > > Regards, > > -Martin Thanks for your reply, Martin. Ah, OK, I didn't know that. Still, since the current Tonel implementation in Pharo uses a custom STONWriter subclass, it is easy to change Symbol printing (not that it is needed, but it would help in appearance). TonelSTONWriter>>#isSimpleSymbol: symbol "Customize STON to only consider very clean symbols as literal, for all others err on the safe side and quote them." symbol isEmpty ifTrue: [ ^ false ]. ('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ' includes: symbol first) ifFalse: [ ^ false ]. ^ symbol allSatisfy: [ :each | 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789' includes: each ] I would also make the following change TonelWriter>>#toSTON: anObject ^ String streamContents: [ :out | (TonelSTONWriter on: out) newLine: self newLine; nextPut: anObject ] Sven

Nov. 9, 2017