April 2018 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Idea and proof of concept: Accept for Test

by Sven Van Caekenberghe

Very cool. Impressive combination of unique Pharo features. > On 4 Apr 2018, at 18:02, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > Hi, > > Some code is very âactiveâ, executed all the time, e.g. collections, graphicsâ¦ or imagine if you work on the compiler or debugger. > > It would really be nice if we could test a change before accepting it. Something like âAccept for Testâ where magically the original method > does not change, yet, when running tests, the version of the code we accepted for testing is executed. > > So how do we implement that? > > 1) menu in the browser. I did a quick hack, a Suggestions AST menu shows for all nodes (to be replaced with the âcorrectâ way later). > > SugsSuggestion subclass: #SugsAcceptForTest > instanceVariableNames: '' > classVariableNames: '' > package: 'SmartSuggestions-Suggestionâ > > label > ^'Accept for testâ > > So, now we can do everything in the #execute method. > > What do we need? > > 1) How we know that we are in a test? â> CurrentExecutionEnvironment value isTest > 2) can we compile the buffer even if there are syntax errors (kind of what I expect to happen in non-accepted code)? > â> yes, remember, we use the same Parser for syntax highlighting and we extended code-generation for SyntaxError Nodes > in the past (it raises a syntax error at runtime, isnât that kind off odd in a fun way?). > > newMethod := context selectedClass compiler > source: context code; > options: #(+ optionParseErrors); > compile. > > 3) Now how to we âhook intoâ the execution of the original unchanged method? We use a MetaLink. We install it as a before link on > the RBMethodNode, the meta-object is a block. This block gets as arguments the thisContext and the arguments of the original method > and then just does what you want it do do: if we are in test, return the result of the execution of the method we just compiled. > (Closures are fun, we can just reference the outer temp newMethod, no need to store it anywhere else explicitly): > > [ :aContext :args | > CurrentExecutionEnvironment value isTest > ifTrue: [ aContext return: (newMethod valueWithReceiver: aContext receiver arguments: args) ] ]; > > Thus, all the code is this: > > execute > | newMethod metaLink | > âwe compile a new method from the not-accepted text editor. Allow syntax errors" > newMethod := context selectedClass compiler > source: context code; > options: #(+ optionParseErrors); > compile. > "the link executes the method we just created and returns" > metaLink := MetaLink new > metaObject: [ :aContext :args | > CurrentExecutionEnvironment value isTest > ifTrue: [ aContext return: (newMethod valueWithReceiver: aContext receiver arguments: args) ] ]; > selector: #value:value:; > arguments: #(context arguments). > context selectedMethod ast link: metaLink > > Cleanup we do not care as everything will be GCed as soon as we accept the method for real. > > And it works! I did not test it too much, but it seems to do what it is supposed to do. (but more tests and testing are for sure needed) > https://github.com/pharo-project/pharo/pull/1182 > > Next step: move it to a better place in the menu. > > Marcus > >

April 4, 2018

Re: [Pharo-dev] [Pharo7] halt inside and outside tests

by Sean P. DeNigris

Peter UhnÃ¡k wrote > Would it be possible to instead have a #haltIf object to which you can > send > additional conditions? Maybe we could use that as an excuse to clean more halting out of object completely e.g. self halt now self halt once self halt ifInsideTest self halt ifOutsideTest self halt xyz Although IIRC when we first started cleaning the idea was too eventually remove all halt-related stuff from Object and only have `Halt xyz`. At the time, it seemed that was a little too scary, but even today I do not understand what objectively is gained by `self halt` vs. `Halt now`â¦ ----- Cheers, Sean -- Sent from: http://forum.world.st/Pharo-Smalltalk-Developers-f1294837.html

April 4, 2018

[Pharo 7.0-dev] Build #754: 21674-Move-AbstractFileReferencehexDumponmax-to-ByteArray

by ci-pharo-ci-jenkins2＠inria.fr

There is a new Pharo build available! The status of the build #754 was: SUCCESS. The Pull Request #1179 was integrated: "21674-Move-AbstractFileReferencehexDumponmax-to-ByteArray" Pull request url: https://github.com/pharo-project/pharo/pull/1179 Issue Url: https://pharo.fogbugz.com/f/cases/21674 Build Url: https://ci.inria.fr/pharo-ci-jenkins2/job/Test%20pending%20pull%20request%2…

April 4, 2018

Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

by Stephane Ducasse

Thanks for this discussion. On Wed, Apr 4, 2018 at 1:37 PM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > Alistair, > > First off, thanks for the discussions and your contributions, I really appreciate them. > > But I want to have a discussion at the high level of the definition and semantics of the stream API in Pharo. > >> On 4 Apr 2018, at 13:20, Alistair Grant <akgrant0710(a)gmail.com> wrote: >> >> On 4 April 2018 at 12:56, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> Playing a bit devil's advocate, the idea is that, in general, >>> >>> [ stream atEnd] whileFalse: [ stream next. "..." ]. >>> >>> is no longer allowed ? >> >> It hasn't been allowed "forever" [1]. It's just been misused for >> almost as long. >> >> [1] Time began when stdio stream support was introduced. :-) > > I am still not convinced. Another way to put it would be that the old #atEnd or #upToEnd do not make sense for these streams and some new loop is needed, based on a new test (it exists for socket streams already). > > [ stream isDataAvailable ] whileTrue: [ stream next ] > >>> And you want to replace it with >>> >>> [ stream next ifNil: [ false ] ifNotNil: [ :x | "..." true ] whileTrue. >>> >>> That is a pretty big change, no ? >> >> That's the way quite a bit of code already operates. >> >> As Denis pointed out, it's obviously problematic in the general sense, >> since nil can be embedded in non-byte oriented streams. I suspect >> that in practice not many people write code that reads streams from >> both byte oriented and non-byte oriented streams. > > Maybe yes, maybe no. As Denis' example shows there is a clear definition problem. > > And I do use streams of byte arrays or strings all the time, this is really important. I want my parsers to work on all kinds of streams. > >>> I think/feel like a proper EOF exception would be better, more correct. >>> >>> [ [ stream next. "..." true ] on: EOF do: [ false ] ] whileTrue. >> >> I agree, but the email thread Nicolas pointed to raises some >> performance questions about this approach. It should be >> straightforward to do a basic performance comparison which I'll get >> around to if other objections aren't raised. > > Reading in bigger blocks, using #readInto:startingAt:count: (which is basically Unix's (2) Read sys call), would solve performance problems, I think. > >>> Will we throw away #atEnd then ? Do we need it if we cannot use it ? >> >> Unix file i/o returns EOF if the end of file has been reach OR if an >> error occurs. You should still check #atEnd after reading past the >> end of the file to make sure no error occurred. Another part of the >> primitive change I'm proposing is to return additional information >> about what went wrong in the event of an error. > > I am sorry, but this kind of semantics (the OR) is way too complex at the general image level, it is too specific and based on certain underlying implementation details. > > Sven > >> We could modify the read primitive so that it fails if an error has >> occurred, and then #atEnd wouldn't be required. >> >> Cheers, >> Alistair >> >> >> >>>> On 4 Apr 2018, at 12:41, Alistair Grant <akgrant0710(a)gmail.com> wrote: >>>> >>>> Hi Nicolas, >>>> >>>> On 4 April 2018 at 12:36, Nicolas Cellier >>>> <nicolas.cellier.aka.nice(a)gmail.com> wrote: >>>>> >>>>> >>>>> 2018-04-04 12:18 GMT+02:00 Alistair Grant <akgrant0710(a)gmail.com>: >>>>>> >>>>>> Hi Sven, >>>>>> >>>>>> On Wed, Apr 04, 2018 at 11:32:02AM +0200, Sven Van Caekenberghe wrote: >>>>>>> Somehow, somewhere there was a change to the implementation of the >>>>>>> primitive called by some streams' #atEnd. >>>>>> >>>>>> That's a proposed change by me, but it hasn't been integrated yet. So >>>>>> the discussion below should apply to the current stable vm (from August >>>>>> last year). >>>>>> >>>>>> >>>>>>> IIRC, someone said it is implemented as 'remaining size being zero' >>>>>>> and some virtual unix files like /dev/random are zero sized. >>>>>> >>>>>> Currently, for files other than sdio (stdout, stderr, stdin) it is >>>>>> effectively defined as: >>>>>> >>>>>> atEnd := stream position >= stream size >>>>>> >>>>>> >>>>>> And, as you say, plenty of virtual unix files report size 0. >>>>>> >>>>>> >>>>>> >>>>>>> Now, all kinds of changes are being done image size to work around this. >>>>>> >>>>>> I would phrase this slightly differently :-) >>>>>> >>>>>> Some code does the right thing, while other code doesn't. E.g.: >>>>>> >>>>>> MultiByteFileStream>>upToEnd is good, while >>>>>> FileStream>>contents is incorrect >>>>>> >>>>>> >>>>>>> I am a strong believer in simple, real (i.e. infinite) streams, but I >>>>>>> am not sure we are doing the right thing here. >>>>>>> >>>>>>> Point is, I am not sure #next returning nil is official and universal. >>>>>>> >>>>>>> Consider the comments: >>>>>>> >>>>>>> Stream>>#next >>>>>>> "Answer the next object accessible by the receiver." >>>>>>> >>>>>>> ReadStream>>#next >>>>>>> "Primitive. Answer the next object in the Stream represented by the >>>>>>> receiver. Fail if the collection of this stream is not an Array or a >>>>>>> String. >>>>>>> Fail if the stream is positioned at its end, or if the position is out >>>>>>> of >>>>>>> bounds in the collection. Optional. See Object documentation >>>>>>> whatIsAPrimitive." >>>>>>> >>>>>>> Note how there is no talk about returning nil ! >>>>>>> >>>>>>> I think we should discuss about this first. >>>>>>> >>>>>>> Was the low level change really correct and the right thing to do ? >>>>>> >>>>>> The primitive change proposed doesn't affect this discussion. It will >>>>>> mean that #atEnd returns false (correctly) sometimes, while currently it >>>>>> returns true (incorrectly). The end result is still incorrect, e.g. >>>>>> #contents returns an empty string for /proc/cpuinfo. >>>>>> >>>>>> You're correct about no mention of nil, but we have: >>>>>> >>>>>> FileStream>>next >>>>>> >>>>>> (position >= readLimit and: [self atEnd]) >>>>>> ifTrue: [^nil] >>>>>> ifFalse: [^collection at: (position := position + 1)] >>>>>> >>>>>> >>>>>> which has been around for a long time (I suspect, before Pharo existed). >>>>>> >>>>>> Having said that, I think that raising an exception is a better >>>>>> solution, but it is a much, much bigger change than the one I proposed >>>>>> in https://github.com/pharo-project/pharo/pull/1180. >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Alistair >>>>>> >>>>> >>>>> Hi, >>>>> yes, if you are after universal behavior englobing Unix streams, the >>>>> Exception might be the best way. >>>>> Because on special stream you can't allways say in advance, you have to try. >>>>> That's the solution adopted by authors of Xtreams. >>>>> But there is a runtime penalty associated to it. >>>>> >>>>> The penalty once was so high that my proposal to generalize EndOfStream >>>>> usage was rejected a few years ago by AndreaRaab. >>>>> http://forum.world.st/EndOfStream-unused-td68806.html >>>> >>>> Thanks for this, I'll definitely take a look. >>>> >>>> Do you have a sense of how Denis' suggestion of using an EndOfStream >>>> object would compare? >>>> >>>> It would keep the same coding style, but avoid the problems with nil. >>>> >>>> Thanks, >>>> Alistair >>>> >>>> >>>> >>>>> I have regularly benched Xtreams, but stopped a few years ago. >>>>> Maybe i can excavate and pass on newer VM. >>>>> >>>>> In the mean time, i had experimented a programmable end of stream behavior >>>>> (via a block, or any other valuable) >>>>> http://www.squeaksource.com/XTream.htm >>>>> so as to reconcile performance and universality, but it was a source of >>>>> complexification at implementation side. >>>>> >>>>> Nicolas >>>>> >>>>>> >>>>>> >>>>>>> Note also that a Guille introduced something new, #closed which is >>>>>>> related to the difference between having no more elements (maybe right now, >>>>>>> like an open network stream) and never ever being able to produce more data. >>>>>>> >>>>>>> Sven >>> >>> >> > >

April 4, 2018

Re: [Pharo-dev] [Pharo7] halt inside and outside tests

by Stephane Ducasse

Why not RIght now I would like to use them to see how they fit my process. On Wed, Apr 4, 2018 at 9:58 AM, Peter UhnÃ¡k <i.uhnak(a)gmail.com> wrote: > Would it be possible to instead have a #haltIf object to which you can send > additional conditions? > > E.g. > > self haltIf insideTest > self haltIf outsideTest > > Makes it (imho) a bit more readable, adds just a single method to Object, > and allows for more potential extensions... > > self haltIf nil. > self haltIf count: 3. > > etc? > > Peter > > On Wed, Apr 4, 2018 at 8:36 AM, Christophe Demarey > <christophe.demarey(a)inria.fr> wrote: >> >> >> > Le 4 avr. 2018 Ã 04:34, Sean P. DeNigris <sean(a)clipperadams.com> a Ã©crit >> > : >> > >> > demarey wrote >> >> What about haltIfTestExecutionEnvironment and >> >> haltIfDefaultExecutionEnvironment ? >> > >> > I thought the names were okay. What did you think was unclear about >> > them? >> >> At a first look #haltInsideTest and #haltOutsideTest do not tell me that >> they are conditional breakpoints. >> maybe just add a if would solve that: #haltIfInsideTest #haltIfOutsideTest > >

April 4, 2018

Re: [Pharo-dev] [Pharo7] halt inside and outside tests

by Stephane Ducasse

This is fun but I was confused and when I read with the if it makes a lot more sense. Tx christophe On Wed, Apr 4, 2018 at 8:36 AM, Christophe Demarey <christophe.demarey(a)inria.fr> wrote: > >> Le 4 avr. 2018 Ã 04:34, Sean P. DeNigris <sean(a)clipperadams.com> a Ã©crit : >> >> demarey wrote >>> What about haltIfTestExecutionEnvironment and >>> haltIfDefaultExecutionEnvironment ? >> >> I thought the names were okay. What did you think was unclear about them? > > At a first look #haltInsideTest and #haltOutsideTest do not tell me that they are conditional breakpoints. > maybe just add a if would solve that: #haltIfInsideTest #haltIfOutsideTest

April 4, 2018

Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

by K K Subbu

On Wednesday 04 April 2018 09:07 PM, Sven Van Caekenberghe wrote: > Good summary, I agree. > > Still, what are the semantics of #next - does the caller always have > to check for nil ? Do we think this is ugly (as the return value is > outside the domain) ? Do we then still need #atEnd ? Senders of #next should be prepared to handle errors and EoS is just one of them. On exhausting all elements in the collection, #next could block waiting for next object to become available or check for errors. Reading past End is just one of the error conditions. One way to recover from EoS error is to return an sentinel. The exact sentinel object could depend on the collection. Eliot suggested a good one earlier in this thread. Regards .. Subbu

April 4, 2018

Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

by Eliot Miranda

On Wed, Apr 4, 2018 at 8:37 AM, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > > > On 4 Apr 2018, at 17:32, K K Subbu <kksubbu.ml(a)gmail.com> wrote: > > > > On Wednesday 04 April 2018 04:06 PM, Nicolas Cellier wrote: > >>> IIRC, someone said it is implemented as 'remaining size being zero' > >>> and some virtual unix files like /dev/random are zero sized. > >> Currently, for files other than sdio (stdout, stderr, stdin) it is > >> effectively defined as: > >> atEnd := stream position >= stream size > > I see a confusion between Stream and its underlying collection. Stream > is an iterator and just does next, nextPut, peek, reset etc. But methods > like size or atEnd depend on its collection and there is no guarantee that > this collection has a known and finite size. > > > > Essentially, a collection's size may be known and finite, unknown but > finite size or infinite. This has nothing do with file descriptor being > std{in,out,err}. If std* is a regular file or special file like /dev/mem, > /dev/null, it's size is known at open time. With console streams or sysfs > files, size is unknown until EOT (^D) or NUL is received. Lastly, special > files like /dev/zero, /dev/random or /proc/cpuinfo don't have a finite size > but report it as zero (!). > > > > [ stream atEnd ] whileFalse: [ stream next. .. ] > > > > will only terminate if its collection size is finite. It won't terminate > for infinite collections. > > > > Regards .. Subbu > > Good summary, I agree. > > Still, what are the semantics of #next - does the caller always have to > check for nil ? Do we think this is ugly (as the return value is outside > the domain) ? The problem is not that the value is outside the domain. The problem is that it may be within the domain. Answering an element within the domain when there are no more elements is insane. Not even C does that. > Do we then still need #atEnd ? > For backward compatibility, yes, even if deprecated. There is so much existing code that uses it, it'll be useful to have. > Sven > _,,,^..^,,,_ best, Eliot

April 4, 2018

Idea and proof of concept: Accept for Test

by Marcus Denker

Hi, Some code is very âactiveâ, executed all the time, e.g. collections, graphicsâ¦ or imagine if you work on the compiler or debugger. It would really be nice if we could test a change before accepting it. Something like âAccept for Testâ where magically the original method does not change, yet, when running tests, the version of the code we accepted for testing is executed. So how do we implement that? 1) menu in the browser. I did a quick hack, a Suggestions AST menu shows for all nodes (to be replaced with the âcorrectâ way later). SugsSuggestion subclass: #SugsAcceptForTest instanceVariableNames: '' classVariableNames: '' package: 'SmartSuggestions-Suggestionâ label ^'Accept for testâ So, now we can do everything in the #execute method. What do we need? 1) How we know that we are in a test? â> CurrentExecutionEnvironment value isTest 2) can we compile the buffer even if there are syntax errors (kind of what I expect to happen in non-accepted code)? â> yes, remember, we use the same Parser for syntax highlighting and we extended code-generation for SyntaxError Nodes in the past (it raises a syntax error at runtime, isnât that kind off odd in a fun way?). newMethod := context selectedClass compiler source: context code; options: #(+ optionParseErrors); compile. 3) Now how to we âhook intoâ the execution of the original unchanged method? We use a MetaLink. We install it as a before link on the RBMethodNode, the meta-object is a block. This block gets as arguments the thisContext and the arguments of the original method and then just does what you want it do do: if we are in test, return the result of the execution of the method we just compiled. (Closures are fun, we can just reference the outer temp newMethod, no need to store it anywhere else explicitly): [ :aContext :args | CurrentExecutionEnvironment value isTest ifTrue: [ aContext return: (newMethod valueWithReceiver: aContext receiver arguments: args) ] ]; Thus, all the code is this: execute | newMethod metaLink | âwe compile a new method from the not-accepted text editor. Allow syntax errors" newMethod := context selectedClass compiler source: context code; options: #(+ optionParseErrors); compile. "the link executes the method we just created and returns" metaLink := MetaLink new metaObject: [ :aContext :args | CurrentExecutionEnvironment value isTest ifTrue: [ aContext return: (newMethod valueWithReceiver: aContext receiver arguments: args) ] ]; selector: #value:value:; arguments: #(context arguments). context selectedMethod ast link: metaLink Cleanup we do not care as everything will be GCed as soon as we accept the method for real. And it works! I did not test it too much, but it seems to do what it is supposed to do. (but more tests and testing are for sure needed) https://github.com/pharo-project/pharo/pull/1182 Next step: move it to a better place in the menu. Marcus

April 4, 2018

Re: [Pharo-dev] Changed #atEnd primitive - #atEnd vs #next returning nil

by Sven Van Caekenberghe

> On 4 Apr 2018, at 17:32, K K Subbu <kksubbu.ml(a)gmail.com> wrote: > > On Wednesday 04 April 2018 04:06 PM, Nicolas Cellier wrote: >>> IIRC, someone said it is implemented as 'remaining size being zero' >>> and some virtual unix files like /dev/random are zero sized. >> Currently, for files other than sdio (stdout, stderr, stdin) it is >> effectively defined as: >> atEnd := stream position >= stream size > I see a confusion between Stream and its underlying collection. Stream is an iterator and just does next, nextPut, peek, reset etc. But methods like size or atEnd depend on its collection and there is no guarantee that this collection has a known and finite size. > > Essentially, a collection's size may be known and finite, unknown but finite size or infinite. This has nothing do with file descriptor being std{in,out,err}. If std* is a regular file or special file like /dev/mem, /dev/null, it's size is known at open time. With console streams or sysfs files, size is unknown until EOT (^D) or NUL is received. Lastly, special files like /dev/zero, /dev/random or /proc/cpuinfo don't have a finite size but report it as zero (!). > > [ stream atEnd ] whileFalse: [ stream next. .. ] > > will only terminate if its collection size is finite. It won't terminate for infinite collections. > > Regards .. Subbu Good summary, I agree. Still, what are the semantics of #next - does the caller always have to check for nil ? Do we think this is ugly (as the return value is outside the domain) ? Do we then still need #atEnd ? Sven

April 4, 2018