June 2016 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] STON toJsonString produces broken JSON

by Peter Uhnak

On Mon, Jun 06, 2016 at 10:36:02AM +0200, Sven Van Caekenberghe wrote: > I just committed: > > === > Name: STON-Core-SvenVanCaekenberghe.76 > Author: SvenVanCaekenberghe > Time: 6 June 2016, 10:34:01.602265 am > UUID: 71eeced4-4b1a-4648-90fa-c89efa709ba6 > Ancestors: STON-Core-SvenVanCaekenberghe.75 > > When in JSON mode, STONWriter should also signal an STONWriterError when an Association is seen (thanks Peter UhnÃ¡k) > > Update #testRestrictedClassesInJSONMode to cover this case > === > Name: STON-Tests-SvenVanCaekenberghe.66 > Author: SvenVanCaekenberghe > Time: 6 June 2016, 10:34:19.068183 am > UUID: 9df675c2-98a7-4644-b4d3-2231ba7c7444 > Ancestors: STON-Tests-SvenVanCaekenberghe.65 > > When in JSON mode, STONWriter should also signal an STONWriterError when an Association is seen (thanks Peter UhnÃ¡k) > > Update #testRestrictedClassesInJSONMode to cover this case > === > > Thanks again, Peter, for reporting this. I'm always happy to report bugs if I don't need to fix them. :) Peter

June 6, 2016

Re: [Pharo-dev] Do we want AST-Debugger?

by Denis Kudriashov

2016-06-06 10:45 GMT+02:00 Denis Kudriashov <dionisiydk(a)gmail.com>: > One area where you'll find the AST debugger worse is in performance. The > bytecode debugger uses perform: to execute at full speed when possible (it > does interpret when doing step into or through, but not when doing over). > In fact this part of the system, because it has to catch no local returns > and exceptions, is the most complex, but it's not because of VM details, > it's because of the use of perform: to run code being debugged to run at > full speed. Same approach could be applied to AST interpreter too.

June 6, 2016

Re: [Pharo-dev] Do we want AST-Debugger?

by Denis Kudriashov

2016-06-06 10:45 GMT+02:00 Denis Kudriashov <dionisiydk(a)gmail.com>: > >> But seriously, the system is based on bytecode, and that bytecode is >> pretty straightforward and easy to learn. Why not put in the effort of >> learning it? >> >> Criticizing something in ignorance is not wise IMO. >> > > I just put this method here: > So to learn how #step method working people should understand this method. And it is not alone.

June 6, 2016

Re: [Pharo-dev] Do we want AST-Debugger?

by Denis Kudriashov

Hi Eliot. 2016-06-06 1:13 GMT+02:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > That's simply not true. In fact the amount known of the VM in both the > bytecode debugger and the AST debugger is the same; both use VM primitives > that aren't specified at the image level. > > Both the AST interpreter and the bytecode interpreter include the > interpreter in Smalltalk in the image. Look at Context and > InstructionStream to find the Smalktalk definitions of both the interpreter > routines and the implementations of the bytecodes. > > > So my question is why our debuggers are not based on AST interpreter? Why > it is bad idea? (if it is bad). > > > One area where you'll find the AST debugger worse is in performance. The > bytecode debugger uses perform: to execute at full speed when possible (it > does interpret when doing step into or through, but not when doing over). > In fact this part of the system, because it has to catch no local returns > and exceptions, is the most complex, but it's not because of VM details, > it's because of the use of perform: to run code being debugged to run at > full speed. > > Not to mention, if it isn't broke don't fix it. > > But seriously, the system is based on bytecode, and that bytecode is > pretty straightforward and easy to learn. Why not put in the effort of > learning it? > > Criticizing something in ignorance is not wise IMO. > I just put this method here: InstructionStream >>interpretNextV3PlusClosureInstructionFor: client > "Send to the argument, client, a message that specifies the type of the > next instruction." > | byte type offset method | > method := self method. > byte := method at: pc. > type := byte // 16. > offset := byte \\ 16. > pc := pc+1. > "We do an inline binary search on each of the possible 16 values of type: > The old, cleaner but slowe code is retained as a comment below" > type < 8 > ifTrue: [type < 4 > ifTrue: [type < 2 > ifTrue: [type < 1 > ifTrue: ["type = 0" > ^ client pushReceiverVariable: offset] > ifFalse: ["type = 1" > ^ client pushTemporaryVariable: offset]] > ifFalse: [type < 3 > ifTrue: ["type = 2" > ^ client > pushConstant: (method literalAt: offset + 1)] > ifFalse: ["type = 3" > ^ client > pushConstant: (method literalAt: offset + 17)]]] > ifFalse: [type < 6 > ifTrue: [type < 5 > ifTrue: ["type = 4" > ^ client > pushLiteralVariable: (method literalAt: offset + 1)] > ifFalse: ["type = 5" > ^ client > pushLiteralVariable: (method literalAt: offset + 17)]] > ifFalse: [type < 7 > ifTrue: ["type = 6" > offset < 8 > ifTrue: [^ client popIntoReceiverVariable: offset] > ifFalse: [^ client popIntoTemporaryVariable: offset - 8]] > ifFalse: ["type = 7" > offset = 0 > ifTrue: [^ client pushReceiver]. > offset < 8 > ifTrue: [^ client > pushConstant: (SpecialConstants at: offset)]. > offset = 8 > ifTrue: [^ client methodReturnReceiver]. > offset < 12 > ifTrue: [^ client > methodReturnConstant: (SpecialConstants at: offset - 8)]. > offset = 12 > ifTrue: [^ client methodReturnTop]. > offset = 13 > ifTrue: [^ client blockReturnTop]. > offset > 13 > ifTrue: [^ self unusedBytecode: client at: pc - 1 ]]]]] > ifFalse: [type < 12 > ifTrue: [type < 10 > ifTrue: [type < 9 > ifTrue: ["type = 8" > ^ self > interpretV3PlusClosureExtension: offset > in: method > for: client] > ifFalse: ["type = 9 (short jumps)" > offset < 8 > ifTrue: [^ client jump: offset + 1]. > ^ client jump: offset - 8 + 1 if: false]] > ifFalse: [type < 11 > ifTrue: ["type = 10 (long jumps)" > byte := method at: pc. > pc := pc + 1. > offset < 8 > ifTrue: [^ client jump: offset - 4 * 256 + byte]. > ^ client jump: (offset bitAnd: 3) > * 256 + byte if: offset < 12] > ifFalse: ["type = 11" > ^ client > send: (Smalltalk specialSelectorAt: offset + 1) > super: false > numArgs: (Smalltalk specialNargsAt: offset + 1)]]] > ifFalse: [type = 12 > ifTrue: [^ client > send: (Smalltalk specialSelectorAt: offset + 17) > super: false > numArgs: (Smalltalk specialNargsAt: offset + 17)] > ifFalse: ["type = 13, 14 or 15" > ^ client > send: (method literalAt: offset + 1) > super: false > numArgs: type - 13]]]. > > " old code > type=0 ifTrue: [^client pushReceiverVariable: offset]. > type=1 ifTrue: [^client pushTemporaryVariable: offset]. > type=2 ifTrue: [^client pushConstant: (method literalAt: offset+1)]. > type=3 ifTrue: [^client pushConstant: (method literalAt: offset+17)]. > type=4 ifTrue: [^client pushLiteralVariable: (method literalAt: offset+1)]. > type=5 ifTrue: [^client pushLiteralVariable: (method literalAt: > offset+17)]. > type=6 > ifTrue: [offset<8 > ifTrue: [^client popIntoReceiverVariable: offset] > ifFalse: [^client popIntoTemporaryVariable: offset-8]]. > type=7 > ifTrue: [offset=0 ifTrue: [^client pushReceiver]. > offset<8 ifTrue: [^client pushConstant: (SpecialConstants at: offset)]. > offset=8 ifTrue: [^client methodReturnReceiver]. > offset<12 ifTrue: [^client methodReturnConstant: > (SpecialConstants at: offset-8)]. > offset=12 ifTrue: [^client methodReturnTop]. > offset=13 ifTrue: [^client blockReturnTop]. > offset>13 ifTrue: [^self error: 'unusedBytecode']]. > type=8 ifTrue: [^self interpretExtension: offset in: method for: client]. > type=9 > ifTrue: short jumps > [offset<8 ifTrue: [^client jump: offset+1]. > ^client jump: offset-8+1 if: false]. > type=10 > ifTrue: long jumps > [byte:= method at: pc. pc:= pc+1. > offset<8 ifTrue: [^client jump: offset-4*256 + byte]. > ^client jump: (offset bitAnd: 3)*256 + byte if: offset<12]. > type=11 > ifTrue: > [^client > send: (Smalltalk specialSelectorAt: offset+1) > super: false > numArgs: (Smalltalk specialNargsAt: offset+1)]. > type=12 > ifTrue: > [^client > send: (Smalltalk specialSelectorAt: offset+17) > super: false > numArgs: (Smalltalk specialNargsAt: offset+17)]. > type>12 > ifTrue: > [^client send: (method literalAt: offset+1) > super: false > numArgs: type-13]"

June 6, 2016

Re: [Pharo-dev] STON toJsonString produces broken JSON

by Sven Van Caekenberghe

I just committed: === Name: STON-Core-SvenVanCaekenberghe.76 Author: SvenVanCaekenberghe Time: 6 June 2016, 10:34:01.602265 am UUID: 71eeced4-4b1a-4648-90fa-c89efa709ba6 Ancestors: STON-Core-SvenVanCaekenberghe.75 When in JSON mode, STONWriter should also signal an STONWriterError when an Association is seen (thanks Peter UhnÃ¡k) Update #testRestrictedClassesInJSONMode to cover this case === Name: STON-Tests-SvenVanCaekenberghe.66 Author: SvenVanCaekenberghe Time: 6 June 2016, 10:34:19.068183 am UUID: 9df675c2-98a7-4644-b4d3-2231ba7c7444 Ancestors: STON-Tests-SvenVanCaekenberghe.65 When in JSON mode, STONWriter should also signal an STONWriterError when an Association is seen (thanks Peter UhnÃ¡k) Update #testRestrictedClassesInJSONMode to cover this case === Thanks again, Peter, for reporting this. Sven > On 05 Jun 2016, at 13:15, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Peter, > >> On 05 Jun 2016, at 13:10, Peter Uhnak <i.uhnak(a)gmail.com> wrote: >> >> Hi, >> >> shouldn't this throw an error instead of producing a broken JSON? >> >> STON toJsonString: ({'a' -> 'b'}) >> ["a":"b"] >> >> (also STON toJsonString: ('a' -> 'b') => "a":"b") >> >> Clearly I forgot to cast it to dictionary, but shouldn't it throw up? >> >> Thanks, >> Peter > > Yes, when working in JSON compatibility mode, it would better throw an error. > > The reason for this behaviour is because STON knows the concept of an association as a separate, primitive object. That was a cool contribution done later on. In normal use, you don't see this. > > I'll have a look. > > Sven > >

June 6, 2016

Re: [Pharo-dev] Pillar configuration bug ?

by Yann Dubois

I add two groups : - parser : To load the Pillar parser - parser tests : To testing the parser Yann 2016-06-04 11:45 GMT+02:00 Cyril Ferlicot D. <cyril.ferlicot(a)gmail.com>: > Le 03/06/2016 14:51, Esteban Lorenzano a Ã©crit : > > Without the parser you canât do anything so I would expect the parser to > be a core part. > > But anyway, even agreeing with your PoV, if I load #(html exporter), > then I have unreferenced pillar classes in the resulting image > (PRDocumentItem class>>#parser has a strong referenceâ¦ and even that method > should not be there according with your statement). > > > > but anyway, for me the important part is: what do I need to load to have > the parser? there is no group âparserâ in ConfigurationOfPillar. > > > > cheers! > > Esteban > > > > > > > > Hi Esteban, > > I agree with Damien that we might want the Model and exporter without > the parser. > > If the parser is missing as a group, this is a problem. The strong > reference was removed already in the development version. > > I wall open an issue to let Yann and Thibault see the problem and > correct it :) > > -- > Cyril Ferlicot > > http://www.synectique.eu > > 165 Avenue Bretagne > Lille 59000 France > >

June 6, 2016

Re: [Pharo-dev] From a mooc user: method source with it' take so long in Pharo 5

by Sven Van Caekenberghe

> On 05 Jun 2016, at 13:52, Henrik Nergaard <henrik.nergaard(a)uia.no> wrote: > > I like this approach, but it has a fatal flaw. If one is debugging file access, then as the debugger causes the source of methods to be fetched as the debugger displays them, so the file pointer in the read-only file the access of which one is trying to debug will change under one's feet, and the results will be completely wrong. > Ah yes, and trying to do something like: > SourceFiles readStreamAtFileIndex: 2 atPosition: 42 ifPresent: [ :stream | Halt now ] ifAbsent: [ ] > Crashes the image. > > Here is another approach using a SharedQueue which caches any created read only copies up to a limit (5?). So for normal usage the queue will only hold 1 entry, but if there is concurrent work reading source files it might hold more so one do not have to always create a read only copy, if the first one is in use. This is really a cool and useful hack ! > readStreamAtFileIndex: index atPosition: position ifPresent: presentBlock ifAbsent: absentBlock > > | stream rofa result | > > rofa := queue nextOrNil. > > rofa ifNil: [ > [ rofa := files collect: [ :f | f readOnlyCopy ] ] > on: FileDoesNotExistException > do: [ ^ absentBlock value ] > ]. > > stream := rofa at: index. > > position > stream size ifTrue: [ ^ absentBlock value ]. > stream position: position. > result := presentBlock value: stream. > > queue size < 5 > ifTrue: [ queue nextPut: rofa ] > ifFalse: [ rofa do: [ :each | each close ] ]. > > ^ result > > > Best regards, > Henrik > > > From: Pharo-dev [mailto:pharo-dev-bounces@lists.pharo.org] On Behalf Of Eliot Miranda > Sent: Sunday, May 22, 2016 5:27 PM > To: Pharo Development List <pharo-dev(a)lists.pharo.org> > Cc: squeak-dev(a)lists.squeakfoundation.org > Subject: Re: [Pharo-dev] From a mooc user: method source with it' take so long in Pharo 5 > > Hi Henrik, > > cc'ing squeak-dev cuz this is relevant to both communities (if and until Pharo takes the unwise decision to double the size of the image by keeping sources in the image). > > On May 22, 2016, at 5:01 AM, Henrik Nergaard <henrik.nergaard(a)uia.no> wrote: > > There is a method in SourceFileArray #localProcessReadStreamAtFileIndex:atPosition:ifPresent:ifAbsent: which uses a ProccessLocalVariable called ProccessAndSessionLocalSourcesFileArray (see #localProcessReadOnlyCopy). Changing the last line in #readStreamAt:ifPresent:ifAbsent: to use this local process one makes the time to run this snippet: > > [ 1 systemNavigation browseMethodsWithSourceString: 'Morph' matchCase: false ] timeProfile > > From ~ 10 seconds to ~ 3 seconds (Windows). > > However I cannot see that the file handles created by this processLocalVariable to ever be closed, so I suspect those are leaked? In that case there might be the need to implement some âclean upâ mechanism for ProccessLocalVariables before they are changed/nilled when a process changes. > > Another approach could be to not use a ProccessLocalVariabe at all, but extend the SourceFilesArray class to also hold one read only handle for each of its files, and use these in readStreamAtFileIndex:::â¦ . I guess that it is also necessary then to have semaphore protecting the two last lines such that setting the position in the stream and reading from it cannot be changed by other threads. > > I like this approach, but it has a fatal flaw. If one is debugging file access, then as the debugger causes the source of methods to be fetched as the debugger displays them, so the file pointer in the read-only file the access of which one is trying to debug will change under one's feet, and the results will be completely wrong. > > An approach which uses a special copy for the debugger doesn't work; one can't debug access to that file for the same reason. > > An approach that /should/ work is for the debugger to install its own copy around file-access. This should be recursive. So around every file access in the debugger there would need to be something like > > SourceFiles substituteFreshReadOnlyCopiesDuring: [...file access...] > > which would remember the current read-only files, evaluate the block using ensure: and have the ensure: block reinstall the previous file. > > Does this make sense? > > > > readStreamAtFileIndex: index atPosition: position ifPresent: presentBlock ifAbsent: absentBlock > > | stream result| > stream := self readOnlyFileAt: index. > stream ifNil: [ ^ absentBlock value ]. "sources file not available" > > position > stream size ifTrue: [ ^ absentBlock value ]. > > readSema critical: [ > stream position: position. > result := presentBlock value: stream > ]. > > ^ result > > > Best regards, > Henrik > > > From: Pharo-dev [mailto:pharo-dev-bounces@lists.pharo.org] On Behalf Of Nicolai Hess > Sent: Thursday, May 19, 2016 9:31 AM > To: Pharo Development List <pharo-dev(a)lists.pharo.org> > Subject: Re: [Pharo-dev] From a mooc user: method source with it' take so long in Pharo 5 > > Squeak caches the opened readonly file(handle). It does not have to reopen the file on every call for reading (readonly). > > 2016-05-18 19:12 GMT+02:00 stepharo <stepharo(a)free.fr>: > I am wondering why does the search 'method source with it' take so long in Pharo 5? On my PC, When I select the text 'menu' and search for all 'method source with it', in Squeak 5 it takes 3 seconds. In Pharo 5 it takes 21 seconds. >

June 6, 2016

Re: [Pharo-dev] moldable debugger challenge - iterator position

by Ben Coman

Cool. Thanks Andrei. Let me know how I can help. cheers -ben On Mon, Jun 6, 2016 at 6:14 AM, Andrei Chis <chisvasileandrei(a)gmail.com> wrote: > Hi Ben, > > This seems like a nice idea. I think a first version of a debugger that does > at least partially what you want is doable. Do you want to give it a try? > I can create a basic skeleton of the debugger and then we can iterate. > > Cheers, > Andrei > > On Sun, Jun 5, 2016 at 2:15 PM, Ben Coman <btc(a)openinworld.com> wrote: >> >> A bit of a pie in the sky idea, but.... >> I am currently tracing through code like this... >> >> XMLDOMVisitor >> visitDocument: theDocument >> theDocument nodes do: [ :each | each accept: self ] >> >> and its hard to get a feel for where "each" is in theDocument. I >> wonder if it would even be possible for the debugger to have a tab >> that showed a list of "theDocument nodes" with a cursor positioned at >> "each" ? Maybe the #do: iterator is automatically identified, or >> maybe you'd need a pragma to define the tab, specifying the list >> contents and which variable is the cursor. When the program crashes >> on an item, you could selected an item a few earlier and choose to >> break on that data value. >> >> So I guess all of it would be tough, but anyway thought I'd share that >> flash of an idea. >> cheers -ben >> >

June 6, 2016

Re: [Pharo-dev] Do we want AST-Debugger?

by Eliot Miranda

Hi Denis, > On May 25, 2016, at 3:12 AM, Denis Kudriashov <dionisiydk(a)gmail.com> wrote: > > Hi. > > Look at AST-Interpreter project which includes AST-Debugger http://smalltalkhub.com/#!/~Pharo/AST-Interpreter. Code is so simple comparing to "bytecode simulation" approach of current debugger. > AST debugger are independent from bytecode set. It just visit AST-nodes to simulate code execution. > > But current debugger simulates bytecode. It depends on current bytecode set. There are few kinds of BytecodeEncoders. To understand how current debugger is working you need to know many VM details. That's simply not true. In fact the amount known of the VM in both the bytecode debugger and the AST debugger is the same; both use VM primitives that aren't specified at the image level. Both the AST interpreter and the bytecode interpreter include the interpreter in Smalltalk in the image. Look at Context and InstructionStream to find the Smalktalk definitions of both the interpreter routines and the implementations of the bytecodes. > > So my question is why our debuggers are not based on AST interpreter? Why it is bad idea? (if it is bad). One area where you'll find the AST debugger worse is in performance. The bytecode debugger uses perform: to execute at full speed when possible (it does interpret when doing step into or through, but not when doing over). In fact this part of the system, because it has to catch no local returns and exceptions, is the most complex, but it's not because of VM details, it's because of the use of perform: to run code being debugged to run at full speed. Not to mention, if it isn't broke don't fix it. But seriously, the system is based on bytecode, and that bytecode is pretty straightforward and easy to learn. Why not put in the effort of learning it? Criticizing something in ignorance is not wise IMO. > Best regards, > Denis _,,,^..^,,,_ (phone)

June 5, 2016

Re: [Pharo-dev] moldable debugger challenge - iterator position

by Andrei Chis

Hi Ben, This seems like a nice idea. I think a first version of a debugger that does at least partially what you want is doable. Do you want to give it a try? I can create a basic skeleton of the debugger and then we can iterate. Cheers, Andrei On Sun, Jun 5, 2016 at 2:15 PM, Ben Coman <btc(a)openinworld.com> wrote: > A bit of a pie in the sky idea, but.... > I am currently tracing through code like this... > > XMLDOMVisitor >> visitDocument: theDocument > theDocument nodes do: [ :each | each accept: self ] > > and its hard to get a feel for where "each" is in theDocument. I > wonder if it would even be possible for the debugger to have a tab > that showed a list of "theDocument nodes" with a cursor positioned at > "each" ? Maybe the #do: iterator is automatically identified, or > maybe you'd need a pragma to define the tab, specifying the list > contents and which variable is the cursor. When the program crashes > on an item, you could selected an item a few earlier and choose to > break on that data value. > > So I guess all of it would be tough, but anyway thought I'd share that > flash of an idea. > cheers -ben > >

June 5, 2016