- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [squeak-dev] Re: A fast Transcript

by Levente Uzonyi

On Sun, 10 May 2015, karl ramberg wrote: > If you disable preference 'Force Transcrip updates to screen' you will get much higher performance.Redrawing is then minimal > Time millisecondsToRun: [ 1000 timesRepeat: Â [ Transcript show: 'x'] ] 694 Right, but then it won't update the contents of the morph, while your code is running in the UI process. So you have to fork your process, just like in Pharo. Levente > > Karl > > On Sun, May 10, 2015 at 4:01 AM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Tudor, > > On Sat, May 9, 2015 at 2:23 PM, Tudor Girba <tudor(a)tudorgirba.com> wrote: > Hi Eliot, > Just a question: Could you describe the situations when you need to have the Transcript editable? > > I am not trying to provoke you, > > > my reputation preceeds me ;-) > > Â > but to understand use cases. While working on GT I found that it is often possible to support the same use cases with multiple UI solutions. And sometimes considering alternatives leads to > interesting results. > > > While I have an over-full transcript that holds progress over time, notes etc, these could be moved to a workspace.Â What I really use editablity in the transcript for is to cut-down output.Â For > example, if I'm analysing some jitted code it will get printed out in the transcript (without a lot of retooling) and it is very convenient to cut down the often large amount of output to focus on what > I'm interested in.Â Or I might be trying to fix a bug with repeated runs and again collect relevant data (print outs of objects or stack frames from the simulator) for use as reference points in later > runs.Â Or I might collect some warnings from the Slang SMalltalk-to-C translator. > > In short, there's a lot of different kinds of data going to the transcript and it can be useful to edit it in place instead of having to copy it elsewhere. > > > Cheers, > Doru > > > > On Sat, May 9, 2015 at 9:32 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Juan, > Â Â I see that your new transcript is indeed much faster but is also not editable.Â I wonder if it would be possible to have the best of both worlds and arrange that, when active, the > transcript is editable.Â For example, selecting the TranscriptWIndow could put it into a mode where the transcript's state was imported into a more conventional editable text morph, and then > when the window was exited, reverted to the standard fast output mode.Â As you can infer I like having an editable transcript.Â I can do without it, because Squeak's slow transcript is > indeed a PITA, but I'm not sure that speed need preclude editability. > > On Sat, May 9, 2015 at 10:17 AM, J. Vuletich (mail lists) <juanlists(a)jvuletich.org> wrote: > Hi Folks, > > (below) > > Quoting Ben Coman <btc(a)openinworld.com>: > > On Sat, May 9, 2015 at 10:35 PM, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > > > On Sat, May 9, 2015 at 7:09 AM, Ben Coman <btc(a)openinworld.com> wrote: > > >From my limited experience bug hunting, calling #changed: from a thread > other than the UI thread is a source of evil.Â There are too many > assumptions throughout the system that the UI is single threaded.Â Can > anyone advise me that is not a proper belief? > > Then that implies that a Transcript implementation where #nextPut: direct > calls #changed: > is not appropriate for use with multi-threaded applications.Â In Pharo, > #changed: is only called from #stepGlobal, which is called from > doOneCycle:.Â (This came about as a last minute bug fix before Pharo 3 > release and maybe could use some cleanup. > > Separating the UI from Transcript into its own viewer might be a good > idea, but actually it would not solve Stef's case since his code would > still be running in the UI thread -- unless the viewer ran in another > thread, which would have its own complexities. > > I think the point about efficiency is significant. The following > example... > Â Â Â Time millisecondsToRun: [ 1000 timesRepeat:Â [ Transcript show: 'x' > ] ] > on Squeak 4.5 --> 12749ms > on Pharo 50029 --> 2ms > > This better performance helped me a lot trying to understand the high > priority timerEventLoop being able to indiscriminately scatter Transcript > tracing through that code.Â I believe its also probably beneficial for > working with externally triggered semaphores and timing sensitive race > conditions. > > So we have two mutually exclusive cases: > * better interactivity, poorer system performance > * faster system performance, worse interactivity > > Which of these is broken depends on your viewpoint. > > > Something that runs fast but is incorrect is still incorrect.Â The fact > that the transcript doesn't output until a world step is possible is a > bug.Â It forces programs that use the transcript to be rewritten in order > to see transcript output. > > > As a point of comparison for correctness, for the following... > > Â Â Transcript clear. > Â Â [Â Â $a asciiValue to: $z asciiValue do: [ :c | > Â [ 1 to: 9 do: [ :i | Transcript show: c asCharacter printString , i > printString , ' ' ] ] forkAt: 40 > ]. > Â Â ] forkAt: 41 > > Squeak 4.5 gives... > $a1 $a2 $a3 $a4 $a5 $a6 $a7 $a8 $a9 $b1 $b2 $b3 $b4 $b5 $b5 $c1 $c2 $c3 $c4 > $c5 $c6 $c7 $c8 $c9 $d1 $d2 $d3 $d4 $d5 $d6 $d7 $d8 $d9 $d9 $e2 $g2 $h2 $h2 > $i2 $k2 $k2 $l2 $n2 $n2 $o2 $o2 $r2 $s2 $t2 $u2 $u2 $v2 $x2 $y2 $z2 $z2 $b7 > $f3 $e3 $e3 $g3 $j3 $h3 $i3 $k3 $k3 $m3 $n3 $p3 $p3 $q3 $o3 $s3 $t3 $t3 $u3 > $v3 $x3 $y3 $z3 $b8 $f4 $e4 $e4 $g4 $h4 $i4 $k4 $l4 $m4 $m4 $n4 $r4 $q4 $o4 > $o4 $s4 $w4 $u4 $u4 $v4 $y4 $y4 $z4 $z4 $f5 $j5 $j5 $g5 $i5 $k5 $l5 $l5 $m5 > $m5 $n5 $q5 $o5 $s5 $s5 $t5 $u5 $u5 $x5 $y5 $z5 $f6 $f6 $h6 $h6 $g6 $g6 $k6 > $p6 $m6 $r6 $r6 $n6 $o6 $s6 $s6 $w6 $u6 $x6 $x6 $e7 $f7 $j7 $h7 $h7 $i7 $l7 > $l7 $k7 $m7 $m7 $q7 $n7 $n7 $o7 $t7 $w7 $w7 $u7 $v7 $x7 $z7 $z7 $e8 $e8 $h8 > $g8 $i8 $i8 $l8 $k8 $k8 $m8 $q8 $n8 $n8 $s8 $t8 $w8 $y8 $y8 $u8 $x8 $z8 $f9 > $f9 $e9 $h9 $h9 $g9 $p9 $p9 $k9 $r9 $r9 $m9 $n9 $n9 $o9 $t9 $t9 $w9 $v9 $u9 > $u9 $z9 $x9 > > Pharo 50041 gives... > $a1 $a2 $a3 $a4 $a5 $a6 $a7 $a8 $a9 $b1 $b2 $b3 $b4 $b5 $b6 $b7 $b8 $b9 $c1 > $c2 $c3 $c4 $c5 $c6 $c7 $c8 $c9 $d1 $d2 $d3 $d4 $d5 $d6 $d7 $d8 $d9 $e1 $e2 > $e3 $e4 $e5 $e6 $e7 $e8 $e9 $f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9 $g1 $g2 $g3 > $g4 $g5 $g6 $g7 $g8 $g9 $h1 $h2 $h3 $h4 $h5 $h6 $h7 $h8 $h9 $i1 $i2 $i3 $i4 > $i5 $i6 $i7 $i8 $i9 $j1 $j2 $j3 $j4 $j5 $j6 $j7 $j8 $j9 $k1 $k2 $k3 $k4 $k5 > $k6 $k7 $k8 $k9 $l1 $l2 $l3 $l4 $l5 $l6 $l7 $l8 $l9 $m1 $m2 $m3 $m4 $m5 $m6 > $m7 $m8 $m9 $n1 $n2 $n3 $n4 $n5 $n6 $n7 $n8 $n9 $o1 $o2 $o3 $o4 $o5 $o6 $o7 > $o8 $o9 $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8 $p9 $q1 $q2 $q3 $q4 $q5 $q6 $q7 $q8 > $q9 $r1 $r2 $r3 $r4 $r5 $r6 $r7 $r8 $r9 $s1 $s2 $s3 $s4 $s5 $s6 $s7 $s8 $s9 > $t1 $t2 $t3 $t4 $t5 $t6 $t7 $t8 $t9 $u1 $u2 $u3 $u4 $u5 $u6 $u7 $u8 $u9 $v1 > $v2 $v3 $v4 $v5 $v6 $v7 $v8 $v9 $w1 $w2 $w3 $w4 $w5 $w6 $w7 $w8 $w9 $x1 $x2 > $x3 $x4 $x5 $x6 $x7 $x8 $x9 $y1 $y2 $y3 $y4 $y5 $y6 $y7 $y8 $y9 $z1 $z2 $z3 > $z4 $z5 $z6 $z7 $z8 $z9 > > (start your comparison at $b5) > > So in one axis Pharo has improved Transcript, but we didn't notice the > significance of the use case we lost. > > cheers -ben > > > Please take a good look at Cuis' Transcript and consider using it. > > By default, the display is updated immediately, but without calling Morphic, it can even work with no UI framework at all. It does updates faster than Squeak or Visualworks: > > Â Â Â Â Time millisecondsToRun: [ 1000 timesRepeat:Â [ Transcript show: 'x' ] ]. 763. > > But if you want minimum overhead without immediate feedback: > > Â Â Â Â Time millisecondsToRun: [ Transcript showOnDisplay: false. 1000 timesRepeat:Â [ Transcript show: 'x' ]. Transcript showOnDisplay: true ]. 1. > Â Â Â Â "As fast as Pharo" > > It is also thread safe, and for Ben's example: > > Transcript clear. > [ > Â Â Â Â $a asciiValue to: $z asciiValue do: [ :c | > Â Â Â Â Â Â Â Â [ 1 to: 9 do: [ :i | Transcript show: c asCharacter printString , i printString , ' ' ] ] forkAt: 40 > Â Â Â Â ]. > ] forkAt: 41 > > it gives the same result as Pharo. > > The fact that the updates are not bound to Morphic also means that it is possible to do #show: deep in Morphic logic, without causing infinite loops or recursions, and get immediate > feedback. It has proved to be a useful aid in debugging Morphic code. > > Cheers, > Juan Vuletich > > > > > > -- > best,Eliot > > > > > -- > www.tudorgirba.com > > "Every thing has its own flow" > > > > > -- > best,Eliot > > > > > >

May 10, 2015

Re: [Pharo-dev] Error on loading GToolkit configuration

by Andrei Chis

Made a small mistake in the configuration. Should work now. Cheers, Andrei On Sun, May 10, 2015 at 11:50 AM, Nicolai Hess <nicolaihess(a)web.de> wrote: > > 15470 <https://pharo.fogbugz.com/default.asp?15470> > Replace Announcer>>#on:send:to:'s senders in GTSpotter > > > Pharo5.0 > Latest update: #50040 > > > Anyone knows why this fails: > > Gofer new > smalltalkhubUser: 'Moose' project: 'GToolkit'; > load; > configurationOf: 'GTSpotter'; > loadVersion:'1.2.4' > -> Could not resolve: GT-SpotterExtensions-Core [ > GT-SpotterExtensions-Core-AndreiChis.145] > >

May 10, 2015

Error on loading GToolkit configuration

by Nicolai Hess

15470 <https://pharo.fogbugz.com/default.asp?15470> Replace Announcer>>#on:send:to:'s senders in GTSpotter Pharo5.0 Latest update: #50040 Anyone knows why this fails: Gofer new smalltalkhubUser: 'Moose' project: 'GToolkit'; load; configurationOf: 'GTSpotter'; loadVersion:'1.2.4' -> Could not resolve: GT-SpotterExtensions-Core [ GT-SpotterExtensions-Core-AndreiChis.145]

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by H. Hirzel

On 5/10/15, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >> On 10 May 2015, at 10:36, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: >> >> +1 >> >> letâs recapitulate: >> >> requirement is >> - fast >> - editable >> - polymorphic with WriteStream >> - and immediate to screen >> from our perspective (pharo), it has to be also >> - thread safe >> - well designed. >> >> our constraints are: >> - bad design of transcripts >> - super naive implementation of text editors (this is, IMO, the reason why >> transcripts are so slow: if you redraw all the window interior each time >> you add a line, you are screw) > > Good summary! +1 >> basicallyâ¦ we need a good log console (if it is a transcript or not, is >> another discussion) :) +1 As an independent pluggable package because not everybody does VM development all the time. There might be several of them focusing on different needs. And there are already libraries claiming to be loggers. Might be used as a start. > Yes, that is really all there is to it > >> Soâ¦ who proposes a solution? >> >> Esteban >> >> >>> On 10 May 2015, at 00:40, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> wrote: >>> >>> For me saying that internals are not important as long as transcript does >>> what is expected from it, is like saying that Parser being a subclass of >>> Scanner and Compiler being as subclass of Parser is ok as long as it >>> compiles source code. We already had that and it was bad. >>> >>> On the other hand itâs obvious that we donât have enough resources to >>> rewrite Pharo from scratch and we cannot leave issues as they are just >>> because we cannot solve them perfectly now. >>> >>> I would say that Igor is correct saying that we need a better >>> architecture, and Stef and Eliot are correct saying that we need tools >>> that fulfill usersâ needs. And we have to say: âok now we hack this >>> around, but we have to do this the right way one dayâ. Is anyone familiar >>> with same-css[1] concept? >>> >>> Uko >>> >>> P.S. maybe my 2 cents are useless, but all the fight around this >>> discussion is really strange for me. In Ukraine we have a saying that >>> itâs better to be healthy and poor than sick and rich. Now I use to say >>> that itâs better to be healthy and rich than sick and poor. And all the >>> fight here was if itâs better to be healthy or rich. Definitely itâs >>> better to be both, we cannot do that instantly but we can set goals and >>> milestones, and find the optimal strategy. >>> >>> >>> [1]: http://csswizardry.com/2013/04/shame-css/ >>> >>>> On 09 May 2015, at 16:17, Eliot Miranda <eliot.miranda(a)gmail.com> >>>> wrote: >>>> >>>> >>>> >>>> On Sat, May 9, 2015 at 5:29 AM, stepharo <stepharo(a)free.fr> wrote: >>>> Eliot >>>> >>>> I changed the transcript because it is not thread safe so I could use it >>>> at all to explain concurrent programming output. >>>> It was terrible. >>>> >>>> I don't see what that has to do with my usability point. The transcript >>>> was >>>> a) a stream >>>> b) something that displayed its output immediately >>>> >>>> Those two features are essential. If you change the transcript so that >>>> it no longer has those features you have broken it. >>>> >>>> Igor waffled on about internals, the desire to separate the UI from the >>>> stream, a point that has nothing to do with the utility of the >>>> transcript. No you're going on about its thread-safety. But no one is >>>> addressing my point. And you have ClÃ©ment telling you that he cannot >>>> develop the VM in Pharo because the transcript is broken, and you have >>>> others proposing various potentially error-prone work-arounds to get >>>> around the fact that the transcript is broken. >>>> >>>> Why won't anyone stand up and say yes, the transcript is broken and yes >>>> we will fix it? This is dysfunctional. >>>> >>>> >>>> Stef >>>> >>>> >>>> Le 8/5/15 16:16, Eliot Miranda a Ã©crit : >>>> >>>> Hi, >>>> >>>> if one uses a at doit transcript then no special action is required >>>> to get output to appear beyond sending flush to Transcript right? So >>>> any solution that requires special action to get the moronic transcript >>>> to work us broken. We should fix the transcript, not expect every >>>> application to work around a bug. >>>> >>>> Eliot (phone) >>>> >>>> On May 8, 2015, at 6:15 AM, Alain Rastoul <alf.mmm.cat(a)gmail.com> >>>> wrote: >>>> >>>> Le 08/05/2015 11:34, stepharo a Ã©crit : >>>> Hi guys >>>> >>>> the Transcript in Pharo is that it's not asynchronous so I can't use it >>>> in VM development to show the current progress of the simulation. For >>>> example: >>>> 1 to: 100 do: [ :i | >>>> 0.1 seconds asDelay wait. >>>> Transcript show: 'x'. ] >>>> => on Squeak, this shows a x every 0.1 second in the Transcript >>>> => on Pharo, nothing happens during 10 seconds then all the x are >>>> shown. >>>> >>>> https://pharo.fogbugz.com/default.asp?15515 >>>> Yes, as do it are evaluated in the World morphic process, running in a >>>> forked process or sending World doOneCycle in the loop solve the >>>> problem. >>>> >>>> Probably in squeak, in Transcript this is done somewhere under the >>>> hood. >>>> >>>> via dependents ? >>>> >>>> TranscriptStream>>endEndtry >>>> "Display all the characters since the last endEntry, and reset the >>>> stream" >>>> self semaphore critical:[ >>>> self changed: #appendEntry. >>>> self reset. >>>> ]. >>>> >>>> Object>>changed: aParameter >>>> self dependents do: [:aDependent | aDependent update: aParameter] >>>> >>>> And probably not doOnecycle since you cannot do anything else during >>>> execution (clicking or moving windows). >>>> >>>> >>>> -- >>>> Regards, >>>> >>>> Alain >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> best, >>>> Eliot >>> >> > > >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by Sven Van Caekenberghe

> On 10 May 2015, at 10:36, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > +1 > > letâs recapitulate: > > requirement is > - fast > - editable > - polymorphic with WriteStream > - and immediate to screen > from our perspective (pharo), it has to be also > - thread safe > - well designed. > > our constraints are: > - bad design of transcripts > - super naive implementation of text editors (this is, IMO, the reason why transcripts are so slow: if you redraw all the window interior each time you add a line, you are screw) Good summary! > basicallyâ¦ we need a good log console (if it is a transcript or not, is another discussion) :) Yes, that is really all there is to it > Soâ¦ who proposes a solution? > > Esteban > > >> On 10 May 2015, at 00:40, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> wrote: >> >> For me saying that internals are not important as long as transcript does what is expected from it, is like saying that Parser being a subclass of Scanner and Compiler being as subclass of Parser is ok as long as it compiles source code. We already had that and it was bad. >> >> On the other hand itâs obvious that we donât have enough resources to rewrite Pharo from scratch and we cannot leave issues as they are just because we cannot solve them perfectly now. >> >> I would say that Igor is correct saying that we need a better architecture, and Stef and Eliot are correct saying that we need tools that fulfill usersâ needs. And we have to say: âok now we hack this around, but we have to do this the right way one dayâ. Is anyone familiar with same-css[1] concept? >> >> Uko >> >> P.S. maybe my 2 cents are useless, but all the fight around this discussion is really strange for me. In Ukraine we have a saying that itâs better to be healthy and poor than sick and rich. Now I use to say that itâs better to be healthy and rich than sick and poor. And all the fight here was if itâs better to be healthy or rich. Definitely itâs better to be both, we cannot do that instantly but we can set goals and milestones, and find the optimal strategy. >> >> >> [1]: http://csswizardry.com/2013/04/shame-css/ >> >>> On 09 May 2015, at 16:17, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: >>> >>> >>> >>> On Sat, May 9, 2015 at 5:29 AM, stepharo <stepharo(a)free.fr> wrote: >>> Eliot >>> >>> I changed the transcript because it is not thread safe so I could use it at all to explain concurrent programming output. >>> It was terrible. >>> >>> I don't see what that has to do with my usability point. The transcript was >>> a) a stream >>> b) something that displayed its output immediately >>> >>> Those two features are essential. If you change the transcript so that it no longer has those features you have broken it. >>> >>> Igor waffled on about internals, the desire to separate the UI from the stream, a point that has nothing to do with the utility of the transcript. No you're going on about its thread-safety. But no one is addressing my point. And you have ClÃ©ment telling you that he cannot develop the VM in Pharo because the transcript is broken, and you have others proposing various potentially error-prone work-arounds to get around the fact that the transcript is broken. >>> >>> Why won't anyone stand up and say yes, the transcript is broken and yes we will fix it? This is dysfunctional. >>> >>> >>> Stef >>> >>> >>> Le 8/5/15 16:16, Eliot Miranda a Ã©crit : >>> >>> Hi, >>> >>> if one uses a at doit transcript then no special action is required to get output to appear beyond sending flush to Transcript right? So any solution that requires special action to get the moronic transcript to work us broken. We should fix the transcript, not expect every application to work around a bug. >>> >>> Eliot (phone) >>> >>> On May 8, 2015, at 6:15 AM, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: >>> >>> Le 08/05/2015 11:34, stepharo a Ã©crit : >>> Hi guys >>> >>> the Transcript in Pharo is that it's not asynchronous so I can't use it >>> in VM development to show the current progress of the simulation. For >>> example: >>> 1 to: 100 do: [ :i | >>> 0.1 seconds asDelay wait. >>> Transcript show: 'x'. ] >>> => on Squeak, this shows a x every 0.1 second in the Transcript >>> => on Pharo, nothing happens during 10 seconds then all the x are shown. >>> >>> https://pharo.fogbugz.com/default.asp?15515 >>> Yes, as do it are evaluated in the World morphic process, running in a forked process or sending World doOneCycle in the loop solve the problem. >>> >>> Probably in squeak, in Transcript this is done somewhere under the hood. >>> >>> via dependents ? >>> >>> TranscriptStream>>endEndtry >>> "Display all the characters since the last endEntry, and reset the stream" >>> self semaphore critical:[ >>> self changed: #appendEntry. >>> self reset. >>> ]. >>> >>> Object>>changed: aParameter >>> self dependents do: [:aDependent | aDependent update: aParameter] >>> >>> And probably not doOnecycle since you cannot do anything else during execution (clicking or moving windows). >>> >>> >>> -- >>> Regards, >>> >>> Alain >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> best, >>> Eliot >> >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by Sven Van Caekenberghe

Hi ClÃ©ment, Thanks for the detailed write up, this is surely helpful. I cannot react to all points, because some are out of my expertise. However, in this whole, very long thread, there is no one who ever said WHICH STREAM API IS MISSING ? It should be quite easy to add something, if only someone would please tell us ! I also would find it hard to believe you have a working solution for coloured/decorated output that also works on a terminal and that is compatible with the general stream API. Sven > On 10 May 2015, at 10:28, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > > > > 2015-05-09 23:21 GMT+02:00 Tudor Girba <tudor(a)tudorgirba.com>: > I do not think there are many people around here that would think that it is irrelevant if the Pharo VM can be developed in Pharo or not. Of course, it is important. > > So, the discussion should not go to challenge this direction, but rather in you telling us the use cases that you need supported. Please note that I did not say which exact code and how it should look like. I would be interested in learning about the use cases you have. I am quite certain that there are a number of ways to support them and when we work on GT it would be useful to have your use cases on our table. > > Well I need many lines to explain each point and there are many... I can talk here about a few points. Then I will deal with Esteban for most of them because it is difficult to explain without an interactive discussion. > > > Let me explain the use cases for the Transcript for example. The issues in Pharo are: > - The Transcript does not show the stream as it is printed. > - The Transcript does not inherit from Stream and thus cannot print with all the methods implemented in Stream. > - The Transcript does not allow the user to decorate the text with bold, italic or colors. > > Usecase 1: Debug printing methods: In the VM you have debug printing methods, for example, to print the call stack. These methods are used from the VM simulator, to output the string in the Transcript, and in gdb, to ouput the string in the commandline. The commandline (FileStream stdout in Pharo) and the Squeak Transcript have the same behavior. In Pharo, the Transcript does not inherit from Stream so you can't use the required stream methods to print the debug printing method on the Transcript. In addition, some printing methods print a lot of things and it is important to show the stream as it is printed. > For this use-case, we want to keep the smallest difference between the gdb/commadline behavior and the VM simulator/Transcript behavior. If you implement advanced tooling in GT, you therefore need to implement gdb extensions (and lldb extensions because some of us use lldb instead of gdb) and maintain them. I don't think this is a solution. > > Usecase 2: CCode generation debugging: The CCodeGenerator or Slang translator translates Slang code into C code. Sometimes there is a bug. To debug, instead of generating the faulty C method into an external C file, we print only the faulty C method in the Transcript. Again, we want to keep the lowest difference between the real usecase (printing on the C file) and the debug usecase (printing on the Transcript). In Squeak the FileStream and the Transcript are both Stream, everything works as expected. In Pharo the Transcript has not the expected behavior. Again the method can be long, you can have to wait several seconds, so you'd like the transcript to show the stream as you print it. > > Usecase 3: VM simulation: Simulating the VM is quite slow, especially the machine code execution simulation. During the simulation process, the UI is non interactive and shows only every while what the simulator is doing in the Transcript. It is important as sometimes when debugging with a test at each machine code instruction it could take several hours before the UI is interactive again and you want to know what is going on. I don't complain that it takes several hours because the alternatives usually require days of debugging and we can launch the VM simulator overnight. In Pharo this does not work as expected. > > Usecase 4: In-image machine-code compilation: While working in the JIT compiler, sometimes the machine code generated for a bytecoded method is faulty. A common way of debugging it is to print the machine code instructions of the machine code version of the method in the Transcript. It can take a while to print, so it is important to have the Transcript showing the text as it prints. Then, the easiest way of debugging is to look at the machine code and understand what is wrong. For this purpose, we add text decoration to color jump addresses or the instructions where the instruction pointer was when the VM crashed. Then, in squeak, we can easily copy the decorated text to a workspace and generate a new version of the machine code method and compare. In machine code, it is very difficult to do analysis to have more information than just the decompiled text. We add some information while simulating because we know for example the address of specific trampolines, therefore we can print the name of the trampoline when we see that its address is called. Again, sometimes we also have to debug in gdb. In this case, we disassemble the machine code and compare it to the one from in-image compilation, so both printed strings have to be similar (similar text, same chariot returns). > > > > Another example is the complexity of the Pharo tools: > > While developing the VM, I have sometimes a VM partially working or with some plugins not working. In the Squeak image, I can open a workspace on top of this half-working VM and run do-its to see what is working and what is not. In the Pharo image, I can't do anything. You can't open the workspace without opening more advanced tools. I tried to open the Playground, but the first time there was a bug with Traits (Playground use Traits somehow and they were not working due to the new bytecode set not being finished), when that first bug was fixed I could not open it because it crashed simply the VM (I believe it tried to access an external file such as playground-cache). Currently, the Pharo team is trying to build a set of basic tools that have few dependencies to debug a partially working system (that I think you will use to debug glamour while editing it, because you cannot use the glamour inspector if glamour is not working). That would solve this issue. > But in no way this point is something that I can do alone to be able to develop the VM in Pharo. This has to be a community effort. And I am saying that because I can't be blamed not to work on the VM in Pharo if to do so I need to spend many months changing Pharo. > > > > An example that I believe is a problem in term of the community is the following: > > I added with Eliot the support for the new bytecode set. Currently, the Squeak image works with the new bytecode set but not the Pharo image. This is because only the Traits are broken, but this is something I could hardly figure out in the Pharo image because nothing is working as the GT tools use Traits. In Squeak I believe there are very few users of Traits so everything worked, and the test suite can reveal that the Traits are broken easily. > > Currently, the VM process to me is to first make new features work in Squeak, because it is simpler, and then make it work with Pharo, which is more complex. In the last section I discussed how Traits were a problem while implementing the new bytecode set. So what is the long term solution for this issue ? > - Will we have a bootstrap process that creates first a Trait-free Kernel and then build the Pharo Kernel out of it ? > - Do we forbid people to use Traits in the Pharo Kernel and does that make sense to have Traits in Pharo in this case ? > - If we don't do anything, maybe the Traits are only a slight difference with low impact in most cases and it's fine. But maybe there are many small aspects like Traits, such as the Slots the way they were used in GT recently (I don't blame GT or anything, it was just using features in the system that created issues for me), and maybe we reached a point where the complexity between the Pharo kernel and the Squeak kernel is big enough so that a VM developer will first make Squeak works when introducing new features and then deals with the complexity of Pharo ? > > So, what do we do ? I don't see any simple solution for this issue. And I believe there are people around that see as the only solution for this issue not to have the Pharo VM development process in Pharo because they will see it as a threat to what they want to do with Pharo. > > > > Best Doru ! > > PS: I am still using the GTInspector with additional views on graphs created with Roassal everyday and I still enjoy it. > > PS2: I am on vacation currently because I was getting crazy looking at machine code all day long, so I may not answer as quick as usually during the next week. > > > > Cheers, > Doru > > > > On Sat, May 9, 2015 at 9:31 PM, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > > > 2015-05-09 20:25 GMT+02:00 stepharo <stepharo(a)free.fr>: > > > Le 9/5/15 20:16, ClÃ©ment Bera a Ã©crit : >> This whole conversation here shows very well the point that I tried to explain to Stef last week. I'm sorry if the mail is a bit long but I think this discussion has to be done. >> >> My whole Smalltalk development life, I have used Pharo and was happy with it. Now I am also working in Cog's JIT compiler and for this specific project, I am working with Squeak. I don't work with Squeak because I don't like Pharo, I told you before, I have worked with Pharo on all my project before, enjoyed it and if it was possible I would use Pharo. I work with Squeak because the VM development tool and development process simply does *not* work in Pharo. This is not only because of VM tools working with the old Morphic not working anymore in Pharo or details like that, it is also due to deeper changes in Pharo. >> >> Stef believes it is important that Pharo is able to host development for its own VM. Therefore, I discussed with him and Esteban about a first list of points that are necessary for Pharo to support its VM development in Pharo, which includes this Transcript behavior. >> >> As of today, and I am honest here, I believe that what is required for Pharo to support the development process of its VM includes points which goes in the opposite direction than a few points in the Pharo roadmap, that people in the Pharo community will see as a regression, as "an intrusion from the Squeak philosophy into Pharo", or as forbidding the integration of features that breaks the VM development process. Therefore, I believe the Pharo community would disapprove to make such changes and I highly doubt that it is possible to have the development process of the >> Pharo VM in Pharo. >> >> I was thinking that only a few points would be a problem such as the increasing memory footprint of the Pharo image that is going to get worse with the sources that will be included in the image in the future, whereas a VM developer needs a small image (See previous threads in this mailing list where Hilaire complains about that for example). > > clement can I ask a simple question? > why did I ask guille to work on minikernels and bootstrap for his phd instead on a topic where we can publish? > - choice A: lack of idea > - choice B: .... > > I have already stated that you believe that it is important that Pharo is able to host development for its own VM. > > I am not against what you did and I am very excited with Guille's work. > > Pharo is community-driven, so I am not asking the question to you only, but to the community. > > >> However, I didn't think that even simple points like the Transcript behavior discussed here, which looks like to me as a regression and is required for VM development, would be seen as an improvement by a non negligible part of the community. >> >> In this mailing-list, the whole Pharo community is present and can see this discussion. So the open questions are: >> >> Do you want to have the development of the Pharo VM in Pharo, or do you want the development of the Pharo VM to remain in Squeak ? >> Do you think a system that is not good enough to handle its own VM development is a good system ? >> >> I am not willing to go against the will of the community because I enjoy community-driven softwares. If the answer is that Pharo should be able to support its own VM development then as I started I will help Esteban and Stef to improve Pharo so that it can support its own VM development. Now, if the answer is that the development of the Pharo VM should remain in Squeak, I will continue developing the VM in Squeak. >> >> You are the Pharo community, you are the ones that make Pharo alive and kicking, so you tell me what you think we should do. >> >> Clement >> >> 2015-05-09 18:23 GMT+02:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >> Hi Ben, >> >> On May 9, 2015, at 7:41 AM, Ben Coman <btc(a)openinworld.com> wrote: >> >>> >>> >>> On Sat, May 9, 2015 at 10:09 PM, Ben Coman <btc(a)openinworld.com> wrote: >>> From my limited experience bug hunting, calling #changed: from a thread other than the UI thread is a source of evil. There are too many assumptions throughout the system that the UI is single threaded. Can anyone advise me that is not a proper belief? >>> >>> Then that implies that a Transcript implementation where #nextPut: direct calls #changed: >>> is not appropriate for use with multi-threaded applications. In Pharo, #changed: is only called from #stepGlobal, which is called from doOneCycle:. (This came about as a last minute bug fix before Pharo 3 release and maybe could use some cleanup. >>> >>> Separating the UI from Transcript into its own viewer might be a good idea, but actually it would not solve Stef's case since his code would still be running in the UI thread -- unless the viewer ran in another thread, which would have its own complexities. >>> >>> I think the point about efficiency is significant. The following example... >>> Time millisecondsToRun: [ 1000 timesRepeat: [ Transcript show: 'x' ] ] >>> on Squeak 4.5 --> 12749ms >>> on Pharo 50029 --> 2ms >>> >>> As a point of comparison, on VW 8.0 --> 43817ms >>> and so you might guess, VW 8.0 outputs each 'x' immediately. >>> cheers -ben >> >> Way to go, Squeak! Actually this is disappointing. I'm rather frustrated with Squeak's slow transcript, and was hoping that VW would demonstrate it could be faster. Looking at the Squeak implementation I only see an obvious 30% or so improvement via tuning. Looks like good performance will take more work :-/ >> >> >> >> Eliot (phone) >> > > > > > > -- > www.tudorgirba.com > > "Every thing has its own flow" >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by Sven Van Caekenberghe

> On 10 May 2015, at 10:22, stepharo <stepharo(a)free.fr> wrote: > > > > Le 9/5/15 20:51, Sven Van Caekenberghe a Ã©crit : >> Yes, the Cuis implementation is cool. Especially because it implements both the limited/circular buffer and timed/batched updating. > > I should finish my circular linkedList. > I will resume it. Don't forget the DoubleLinkedList we already have. > >> >> It does however mix different things in 1 class (side!): the transcript stream behavior, the display/tool part, an alternative file output, some sort of newer API, and more logging oriented thing (adding a timestamp). These are all things that we would like to separate, make modular/pluggable. >> >>> On 09 May 2015, at 19:17, J. Vuletich (mail lists) <juanlists(a)jvuletich.org> wrote: >>> >>> Hi Folks, >>> >>> (below) >>> >>> Quoting Ben Coman <btc(a)openinworld.com>: >>> >>>> On Sat, May 9, 2015 at 10:35 PM, Eliot Miranda <eliot.miranda(a)gmail.com> >>>> wrote: >>>> >>>>> >>>>> On Sat, May 9, 2015 at 7:09 AM, Ben Coman <btc(a)openinworld.com> wrote: >>>>> >>>>>> From my limited experience bug hunting, calling #changed: from a thread >>>>>> other than the UI thread is a source of evil. There are too many >>>>>> assumptions throughout the system that the UI is single threaded. Can >>>>>> anyone advise me that is not a proper belief? >>>>>> >>>>>> Then that implies that a Transcript implementation where #nextPut: direct >>>>>> calls #changed: >>>>>> is not appropriate for use with multi-threaded applications. In Pharo, >>>>>> #changed: is only called from #stepGlobal, which is called from >>>>>> doOneCycle:. (This came about as a last minute bug fix before Pharo 3 >>>>>> release and maybe could use some cleanup. >>>>>> >>>>>> Separating the UI from Transcript into its own viewer might be a good >>>>>> idea, but actually it would not solve Stef's case since his code would >>>>>> still be running in the UI thread -- unless the viewer ran in another >>>>>> thread, which would have its own complexities. >>>>>> >>>>>> I think the point about efficiency is significant. The following >>>>>> example... >>>>>> Time millisecondsToRun: [ 1000 timesRepeat: [ Transcript show: 'x' >>>>>> ] ] >>>>>> on Squeak 4.5 --> 12749ms >>>>>> on Pharo 50029 --> 2ms >>>>>> >>>>>> This better performance helped me a lot trying to understand the high >>>>>> priority timerEventLoop being able to indiscriminately scatter Transcript >>>>>> tracing through that code. I believe its also probably beneficial for >>>>>> working with externally triggered semaphores and timing sensitive race >>>>>> conditions. >>>>>> >>>>>> So we have two mutually exclusive cases: >>>>>> * better interactivity, poorer system performance >>>>>> * faster system performance, worse interactivity >>>>>> >>>>>> Which of these is broken depends on your viewpoint. >>>>>> >>>>> Something that runs fast but is incorrect is still incorrect. The fact >>>>> that the transcript doesn't output until a world step is possible is a >>>>> bug. It forces programs that use the transcript to be rewritten in order >>>>> to see transcript output. >>>>> >>>> As a point of comparison for correctness, for the following... >>>> >>>> Transcript clear. >>>> [ $a asciiValue to: $z asciiValue do: [ :c | >>>> [ 1 to: 9 do: [ :i | Transcript show: c asCharacter printString , i >>>> printString , ' ' ] ] forkAt: 40 >>>> ]. >>>> ] forkAt: 41 >>>> >>>> Squeak 4.5 gives... >>>> $a1 $a2 $a3 $a4 $a5 $a6 $a7 $a8 $a9 $b1 $b2 $b3 $b4 $b5 $b5 $c1 $c2 $c3 $c4 >>>> $c5 $c6 $c7 $c8 $c9 $d1 $d2 $d3 $d4 $d5 $d6 $d7 $d8 $d9 $d9 $e2 $g2 $h2 $h2 >>>> $i2 $k2 $k2 $l2 $n2 $n2 $o2 $o2 $r2 $s2 $t2 $u2 $u2 $v2 $x2 $y2 $z2 $z2 $b7 >>>> $f3 $e3 $e3 $g3 $j3 $h3 $i3 $k3 $k3 $m3 $n3 $p3 $p3 $q3 $o3 $s3 $t3 $t3 $u3 >>>> $v3 $x3 $y3 $z3 $b8 $f4 $e4 $e4 $g4 $h4 $i4 $k4 $l4 $m4 $m4 $n4 $r4 $q4 $o4 >>>> $o4 $s4 $w4 $u4 $u4 $v4 $y4 $y4 $z4 $z4 $f5 $j5 $j5 $g5 $i5 $k5 $l5 $l5 $m5 >>>> $m5 $n5 $q5 $o5 $s5 $s5 $t5 $u5 $u5 $x5 $y5 $z5 $f6 $f6 $h6 $h6 $g6 $g6 $k6 >>>> $p6 $m6 $r6 $r6 $n6 $o6 $s6 $s6 $w6 $u6 $x6 $x6 $e7 $f7 $j7 $h7 $h7 $i7 $l7 >>>> $l7 $k7 $m7 $m7 $q7 $n7 $n7 $o7 $t7 $w7 $w7 $u7 $v7 $x7 $z7 $z7 $e8 $e8 $h8 >>>> $g8 $i8 $i8 $l8 $k8 $k8 $m8 $q8 $n8 $n8 $s8 $t8 $w8 $y8 $y8 $u8 $x8 $z8 $f9 >>>> $f9 $e9 $h9 $h9 $g9 $p9 $p9 $k9 $r9 $r9 $m9 $n9 $n9 $o9 $t9 $t9 $w9 $v9 $u9 >>>> $u9 $z9 $x9 >>>> >>>> Pharo 50041 gives... >>>> $a1 $a2 $a3 $a4 $a5 $a6 $a7 $a8 $a9 $b1 $b2 $b3 $b4 $b5 $b6 $b7 $b8 $b9 $c1 >>>> $c2 $c3 $c4 $c5 $c6 $c7 $c8 $c9 $d1 $d2 $d3 $d4 $d5 $d6 $d7 $d8 $d9 $e1 $e2 >>>> $e3 $e4 $e5 $e6 $e7 $e8 $e9 $f1 $f2 $f3 $f4 $f5 $f6 $f7 $f8 $f9 $g1 $g2 $g3 >>>> $g4 $g5 $g6 $g7 $g8 $g9 $h1 $h2 $h3 $h4 $h5 $h6 $h7 $h8 $h9 $i1 $i2 $i3 $i4 >>>> $i5 $i6 $i7 $i8 $i9 $j1 $j2 $j3 $j4 $j5 $j6 $j7 $j8 $j9 $k1 $k2 $k3 $k4 $k5 >>>> $k6 $k7 $k8 $k9 $l1 $l2 $l3 $l4 $l5 $l6 $l7 $l8 $l9 $m1 $m2 $m3 $m4 $m5 $m6 >>>> $m7 $m8 $m9 $n1 $n2 $n3 $n4 $n5 $n6 $n7 $n8 $n9 $o1 $o2 $o3 $o4 $o5 $o6 $o7 >>>> $o8 $o9 $p1 $p2 $p3 $p4 $p5 $p6 $p7 $p8 $p9 $q1 $q2 $q3 $q4 $q5 $q6 $q7 $q8 >>>> $q9 $r1 $r2 $r3 $r4 $r5 $r6 $r7 $r8 $r9 $s1 $s2 $s3 $s4 $s5 $s6 $s7 $s8 $s9 >>>> $t1 $t2 $t3 $t4 $t5 $t6 $t7 $t8 $t9 $u1 $u2 $u3 $u4 $u5 $u6 $u7 $u8 $u9 $v1 >>>> $v2 $v3 $v4 $v5 $v6 $v7 $v8 $v9 $w1 $w2 $w3 $w4 $w5 $w6 $w7 $w8 $w9 $x1 $x2 >>>> $x3 $x4 $x5 $x6 $x7 $x8 $x9 $y1 $y2 $y3 $y4 $y5 $y6 $y7 $y8 $y9 $z1 $z2 $z3 >>>> $z4 $z5 $z6 $z7 $z8 $z9 >>>> >>>> (start your comparison at $b5) >>>> >>>> So in one axis Pharo has improved Transcript, but we didn't notice the >>>> significance of the use case we lost. >>>> >>>> cheers -ben >>> Please take a good look at Cuis' Transcript and consider using it. >>> >>> By default, the display is updated immediately, but without calling Morphic, it can even work with no UI framework at all. It does updates faster than Squeak or Visualworks: >>> >>> Time millisecondsToRun: [ 1000 timesRepeat: [ Transcript show: 'x' ] ]. 763. >>> >>> But if you want minimum overhead without immediate feedback: >>> >>> Time millisecondsToRun: [ Transcript showOnDisplay: false. 1000 timesRepeat: [ Transcript show: 'x' ]. Transcript showOnDisplay: true ]. 1. >>> "As fast as Pharo" >>> >>> It is also thread safe, and for Ben's example: >>> >>> Transcript clear. >>> [ >>> $a asciiValue to: $z asciiValue do: [ :c | >>> [ 1 to: 9 do: [ :i | Transcript show: c asCharacter printString , i printString , ' ' ] ] forkAt: 40 >>> ]. >>> ] forkAt: 41 >>> >>> it gives the same result as Pharo. >>> >>> The fact that the updates are not bound to Morphic also means that it is possible to do #show: deep in Morphic logic, without causing infinite loops or recursions, and get immediate feedback. It has proved to be a useful aid in debugging Morphic code. >>> >>> Cheers, >>> Juan Vuletich >>> >>> >> >> > >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by Esteban Lorenzano

> On 10 May 2015, at 10:28, ClÃ©ment Bera <bera.clement(a)gmail.com> wrote: > > > > 2015-05-09 23:21 GMT+02:00 Tudor Girba <tudor(a)tudorgirba.com <mailto:tudor@tudorgirba.com>>: > I do not think there are many people around here that would think that it is irrelevant if the Pharo VM can be developed in Pharo or not. Of course, it is important. > > So, the discussion should not go to challenge this direction, but rather in you telling us the use cases that you need supported. Please note that I did not say which exact code and how it should look like. I would be interested in learning about the use cases you have. I am quite certain that there are a number of ways to support them and when we work on GT it would be useful to have your use cases on our table. > > Well I need many lines to explain each point and there are many... I can talk here about a few points. Then I will deal with Esteban for most of them because it is difficult to explain without an interactive discussion. > > > Let me explain the use cases for the Transcript for example. The issues in Pharo are: > - The Transcript does not show the stream as it is printed. > - The Transcript does not inherit from Stream and thus cannot print with all the methods implemented in Stream. > - The Transcript does not allow the user to decorate the text with bold, italic or colors. sorryâ¦ you can do that with squeak's transcript? > > Usecase 1: Debug printing methods: In the VM you have debug printing methods, for example, to print the call stack. These methods are used from the VM simulator, to output the string in the Transcript, and in gdb, to ouput the string in the commandline. The commandline (FileStream stdout in Pharo) and the Squeak Transcript have the same behavior. In Pharo, the Transcript does not inherit from Stream so you can't use the required stream methods to print the debug printing method on the Transcript. In addition, some printing methods print a lot of things and it is important to show the stream as it is printed. > For this use-case, we want to keep the smallest difference between the gdb/commadline behavior and the VM simulator/Transcript behavior. If you implement advanced tooling in GT, you therefore need to implement gdb extensions (and lldb extensions because some of us use lldb instead of gdb) and maintain them. I don't think this is a solution. > > Usecase 2: CCode generation debugging: The CCodeGenerator or Slang translator translates Slang code into C code. Sometimes there is a bug. To debug, instead of generating the faulty C method into an external C file, we print only the faulty C method in the Transcript. Again, we want to keep the lowest difference between the real usecase (printing on the C file) and the debug usecase (printing on the Transcript). In Squeak the FileStream and the Transcript are both Stream, everything works as expected. In Pharo the Transcript has not the expected behavior. Again the method can be long, you can have to wait several seconds, so you'd like the transcript to show the stream as you print it. > > Usecase 3: VM simulation: Simulating the VM is quite slow, especially the machine code execution simulation. During the simulation process, the UI is non interactive and shows only every while what the simulator is doing in the Transcript. It is important as sometimes when debugging with a test at each machine code instruction it could take several hours before the UI is interactive again and you want to know what is going on. I don't complain that it takes several hours because the alternatives usually require days of debugging and we can launch the VM simulator overnight. In Pharo this does not work as expected. > > Usecase 4: In-image machine-code compilation: While working in the JIT compiler, sometimes the machine code generated for a bytecoded method is faulty. A common way of debugging it is to print the machine code instructions of the machine code version of the method in the Transcript. It can take a while to print, so it is important to have the Transcript showing the text as it prints. Then, the easiest way of debugging is to look at the machine code and understand what is wrong. For this purpose, we add text decoration to color jump addresses or the instructions where the instruction pointer was when the VM crashed. Then, in squeak, we can easily copy the decorated text to a workspace and generate a new version of the machine code method and compare. In machine code, it is very difficult to do analysis to have more information than just the decompiled text. We add some information while simulating because we know for example the address of specific trampolines, therefore we can print the name of the trampoline when we see that its address is called. Again, sometimes we also have to debug in gdb. In this case, we disassemble the machine code and compare it to the one from in-image compilation, so both printed strings have to be similar (similar text, same chariot returns). > > > > Another example is the complexity of the Pharo tools: > > While developing the VM, I have sometimes a VM partially working or with some plugins not working. In the Squeak image, I can open a workspace on top of this half-working VM and run do-its to see what is working and what is not. In the Pharo image, I can't do anything. You can't open the workspace without opening more advanced tools. I tried to open the Playground, but the first time there was a bug with Traits (Playground use Traits somehow and they were not working due to the new bytecode set not being finished), when that first bug was fixed I could not open it because it crashed simply the VM (I believe it tried to access an external file such as playground-cache). Currently, the Pharo team is trying to build a set of basic tools that have few dependencies to debug a partially working system (that I think you will use to debug glamour while editing it, because you cannot use the glamour inspector if glamour is not working). That would solve this issue. > But in no way this point is something that I can do alone to be able to develop the VM in Pharo. This has to be a community effort. And I am saying that because I can't be blamed not to work on the VM in Pharo if to do so I need to spend many months changing Pharo. > > > > An example that I believe is a problem in term of the community is the following: > > I added with Eliot the support for the new bytecode set. Currently, the Squeak image works with the new bytecode set but not the Pharo image. This is because only the Traits are broken, but this is something I could hardly figure out in the Pharo image because nothing is working as the GT tools use Traits. In Squeak I believe there are very few users of Traits so everything worked, and the test suite can reveal that the Traits are broken easily. > > Currently, the VM process to me is to first make new features work in Squeak, because it is simpler, and then make it work with Pharo, which is more complex. In the last section I discussed how Traits were a problem while implementing the new bytecode set. So what is the long term solution for this issue ? > - Will we have a bootstrap process that creates first a Trait-free Kernel and then build the Pharo Kernel out of it ? > - Do we forbid people to use Traits in the Pharo Kernel and does that make sense to have Traits in Pharo in this case ? > - If we don't do anything, maybe the Traits are only a slight difference with low impact in most cases and it's fine. But maybe there are many small aspects like Traits, such as the Slots the way they were used in GT recently (I don't blame GT or anything, it was just using features in the system that created issues for me), and maybe we reached a point where the complexity between the Pharo kernel and the Squeak kernel is big enough so that a VM developer will first make Squeak works when introducing new features and then deals with the complexity of Pharo ? > > So, what do we do ? I don't see any simple solution for this issue. And I believe there are people around that see as the only solution for this issue not to have the Pharo VM development process in Pharo because they will see it as a threat to what they want to do with Pharo. > > > > Best Doru ! > > PS: I am still using the GTInspector with additional views on graphs created with Roassal everyday and I still enjoy it. > > PS2: I am on vacation currently because I was getting crazy looking at machine code all day long, so I may not answer as quick as usually during the next week. > > > > Cheers, > Doru > > > > On Sat, May 9, 2015 at 9:31 PM, ClÃ©ment Bera <bera.clement(a)gmail.com <mailto:bera.clement@gmail.com>> wrote: > > > 2015-05-09 20:25 GMT+02:00 stepharo <stepharo(a)free.fr <mailto:stepharo@free.fr>>: > > > Le 9/5/15 20:16, ClÃ©ment Bera a Ã©crit : >> This whole conversation here shows very well the point that I tried to explain to Stef last week. I'm sorry if the mail is a bit long but I think this discussion has to be done. >> >> My whole Smalltalk development life, I have used Pharo and was happy with it. Now I am also working in Cog's JIT compiler and for this specific project, I am working with Squeak. I don't work with Squeak because I don't like Pharo, I told you before, I have worked with Pharo on all my project before, enjoyed it and if it was possible I would use Pharo. I work with Squeak because the VM development tool and development process simply does *not* work in Pharo. This is not only because of VM tools working with the old Morphic not working anymore in Pharo or details like that, it is also due to deeper changes in Pharo. >> >> Stef believes it is important that Pharo is able to host development for its own VM. Therefore, I discussed with him and Esteban about a first list of points that are necessary for Pharo to support its VM development in Pharo, which includes this Transcript behavior. >> >> As of today, and I am honest here, I believe that what is required for Pharo to support the development process of its VM includes points which goes in the opposite direction than a few points in the Pharo roadmap, that people in the Pharo community will see as a regression, as "an intrusion from the Squeak philosophy into Pharo", or as forbidding the integration of features that breaks the VM development process. Therefore, I believe the Pharo community would disapprove to make such changes and I highly doubt that it is possible to have the development process of the >> Pharo VM in Pharo. >> >> I was thinking that only a few points would be a problem such as the increasing memory footprint of the Pharo image that is going to get worse with the sources that will be included in the image in the future, whereas a VM developer needs a small image (See previous threads in this mailing list where Hilaire complains about that for example). > > clement can I ask a simple question? > why did I ask guille to work on minikernels and bootstrap for his phd instead on a topic where we can publish? > - choice A: lack of idea > - choice B: .... > > I have already stated that you believe that it is important that Pharo is able to host development for its own VM. > > I am not against what you did and I am very excited with Guille's work. > > Pharo is community-driven, so I am not asking the question to you only, but to the community. > > >> However, I didn't think that even simple points like the Transcript behavior discussed here, which looks like to me as a regression and is required for VM development, would be seen as an improvement by a non negligible part of the community. >> >> In this mailing-list, the whole Pharo community is present and can see this discussion. So the open questions are: >> >> Do you want to have the development of the Pharo VM in Pharo, or do you want the development of the Pharo VM to remain in Squeak ? >> Do you think a system that is not good enough to handle its own VM development is a good system ? >> >> I am not willing to go against the will of the community because I enjoy community-driven softwares. If the answer is that Pharo should be able to support its own VM development then as I started I will help Esteban and Stef to improve Pharo so that it can support its own VM development. Now, if the answer is that the development of the Pharo VM should remain in Squeak, I will continue developing the VM in Squeak. >> >> You are the Pharo community, you are the ones that make Pharo alive and kicking, so you tell me what you think we should do. >> >> Clement >> >> 2015-05-09 18:23 GMT+02:00 Eliot Miranda <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>>: >> Hi Ben, >> >> On May 9, 2015, at 7:41 AM, Ben Coman <btc(a)openinworld.com <mailto:btc@openinworld.com>> wrote: >> >>> >>> >>> On Sat, May 9, 2015 at 10:09 PM, Ben Coman <btc(a)openinworld.com <mailto:btc@openinworld.com>> wrote: >>> From my limited experience bug hunting, calling #changed: from a thread other than the UI thread is a source of evil. There are too many assumptions throughout the system that the UI is single threaded. Can anyone advise me that is not a proper belief? >>> >>> Then that implies that a Transcript implementation where #nextPut: direct calls #changed: >>> is not appropriate for use with multi-threaded applications. In Pharo, #changed: is only called from #stepGlobal, which is called from doOneCycle:. (This came about as a last minute bug fix before Pharo 3 release and maybe could use some cleanup. >>> >>> Separating the UI from Transcript into its own viewer might be a good idea, but actually it would not solve Stef's case since his code would still be running in the UI thread -- unless the viewer ran in another thread, which would have its own complexities. >>> >>> I think the point about efficiency is significant. The following example... >>> Time millisecondsToRun: [ 1000 timesRepeat: [ Transcript show: 'x' ] ] >>> on Squeak 4.5 --> 12749ms >>> on Pharo 50029 --> 2ms >>> >>> As a point of comparison, on VW 8.0 --> 43817ms >>> and so you might guess, VW 8.0 outputs each 'x' immediately. >>> cheers -ben >> >> Way to go, Squeak! Actually this is disappointing. I'm rather frustrated with Squeak's slow transcript, and was hoping that VW would demonstrate it could be faster. Looking at the Squeak implementation I only see an obvious 30% or so improvement via tuning. Looks like good performance will take more work :-/ >> >> >> >> Eliot (phone) >> > > > > > > -- > www.tudorgirba.com <http://www.tudorgirba.com/> > > "Every thing has its own flow" >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by Esteban Lorenzano

+1 letâs recapitulate: requirement is - fast - editable - polymorphic with WriteStream - and immediate to screen from our perspective (pharo), it has to be also - thread safe - well designed. our constraints are: - bad design of transcripts - super naive implementation of text editors (this is, IMO, the reason why transcripts are so slow: if you redraw all the window interior each time you add a line, you are screw) basicallyâ¦ we need a good log console (if it is a transcript or not, is another discussion) :) Soâ¦ who proposes a solution? Esteban > On 10 May 2015, at 00:40, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> wrote: > > For me saying that internals are not important as long as transcript does what is expected from it, is like saying that Parser being a subclass of Scanner and Compiler being as subclass of Parser is ok as long as it compiles source code. We already had that and it was bad. > > On the other hand itâs obvious that we donât have enough resources to rewrite Pharo from scratch and we cannot leave issues as they are just because we cannot solve them perfectly now. > > I would say that Igor is correct saying that we need a better architecture, and Stef and Eliot are correct saying that we need tools that fulfill usersâ needs. And we have to say: âok now we hack this around, but we have to do this the right way one dayâ. Is anyone familiar with same-css[1] concept? > > Uko > > P.S. maybe my 2 cents are useless, but all the fight around this discussion is really strange for me. In Ukraine we have a saying that itâs better to be healthy and poor than sick and rich. Now I use to say that itâs better to be healthy and rich than sick and poor. And all the fight here was if itâs better to be healthy or rich. Definitely itâs better to be both, we cannot do that instantly but we can set goals and milestones, and find the optimal strategy. > > > [1]: http://csswizardry.com/2013/04/shame-css/ <http://csswizardry.com/2013/04/shame-css/> >> On 09 May 2015, at 16:17, Eliot Miranda <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> wrote: >> >> >> >> On Sat, May 9, 2015 at 5:29 AM, stepharo <stepharo(a)free.fr <mailto:stepharo@free.fr>> wrote: >> Eliot >> >> I changed the transcript because it is not thread safe so I could use it at all to explain concurrent programming output. >> It was terrible. >> >> I don't see what that has to do with my usability point. The transcript was >> a) a stream >> b) something that displayed its output immediately >> >> Those two features are essential. If you change the transcript so that it no longer has those features you have broken it. >> >> Igor waffled on about internals, the desire to separate the UI from the stream, a point that has nothing to do with the utility of the transcript. No you're going on about its thread-safety. But no one is addressing my point. And you have ClÃ©ment telling you that he cannot develop the VM in Pharo because the transcript is broken, and you have others proposing various potentially error-prone work-arounds to get around the fact that the transcript is broken. >> >> Why won't anyone stand up and say yes, the transcript is broken and yes we will fix it? This is dysfunctional. >> >> >> Stef >> >> >> Le 8/5/15 16:16, Eliot Miranda a Ã©crit : >> >> Hi, >> >> if one uses a at doit transcript then no special action is required to get output to appear beyond sending flush to Transcript right? So any solution that requires special action to get the moronic transcript to work us broken. We should fix the transcript, not expect every application to work around a bug. >> >> Eliot (phone) >> >> On May 8, 2015, at 6:15 AM, Alain Rastoul <alf.mmm.cat(a)gmail.com <mailto:alf.mmm.cat@gmail.com>> wrote: >> >> Le 08/05/2015 11:34, stepharo a Ã©crit : >> Hi guys >> >> the Transcript in Pharo is that it's not asynchronous so I can't use it >> in VM development to show the current progress of the simulation. For >> example: >> 1 to: 100 do: [ :i | >> 0.1 seconds asDelay wait. >> Transcript show: 'x'. ] >> => on Squeak, this shows a x every 0.1 second in the Transcript >> => on Pharo, nothing happens during 10 seconds then all the x are shown. >> >> https://pharo.fogbugz.com/default.asp?15515 <https://pharo.fogbugz.com/default.asp?15515> >> Yes, as do it are evaluated in the World morphic process, running in a forked process or sending World doOneCycle in the loop solve the problem. >> >> Probably in squeak, in Transcript this is done somewhere under the hood. >> >> via dependents ? >> >> TranscriptStream>>endEndtry >> "Display all the characters since the last endEntry, and reset the stream" >> self semaphore critical:[ >> self changed: #appendEntry. >> self reset. >> ]. >> >> Object>>changed: aParameter >> self dependents do: [:aDependent | aDependent update: aParameter] >> >> And probably not doOnecycle since you cannot do anything else during execution (clicking or moving windows). >> >> >> -- >> Regards, >> >> Alain >> >> >> >> >> >> >> >> >> >> -- >> best, >> Eliot >

May 10, 2015

Re: [Pharo-dev] Transcript needs your love

by ClÃ©ment Bera

2015-05-09 23:21 GMT+02:00 Tudor Girba <tudor(a)tudorgirba.com>: > I do not think there are many people around here that would think that it > is irrelevant if the Pharo VM can be developed in Pharo or not. Of course, > it is important. > > So, the discussion should not go to challenge this direction, but rather > in you telling us the use cases that you need supported. Please note that I > did not say which exact code and how it should look like. I would be > interested in learning about the use cases you have. I am quite certain > that there are a number of ways to support them and when we work on GT it > would be useful to have your use cases on our table. > Well I need many lines to explain each point and there are many... I can talk here about a few points. Then I will deal with Esteban for most of them because it is difficult to explain without an interactive discussion. Let me explain the use cases for the Transcript for example. The issues in Pharo are: - The Transcript does not show the stream as it is printed. - The Transcript does not inherit from Stream and thus cannot print with all the methods implemented in Stream. - The Transcript does not allow the user to decorate the text with bold, italic or colors. *Usecase 1: Debug printing methods:* In the VM you have debug printing methods, for example, to print the call stack. These methods are used from the VM simulator, to output the string in the Transcript, and in gdb, to ouput the string in the commandline. The commandline (FileStream stdout in Pharo) and the Squeak Transcript have the same behavior. In Pharo, the Transcript does not inherit from Stream so you can't use the required stream methods to print the debug printing method on the Transcript. In addition, some printing methods print a lot of things and it is important to show the stream as it is printed. For this use-case, we want to keep the smallest difference between the gdb/commadline behavior and the VM simulator/Transcript behavior. If you implement advanced tooling in GT, you therefore need to implement gdb extensions (and lldb extensions because some of us use lldb instead of gdb) and maintain them. I don't think this is a solution. *Usecase 2: CCode generation debugging:* The CCodeGenerator or Slang translator translates Slang code into C code. Sometimes there is a bug. To debug, instead of generating the faulty C method into an external C file, we print only the faulty C method in the Transcript. Again, we want to keep the lowest difference between the real usecase (printing on the C file) and the debug usecase (printing on the Transcript). In Squeak the FileStream and the Transcript are both Stream, everything works as expected. In Pharo the Transcript has not the expected behavior. Again the method can be long, you can have to wait several seconds, so you'd like the transcript to show the stream as you print it. *Usecase 3: VM simulation:* Simulating the VM is quite slow, especially the machine code execution simulation. During the simulation process, the UI is non interactive and shows only every while what the simulator is doing in the Transcript. It is important as sometimes when debugging with a test at each machine code instruction it could take several hours before the UI is interactive again and you want to know what is going on. I don't complain that it takes several hours because the alternatives usually require days of debugging and we can launch the VM simulator overnight. In Pharo this does not work as expected. *Usecase 4: In-image machine-code compilation:* While working in the JIT compiler, sometimes the machine code generated for a bytecoded method is faulty. A common way of debugging it is to print the machine code instructions of the machine code version of the method in the Transcript. It can take a while to print, so it is important to have the Transcript showing the text as it prints. Then, the easiest way of debugging is to look at the machine code and understand what is wrong. For this purpose, we add text decoration to color jump addresses or the instructions where the instruction pointer was when the VM crashed. Then, in squeak, we can easily copy the decorated text to a workspace and generate a new version of the machine code method and compare. In machine code, it is very difficult to do analysis to have more information than just the decompiled text. We add some information while simulating because we know for example the address of specific trampolines, therefore we can print the name of the trampoline when we see that its address is called. Again, sometimes we also have to debug in gdb. In this case, we disassemble the machine code and compare it to the one from in-image compilation, so both printed strings have to be similar (similar text, same chariot returns). Another example is the complexity of the Pharo tools: While developing the VM, I have sometimes a VM partially working or with some plugins not working. In the Squeak image, I can open a workspace on top of this half-working VM and run do-its to see what is working and what is not. In the Pharo image, I can't do anything. You can't open the workspace without opening more advanced tools. I tried to open the Playground, but the first time there was a bug with Traits (Playground use Traits somehow and they were not working due to the new bytecode set not being finished), when that first bug was fixed I could not open it because it crashed simply the VM (I believe it tried to access an external file such as playground-cache). Currently, the Pharo team is trying to build a set of basic tools that have few dependencies to debug a partially working system (that I think you will use to debug glamour while editing it, because you cannot use the glamour inspector if glamour is not working). That would solve this issue. But in no way this point is something that I can do alone to be able to develop the VM in Pharo. This has to be a community effort. And I am saying that because I can't be blamed not to work on the VM in Pharo if to do so I need to spend many months changing Pharo. An example that I believe is a problem in term of the community is the following: I added with Eliot the support for the new bytecode set. Currently, the Squeak image works with the new bytecode set but not the Pharo image. This is because only the Traits are broken, but this is something I could hardly figure out in the Pharo image because nothing is working as the GT tools use Traits. In Squeak I believe there are very few users of Traits so everything worked, and the test suite can reveal that the Traits are broken easily. Currently, the VM process to me is to first make new features work in Squeak, because it is simpler, and then make it work with Pharo, which is more complex. In the last section I discussed how Traits were a problem while implementing the new bytecode set. So what is the long term solution for this issue ? - Will we have a bootstrap process that creates first a Trait-free Kernel and then build the Pharo Kernel out of it ? - Do we forbid people to use Traits in the Pharo Kernel and does that make sense to have Traits in Pharo in this case ? - If we don't do anything, maybe the Traits are only a slight difference with low impact in most cases and it's fine. But maybe there are many small aspects like Traits, such as the Slots the way they were used in GT recently (I don't blame GT or anything, it was just using features in the system that created issues for me), and maybe we reached a point where the complexity between the Pharo kernel and the Squeak kernel is big enough so that a VM developer will first make Squeak works when introducing new features and then deals with the complexity of Pharo ? So, what do we do ? I don't see any simple solution for this issue. And I believe there are people around that see as the only solution for this issue not to have the Pharo VM development process in Pharo because they will see it as a threat to what they want to do with Pharo. Best Doru ! PS: I am still using the GTInspector with additional views on graphs created with Roassal everyday and I still enjoy it. PS2: I am on vacation currently because I was getting crazy looking at machine code all day long, so I may not answer as quick as usually during the next week. Cheers, > Doru > > > > On Sat, May 9, 2015 at 9:31 PM, ClÃ©ment Bera <bera.clement(a)gmail.com> > wrote: > >> >> >> 2015-05-09 20:25 GMT+02:00 stepharo <stepharo(a)free.fr>: >> >>> >>> >>> Le 9/5/15 20:16, ClÃ©ment Bera a Ã©crit : >>> >>> This whole conversation here shows very well the point that I tried to >>> explain to Stef last week. I'm sorry if the mail is a bit long but I think >>> this discussion has to be done. >>> >>> My whole Smalltalk development life, I have used Pharo and was happy >>> with it. Now I am also working in Cog's JIT compiler and for this specific >>> project, I am working with Squeak. I don't work with Squeak because I don't >>> like Pharo, I told you before, I have worked with Pharo on all my project >>> before, enjoyed it and if it was possible I would use Pharo. I work with >>> Squeak because the VM development tool and development process simply does >>> *not* work in Pharo. This is not only because of VM tools working with the >>> old Morphic not working anymore in Pharo or details like that, it is also >>> due to deeper changes in Pharo. >>> >>> Stef believes it is important that Pharo is able to host development >>> for its own VM. Therefore, I discussed with him and Esteban about a first >>> list of points that are necessary for Pharo to support its VM development >>> in Pharo, which includes this Transcript behavior. >>> >>> As of today, and I am honest here, I believe that what is required for >>> Pharo to support the development process of its VM includes points which >>> goes in the opposite direction than a few points in the Pharo roadmap, that >>> people in the Pharo community will see as a regression, as "an intrusion >>> from the Squeak philosophy into Pharo", or as forbidding the integration of >>> features that breaks the VM development process. Therefore, I believe the >>> Pharo community would disapprove to make such changes and I highly doubt >>> that it is possible to have the development process of the >>> Pharo VM in Pharo. >>> >>> I was thinking that only a few points would be a problem such as the >>> increasing memory footprint of the Pharo image that is going to get worse >>> with the sources that will be included in the image in the future, whereas >>> a VM developer needs a small image (See previous threads in this mailing >>> list where Hilaire complains about that for example). >>> >>> >>> clement can I ask a simple question? >>> why did I ask guille to work on minikernels and bootstrap for his phd >>> instead on a topic where we can publish? >>> - choice A: lack of idea >>> - choice B: .... >>> >> >> I have already stated that you believe that it is important that Pharo is >> able to host development for its own VM. >> >> I am not against what you did and I am very excited with Guille's work. >> >> Pharo is community-driven, so I am not asking the question to you only, >> but to the community. >> >>> >>> >>> However, I didn't think that even simple points like the Transcript >>> behavior discussed here, which looks like to me as a regression and is >>> required for VM development, would be seen as an improvement by a non >>> negligible part of the community. >>> >>> In this mailing-list, the whole Pharo community is present and can see >>> this discussion. So the open questions are: >>> >>> *Do you want to have the development of the Pharo VM in Pharo, or do >>> you want the development of the Pharo VM to remain in Squeak ?* >>> *Do you think a system that is not good enough to handle its own VM >>> development is a good system ?* >>> >>> I am not willing to go against the will of the community because I >>> enjoy community-driven softwares. If the answer is that Pharo should be >>> able to support its own VM development then as I started I will help >>> Esteban and Stef to improve Pharo so that it can support its own VM >>> development. Now, if the answer is that the development of the Pharo VM >>> should remain in Squeak, I will continue developing the VM in Squeak. >>> >>> You are the Pharo community, you are the ones that make Pharo alive >>> and kicking, so you tell me what you think we should do. >>> >>> Clement >>> >>> 2015-05-09 18:23 GMT+02:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >>> >>>> Hi Ben, >>>> >>>> On May 9, 2015, at 7:41 AM, Ben Coman <btc(a)openinworld.com> wrote: >>>> >>>> >>>> >>>> On Sat, May 9, 2015 at 10:09 PM, Ben Coman <btc(a)openinworld.com> wrote: >>>> >>>>> From my limited experience bug hunting, calling #changed: from a >>>>> thread other than the UI thread is a source of evil. There are too many >>>>> assumptions throughout the system that the UI is single threaded. Can >>>>> anyone advise me that is not a proper belief? >>>>> >>>>> Then that implies that a Transcript implementation where #nextPut: >>>>> direct calls #changed: >>>>> is not appropriate for use with multi-threaded applications. In >>>>> Pharo, #changed: is only called from #stepGlobal, which is called from >>>>> doOneCycle:. (This came about as a last minute bug fix before Pharo 3 >>>>> release and maybe could use some cleanup. >>>>> >>>>> Separating the UI from Transcript into its own viewer might be a >>>>> good idea, but actually it would not solve Stef's case since his code would >>>>> still be running in the UI thread -- unless the viewer ran in another >>>>> thread, which would have its own complexities. >>>>> >>>>> I think the point about efficiency is significant. The following >>>>> example... >>>>> Time millisecondsToRun: [ 1000 timesRepeat: [ Transcript show: >>>>> 'x' ] ] >>>>> on Squeak 4.5 --> 12749ms >>>>> on Pharo 50029 --> 2ms >>>>> >>>> >>>> As a point of comparison, on VW 8.0 --> 43817ms >>>> and so you might guess, VW 8.0 outputs each 'x' immediately. >>>> cheers -ben >>>> >>>> >>>> Way to go, Squeak! Actually this is disappointing. I'm rather >>>> frustrated with Squeak's slow transcript, and was hoping that VW would >>>> demonstrate it could be faster. Looking at the Squeak implementation I >>>> only see an obvious 30% or so improvement via tuning. Looks like good >>>> performance will take more work :-/ >>>> >>>> >>>> >>>> Eliot (phone) >>>> >>> >>> >>> >> > > > -- > www.tudorgirba.com > > "Every thing has its own flow" >

May 10, 2015