Hi again, On 10/7/2014 2:42 PM, Thierry Goubier wrote:
Hi Florin,
I've changed the header for our discussion :)
good idea :)
Le 07/10/2014 05:29, Florin Mateoc a écrit :
Hi Thierry,
Thanks for pointing it out. It is indeed interesting, but not what I had in mind. I am not talking about debugging here so much. I am quite good at that, and I find it natural to build tools that help me do it (and of course, Smalltalk debuggers are very good already (restarting a method/block pretty much gives you the benefits of stepping backwards)), but debugging broken behavior is more about mapping an internalized model of the execution with what's really happening, seeing where they diverge. You are just checking that what you think you know is true. Of course, when you find out it's not, you gain insight/experience. But what to do when there is nothing to check? What if, like in the situation you mentioned, the algorithm is already working (so it's producing the correct/expected results)? But you still want to make it better....
First, when dealing with complex algorithms, a restart a method/a block does not match what a full trace allows, first because you may have side-effects (which defeats the restarting thing), second because you don't have to set the next halt point when you go backward and restart, third because the debugger may lock your environment (think about halting one of the Rectangle methods, for example). However, you may loose some of the context in the trace.
I was only comparing a debugger with restart capabilities to a debugger that can step backwards. Side-effects are equally bad for stepping backwards (you cannot unsend that tcp-ip packet nor make the native UI retract its actions). To debug side-effects (or UI in particular) specialized (unfortunately mostly ad-hoc, sometimes bug-specific (such as UI locking, UI resource leaks) tools need to be created anyway.
Second, your last statement is self-defeating: you want to make your algorithm better, which means you have the knowledge of what makes it tick and improve. I describe that process as moving in the design space of the algorithm, and that great tools are the ones which help you find the right direction in that design space.
I don't understand the self-defeating part. Indeed, I do start with some knowledge, and because of that, initially I can come up with some external representation, which reflects that understanding. After staring at it enough (in its dynamic behavior), I can notice ways to improve things. I "milk" the representation until the initial behavior is barely recognizable. At that point it is not helpful anymore. I need a different representation now to stare at, since I have the nagging feeling that I am still leaving something on the table. How do I get to the next one? Moving in the design space sounds exactly right. So, how do you do that? What I want help with is what I am not as good at. I want to
literally "see" what's wrong (not good enough) with the picture, since the brain has those very powerful visual pattern matching capabilities. They remain unused when you are lost in the minutiae of the debugger, where you only see text and numbers, the granularity/level of abstraction is too low. Profilers are a little better - they also succinctly capture the execution tree, and I have also used traces for hard to debug sequences.
The brain has some good abilities, but they are not what I'd call pattern matching (or at least you need to be very carefull with which visual variable you want to exploit: some are not suited at all to that type of visual analysis). I refer to Sémiologie Graphique, J. Bertin, for that.
What I have considered in the work I did in the past on that is that visual systems have natural abstraction capabilities if we use / exploit them, and that they avoid the problems associated with building a formal abstraction process which may miss the point or the analysis we may extract from that.
That is a very nice way of expressing it, except I would still say that it is not the visual system themselves that have the natural abstraction capabilities, it is a specialized part of our brain that extracts that automatically/unconsciously for us (what I called pattern matching (since it can only identify/notice things that it can match somehow)). But you expressed much better than I did why this is/would be so valuable. And I will check out the reference, thank you - the title sounds promising :).
For example, a very high level view of a profile or a trace, as a tree with just a one pixel wide line per item (with identation according to depth + length related to some other execution property) may convey a lot about an execution.
Another path that Alexandre Bergel makes wonder with is to use interaction to add and remove additional contextual data; you can't do comparative analysis visually with that, but, given a fast system, you may scan for a large amount of additional data without overloading the base display. I have used it to couple a code visualisation with a large C source file, with hovering over nodes jumping instantaneously to the right place in the source, and this is really effective for fast understanding of code structures (and loop / field of view focus effects).
Coming back to the WhyLine example, they already had something to (visually) look at: objects colliding. Other times, the visual representation is obvious/implied (say an algorithm for driving a car on a given course - even if all you are given are coordinates, it is common sense that you would draw the course on the screen and watch the car as it makes progress when the algorithm ticks). In many algorithms you don't have that. How do you find, in such cases, what pictures/graphs/animations to look at? For an algorithm one obvious candidate would be to the dynamic behavior of the main data structures used (although, say you do this for the car example and compare and contrast visualizing the data structures used by the algorithm vs visualizing the car moving, or the objects colliding). And you still have to find the right scale and level of abstraction.
I guess here you have ideas: heat maps on data accesses, evolving over time; a representation of all objects in memory, with their color which evolves according to freshness (animated). Could be very interesting to compare that way two versions of the same algorithm, to see how they differ in time? Trace may be very valuable there, because it may be tuned to record the type of events you are interested in, and then be exploited to build a visualisation on top.
With experience, what I value now a lot is code interaction: anything that very easily couple the visualisation with the relevant code (and not only the method, but the point in the code). I'm also dealing with safety properties of the code: safe, correct multithreaded code, either through run time analysis (see VectorFabrics tools :)) or through static analysis (Frama-C).
Interesting subject and I am totally out of available work hours to implement stuff on that :)
Thierry
Well, same here, and right now I am busy with another spare time project anyway (but I couldn't resist asking about your tools :) ), but if I ever manage to get back to polish that algorithm which has been in the back of my mind during this discussion (for the longest common subsequence), and which I worked on just for fun anyway, but which left me, as I said, with the nagging feeling that I could do more (and I don't mean some peephole optimization), I will try out some of these visualization techniques to see if anything comes out of it. Florin