Needless to say it runs only on pharo6? :) Norbert

Am 18.07.2016 um 12:19 schrieb Norbert Hartl <norbert@hartl.name>:

Awesome! I'll immediately will stop all my important work and start playing with it. I'm really interested how it will perform over high latency connections.

Well done,

Norbert

...

Am 18.07.2016 um 12:03 schrieb Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>>:

I glad to release first version of RemoteDebuggingTools project. It allows explore and debug remote images.

You can read details at http://dionisiydk.blogspot.fr/2016/07/remote-debugging-tools-is-ready.html <http://dionisiydk.blogspot.fr/2016/07/remote-debugging-tools-is-ready.html> and watch demo at https://youtu.be/SgFjgQpo_nU <https://youtu.be/SgFjgQpo_nU>.

Any feedback is welcome.

Best regards, Denis.

2016-07-18 12:33 GMT+02:00 Norbert Hartl <norbert@hartl.name>:

Needless to say it runs only on pharo6? :)

I make special version of Seamless which works with Pharo 5. So both Pharo 5 and 6 are now supported. Just use latest config of RemoteDebuggingTools. All tests are green.

Thank's 2016-07-18 12:19 GMT+02:00 Norbert Hartl <norbert@hartl.name>:

Awesome! I'll immediately will stop all my important work and start playing with it. I'm really interested how it will perform over high latency connections.

Yes, please report your experience. Currently all optimisations are done on minimizing network communication without tools modification. So debugger is same GTDebugger but communication is configured accordingly to debugger behaviour. For example when debugger requests suspendedContext of remote process server returns context proxy together with method, receiver, receiver class name and other properties which debugger is going to use. Other optimization could be done on protocol level. Now transferred objects are traversed twice: by Seamless and by Fuel. It could be merged somehow.

Well done,

Hi Norbert. Big thank's for feedback. I commit new version of Seamless with compressed protocol (by gzip). Not big change but results are better. I see ~25% improvements. Here is statistics of my Seaside demo with "gziped" fuel protocol (it is default now): - 3 kb to connect debugger - 100 kb for opening debugger preview. - 100 kb for full debugger - 60 kb for selecting another context. - 868 bytes to evaluate "[1+2]" (with returned result). I improved logging in this version to collect "bytes statistics". You can analyze communication by starting SeamlessLogger and collecting statistics when you are ready: SeamlessLogger startAfresh. "to clear old history if exists" SeamlessLogger collectStatistics. "inspect it when you are ready to look at what was happen" Here is what you will see for debugger connecting: ​ 2016-07-18 19:11 GMT+02:00 Norbert Hartl <norbert@hartl.name>:

Ok, I experimented a bit with network qualities. I just tested connecting and executing 3 + 4 on the remote side. Connecting transfers already 2-4kb of data which should add some latency when the network is bad. Local wifi and DSL connections work ok for the simple case because latencies are low.

On a 3G network it can take between 4 and 7 seconds to establish the connection and around half a second to execute 3 + 4. I defined a bad mobile network with 20kb/s bandwidth, 10% packet loss and a delay of 100ms in transmitting. This is equally for down and up directions. The connection time raises to 11 seconds and the execution of 3 + 4 to 4 seconds.

In order to be able to test I needed to tweak BasysNetwork>>#initialize to set a timeout greater than 45 milliseconds which is pretty low. Usual timeout settings for browsers and mobile applications are around 30 seconds. I raised it to 10 seconds in order to test the bad network.

To me the results are more or less expected. Well, I hoped they would be much better :) It works reliably even over slow networks. The main problem is that when it comes to remote debugging environments or browsers the number of message sends is quite high. So it is to see what kind of environment these tools can serve. For sure there is plenty of space to optimize (I hope). I did the tests with bad/mobile networks just get a clear mark where it cannot be used. To me personally the benchmark is DSL upload speeds. If it is doable over these (could be 256k/s) I can connect from the office (DSL down) the my little devices at home (DSL up) ;)

I see big room for optimizations here. Look at very small unit of communication: delivering integer result (from [1+2] evaluation). Following object is responsible for this: request := SeamlessDeliverResultRequest result: (SeamlessReturnValueResult with: 3) to: (SeamlessRequestContextReference id: 4 peerId: UUID new) . stream := ByteArray new writeStream. FLSerializer newDefault serialize: request on: stream. bytes := stream contents. bytes size. "==> 412" (ObjectTraveler on: request ) countReferences "==>10" Last line shows that only 10 objects consists full object graph of request. And Fuel serialization takes for it 412 bytes. It definitely could be improved with specialized serialization protocol. Also it could allow object traversal once which will lead to another performance improvements.

2016-07-29 10:20 GMT+02:00 Denis Kudriashov <dionisiydk@gmail.com>:

Hi Norbert.

2016-07-19 16:11 GMT+02:00 Denis Kudriashov <dionisiydk@gmail.com>:

...

I see big room for optimizations here. Look at very small unit of communication: delivering integer result (from [1+2] evaluation). Following object is responsible for this:

request := SeamlessDeliverResultRequest result: (SeamlessReturnValueResult with: 3) to: (SeamlessRequestContextReference id: 4 peerId: UUID new) . stream := ByteArray new writeStream. FLSerializer newDefault serialize: request on: stream. bytes := stream contents. bytes size. "==> 412" (ObjectTraveler on: request ) countReferences "==>10"

Last line shows that only 10 objects consists full object graph of request. And Fuel serialization takes for it 412 bytes. It definitely could be improved with specialized serialization protocol. Also it could allow object traversal once which will lead to another performance improvements.

I start implement optimized protocol and I got really nice results. I reduce size of packet in my example to 16 bytes. I not measure performance yet but it should be better too. Also new protocol will allow application specific optimisations. So soon we will have really usable tools for your case with slow network.

Nice!

2016-07-29 10:45 GMT+02:00 Norbert Hartl <norbert@hartl.name>:

That sounds wonderful! Thank you very much! Is the new code online?

Not yet. Probably I will finish it next week.

On Mon, Jul 18, 2016 at 6:19 PM, Norbert Hartl <norbert@hartl.name> wrote:

Awesome! I'll immediately will stop all my important work and start playing with it. I'm really interested how it will perform over high latency connections.

Well done,

Norbert

Am 18.07.2016 um 12:03 schrieb Denis Kudriashov <dionisiydk@gmail.com>:

I glad to release first version of RemoteDebuggingTools project. It allows explore and debug remote images.

You can read details at http://dionisiydk.blogspot.fr/2016/07/remote-debugging-tools-is-ready.html

Cool stuff Dennis. Does it utilise the remote image's DebuggerUI, or DebuggerSession directly? i.e. Is the aim to remotely debug a headless/UI-less image? This could open up some great possibilities running a minimal image on embedded hardware. When I read "Async evaluation ... does not wait for a result" I wonder if there a particular need for the fork is used here?... debugger evaluateAsync: [ [1/0] fork ]. An interesting proof demonstration would be when setting up the server image do... Smalltalk at: #whoami put: #IAmTheServer then in the client do... debugger evaluate: [ Smalltalk at: #whoami ] although maybe its hard to bind to the remote Smalltalk rather than the local one? What is the tradeoff of the #evaluate: parameter being a string rather than a block? Without having looked at the Seamless mechanism, intuitively there would seem some benefit having the remote system compile the string to bind to the remote classes. Another interesting demo might be remotely evaluating... Object inform: 'Test' and the notification showing up on the remote (if indeed that is the desired behaviour)

and watch demo at https://youtu.be/SgFjgQpo_nU.

I got a bit lost which one was client and which remote. Maybe the next demo you could have the server and client side by side. A bit of overlap should be okay but it would be useful for the debug notification appearing in the client to be immediately visible when an error is is caused in the server image. It would be intuitive if the web browser fully overlapped the server image it connected to. cheers -ben

On Mon, Jul 18, 2016 at 10:26 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

Hi Ben.

2016-07-18 14:46 GMT+02:00 Ben Coman <btc@openinworld.com>:

...

Cool stuff Dennis. Does it utilise the remote image's DebuggerUI, or DebuggerSession directly? i.e. Is the aim to remotely debug a headless/UI-less image? This could open up some great possibilities running a minimal image on embedded hardware.

Local debugger is opened on remote process. No UI needed to work with remote image. Also there is command line option to start image with running server:

./pharo PharoWithServer.image debuggerServer --port=40423

With Marcus we played with beaglebon device. It works fine like in my demo.

I've been meaning to get a BBB. I looks like bit better platform for an industrial controller / robotics than the Rpi - in particular its Programming Realtime Unit. Knowing someone else has one maybe I'll get one now. In the back of my mind I've been considering whether its PRU could be modelled as a CPU target for Slang , so Slang could be re-purposed from just compiling VM, to compiling embedded programs for the PRU, developed from within Pharo and loaded on the fly into the PRU.

...

When I read "Async evaluation ... does not wait for a result" I wonder if there a particular need for the fork is used here?... debugger evaluateAsync: [ [1/0] fork ].

Yes, it looks strange. Now all remote requests are executed such way that errors will be returned as SeamlessRemoteException and server signaller process will die. So instead of debugger with failed remote expression you will see debugger with local process which performed remote message send (but in case of async evaluation no result will returned to client and error will be just ignored). To escape such "exception handler" fork is needed. Fork will lead to unhanded error which will ask registered remote debugger to debug it.

Could you not just fork at the remote side? Perhaps the invocation would be #evaluateFork:

It was simplest strategy for error handling and in future it is probably possible to implement real smalltalk semantics over distributed errors.

...

An interesting proof demonstration would be when setting up the server image do... Smalltalk at: #whoami put: #IAmTheServer

then in the client do... debugger evaluate: [ Smalltalk at: #whoami ]

It works exactly like you want. Evaluation scripts transfer block to remote side by value. And for blocks "by value" means transfer all literals as "well known objects" (which are known on remote side). If globals not exist on remote side then error will be returned. Some literals could be transferred by reference. For example workspace variables.

...

although maybe its hard to bind to the remote Smalltalk rather than the local one? What is the tradeoff of the #evaluate: parameter being a string rather than a block?

In this case you can't execute scripts which reference local objects like self, temps or workspace variables. Also it would be impossible to implement non local return with regular semantics.

...

Without having looked at the Seamless mechanism, intuitively there would seem some benefit having the remote system compile the string to bind to the remote classes.

There is scenario which needs to be covered: debugging images which has no compiler. I probably will work on it at some point.

...

Another interesting demo might be remotely evaluating... Object inform: 'Test' and the notification showing up on the remote (if indeed that is the desired behaviour)

Yes, it works: https://youtu.be/Aop7KOGOFN0

HAharrrgh!! Thats *very* cool indeed !! cheers -ben

On Tue, Jul 19, 2016 at 10:30 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:

2016-07-18 18:26 GMT+02:00 Ben Coman <btc@openinworld.com>:

...

I've been meaning to get a BBB. I looks like bit better platform for an industrial controller / robotics than the Rpi - in particular its Programming Realtime Unit. Knowing someone else has one maybe I'll get one now. In the back of my mind I've been considering whether its PRU could be modelled as a CPU target for Slang , so Slang could be re-purposed from just compiling VM, to compiling embedded programs for the PRU, developed from within Pharo and loaded on the fly into the PRU.

Do you have link to BBB? (google not helps with such 3 letters).

BeagleBoneBlack. btw here is a nice overview of the PRU... http://beagleboard.org/static/presentations/MakerFaireNY20140920_UsingBeagle... and some videos... http://beagleboard.org/pru cheers -ben

...

...

...

When I read "Async evaluation ... does not wait for a result" I wonder if there a particular need for the fork is used here?... debugger evaluateAsync: [ [1/0] fork ].

Yes, it looks strange. Now all remote requests are executed such way that errors will be returned as SeamlessRemoteException and server signaller process will die. So instead of debugger with failed remote expression you will see debugger with local process which performed remote message send (but in case of async evaluation no result will returned to client and error will be just ignored). To escape such "exception handler" fork is needed. Fork will lead to unhanded error which will ask registered remote debugger to debug it.

Could you not just fork at the remote side? Perhaps the invocation would be #evaluateFork:

Here is screen which shows "evaluate: [1/0]":

​No remote stack here. And on remote side there is no waiting process.

In future real exception semantics could be supported. In that case "evaluate: [1/0]" will open debugger with half stack from local process (doIt from workspace) and half from remote process (where ZeroDivide happens).

("debugger evaluateAsync: [1/0]" will never show any error because result is not supposed to be returned for async)