Hi Ralph, Currently I know three job opportunities now open in Argentina, if you're willing to relocate I can help you with the hosting during the interviewing period. Please contact me privately for details (or any smalltalker in your position). Cheers, Hernán 2010/1/22 Ralph Boland <rpboland@gmail.com>:

...

On Fri, Jan 22, 2010 at 1:32 PM, St?phane Ducasse <stephane.ducasse@inria.fr

...

wrote:

...

Hi ralph and eliot

Wow, for once one of my Ideas seems to be popular, even if, apparently, it is not new.

Some points:

1)  I had not thought of the advantages of using Smalltalk to debug/develop an application remotely.  Commenters have made clear that the remote aspect is very useful.  I was just thinking of development in general where my application and development images are on the same machine. Does it make sense to first create a system having both a development image and application image and once that is complete add the capability to work remotely?  Or is it best that both be implemented in parallel? If the two tasks can be separated then I suggest we worry about achieving the first while keeping the second in mind during development.

2)  While this seems like an exciting project to me I do not have the necessary skills/knowledge.  The best I could offer would be as a serf in some corner implementing some component that I could handle.  Actually I cannot really offer that because I have my own open source project that I am working on for the next several man years.  Worse still I am going broke (Hi Keith) and desperately need to find a job.  Unfortunately Smalltalk jobs in Canada are extremely rare so I can going to have to program in C or Java or learn Python or  Php or Ruby (any recommendations as to which of those languages is the best) just so I can be employed. I would really like to see this project go forward though. I also much prefer some current version of Smalltalk make this modification rather than forking a new Smalltalk (call it  Orapple?).  Can this project go forward in stages,  say in 4 stages, with a stage completed with each major release? Could the project be a loadable package or are things too intimate for that?

3)  I don't know much about stripping but I don't believe this idea removes it. Even if the application and IDE are separated the application image is likely to have many things not meant for the client such as application specific debugging tools (tools separate from the IDE),  testing software such as SUnit tests, features meant for one client but not another,  etc.  As pointed out, however, stripping becomes much easier and effective if the IDE is separated from the image.    An alternative to stripping is to have three images,  The IDE image, the application development version image and the application release image.  Then the application release image never has anything in it that needs to be stripped (ignoring the multiple client issue). Admittedly, having three imaging running at the same time seems a bit wild. To be honest, I really don't know if having a third image is a good idea. I wouldn't worry too much about this though until we actually have a much progressed real project on our hands.

Regards,

Ralph Boland  (Available Smalltalker,  will travel anywhere)

...

...

...

Nevertheless there are advantages to both and I wonder to what extent I can have my cake and eat it too  (that would be an orapple cake I suppose).  I should also point out that my discussion is abstract; I am asking if it would have been better if Smalltalk had been designed the way I propose and not should some Smalltalk, say Squeak, now be modified to be so.

Yes I think that we can the best of both world. A minimal (but not smaller) runtime with a good separation between the infrastructure for the runtime and the loadeble ide and others.

...

The thing, particular to this discussion, about Eclipse that I like is that the Eclipse (image I will call it) is separate from the application (image I will call it).  In Smalltalk the two images are combined.  This means that a lot of code is encorporated into your application image that you may not want.

I imagine that in Pharo in a couple of versions this will be different. At least this is where we want to go.

...

Of course, before you release your application you can run a stripping program or process that strips out code that is not used in your application. (I have never done this but my understanding is that this is possible.)

Stripping should not be the process.

...

Never having done this I assume that stripping is not that satisfactory because there is much code that cannot be stripped out; certainly unused methods and likely unused classes as well; the dynamic nature of Smalltalk means that much necessary information to determine that a method or class is not used is not available so such methods and classes must be included in the application release.

Stripping in VW was not satisfactory not really easy.

It was not easy precisely because it had to strip out its own tools.  If the tools are remote then stripping is much easier.  I'm not proposing stripping as a standard step in deploying applications.  I;m proposing stripping via remote tools as a way to get down to a small kernel image.  Part of the difficulty in producing the small kernel image is needing to keep tools around so that the development system keeps working.  This is all much easier if that development system is remote.

...

...

Some may argue, that with all the memory processing power available today, who cares; but I care.  For example, I would like to be able to put a Smalltalk application into a linux pipe as in:

a | b > c

where a or b or both are applications written in Smalltalk.

me too :)

...

Similarly I would like to use Smalltalk applications in bash (Linux) shells scripts or use it in other scripting languages. So it may be important that my image not be unnecessarily large as getting it started may take longer than what it does once it is started.

Yes we want that too. So any help in that direction is welcomed.

...

Of course I could use GNU Smalltalk for this but is it really necessary that I know/use two smalltalks?

I think the Eclipse model suggests an improvement to Smalltalk. It seems to me that when you run Smalltalk ideally you would run two images.  One would be the development environment image and the other would be the application image.  When you started a new project you would choose from a selection of start images each including some subset of the development classes you expect to need.

for minimal image to work, we need a way to manage parts this is why metacello is important. Because you want to have ways to build coherent images out of components with dependencies. This is just because of that that we did not started to heavily remove packages from the image.

...

I can't speak in detail to the Firewall work done at Digitalk by people like Steve Messick, but they did get stuff done in the late 90's, although I don't think anything was productised.  I think they made the mistake of focussing on being able to produce very small executables rather than just separating the development and deployment images.  I think Steve was able to produce a tiny executable that only contained SmallInteger and hence was able to eliminate tagged pointers and method lookup.  But the focus on optimization was I think a step too far.

+ 1.

...

One thing I remember was that the communications stub in the deployment image was in the VM, not in Smalltalk code, again I think a mistake. Can anyone from Digitalk put flesh on these bones? ?

eliot what is the communications stub

Is is some code that sits on a socket in the target image and receives remote message send requests from the UI image, evaluates them and returns the results.  So it is a minimal communications port through which the UI can send messages to objects in the target image.  It should be written entirely in Smalltalk for flexibility and simplicity.  We don't need to get to 100's of bytes of image (e.g. being able to implement the unix mv command) we need to get down below 1Mb.

I think Craig has such a remote messaging scheme in Spoon.

...

...

We spent a lot of time discussing doing this in the VisualWorks team about 10 years ago but as far as I'm aware none of it was realised.  Splitting tools UIs from the image under development (the target image) was a major design goal behind OpenTalk, the distributed messaging framework that Xu Wang did.  We had the idea of the Model in an MVC triad being the interface between the UI and target images, the natural point at which one would define an API narrow enough to be efficiently distributed.  If the API is designed appropriately one would be exchanging symbols (class names, selectors etc) not complex objects (classes, compiled methods) and that because symbols and numbers are immutable they can be passed by value.

We had the idea of the model being split in two, one half being specific to a tool and the same whether one was in a single image or in the ui/target pair, and the other half being an adaptor to either the local image or the remote target.  This adaptor would have sat between a model and some complex object graph (like the class hierarchy) and defined the entire API through which the object graph was accessed, narrowing the interface to make it suitable for distribution, and doumenting the interface to make it easier to understand.  Vassili named these things between the model and an object graph TTIBs for "The Things In Between".

Was using distribution just one way to make sure that your two systems were decoupled? Because we could achieve the same using seaside? or

Exactly.  The TTIB serves as a specification and a filter.  But it is independent of the transport.  Seaside is a valid transport.  If things are done right one can have many different kinds of UI.  But one is making the choice that the transport is Smalltalk messages over some non-Smalltalk stream.  No one is contemplating using SOAP or CORBA or... (I hope).

...

...

We very much wanted to produce a set of tools in a tools UI image that could be targetted either at itself or at a remote target, and wanted to be able to attach the tools image to a remote target dynamically (which we called "dynamic capitation", for dynamically gaining a head) so that one could have full debuggability on a remote image deployed in the field.

:) I would really like to know what are the issue at the level of the Display.... VM code. What are the existing hypotheses that have to be broken. Like global access to sensor or

Yes, lots of things break potentally.  Transcript, self confirm: etc.  Part of the challenge is in coming up with "pluggable" replacements.  So that for example Transcript in the target image is something that sends its output to stdout until one attaches a tools UI, at which it becomes something that sends its output to the Transcript in the UI image.  This takes some thought :)

...

Alas we never managed to get the priority of the project high enough to actually commit major resources to it.  There were always more pressing more mundane things to do.  Frustration at not doing things like this was in part behind my leaving Cincom.  Xu got quite far with a prototype, demonstrating things like a distributed inspector that allowed one to walk though an object graph spanning more than one image.  The inspector would change colour to indicate which machine the current object was one.

...

Personally I think that Smalltalk is the ideal vehicle for this kind of

approach.  We have lightweight remote messaging which is relatively easy to implement.  We have UI architectures which are relatively well-decomposed and amenable to distribution.  We have a reified exception system which is not pushed down into hidden execution machinery and which has resume semantics.  We now have a much faster much more reliable internet.  We could relatively easily add things like delegation which can enable interesting interleaving of tools and target images (see below).  I think we could have something truly revolutionary, the ability to interact with a remote headless image that contains no development tools or UI frameworks and only a relatively small communications module (remote messaging interface).  This would enable us to get to images of a few 10s or 100s of kilobytes.  I fervently hope you work on this.  Lots of people want this.  I was reading Colin's Thin Air pages on wiresong yesterday and what he wants is realised by this.

Yes this would be good to have that. I think that cleaning the left over of years of experimentations (which we started with 3.6 Kernel Cleaning project and continue in 3.9 and now is taking a large part in current pharo) is the first phase so that we can make sure kind of system happening.

...

If you do work on this remember your goals.  The most important thing is incremental deliverables and useful progress.  Don't focus on whizzy optimisations or cool demos (such as the inspector changing colour).  Dn't focus on arbitrarily complex topologies of remote images.  Instead focus on building a robust, well-engineered remote tools framework that just allows one to do what one can do today in a browser and debugger, but remotely between two images.

We were thinking just about that with marcus and noury for mariano PhD. this could be used to debug remote image running on robots.

...

I would start with browser, debugger and stripper, where stripper would be something that allowed you to eliminate code from the target image using the UI image to discover what was there and what was needed or not, i.e. remote stripping, not the much trickier auto-stripping.

On delegation I think one really cool thing is to wrap remote objects with local development-only code.  In a deployed image I would expect there to be no inspectorClass methods or debugPrintOn: messages etc.  The target is lean and mean.  Anything not needed for deployment except that which supports adequate remote messaging is absent.  So one wants to be able to add methods that add debugging features to remote objects, and delegation is the way.  If the local handle on a remote object is wrapped by a delegate, the delegate can implement the debugging methods.  Catching doesNotUnderstand: in the target mage can allow one to intercept computations in the target and resume them in the development image.  So when connecting a tools image to a target one would layer tools and UI code upon objects in the target but that code would reside only in the tools image.

Naming conventions in packaging, like splitting a Foo package into Foo-Deployment & Foo-Development could help in making this kind of split easy to navigate and conceptualise.  So after producing the tools/target split I would next focus on packaging tools, including analysis of call graphs etc to help automating the deployment/development split.

You forget deliver often :)

Right!

best Eliot

...

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

...

...

Wow, for once one of my Ideas seems to be popular, even if, apparently, it is not new.

ralph this is not by accident that we change Smalltalk into self class environment. Nor that we introduce SmalltalkImage current this was with the idea that we could have remote browsing minimal images :) We started to clean for such a kind far too long ago and were not good to bring people with us. but now we will continue.

Some points:

1) I had not thought of the advantages of using Smalltalk to debug/develop an application remotely. Commenters have made clear that the remote aspect is very useful. I was just thinking of development in general where my application and development images are on the same machine. Does it make sense to first create a system having both a development image and application image and once that is complete add the capability to work remotely? Or is it best that both be implemented in parallel? If the two tasks can be separated then I suggest we worry about achieving the first while keeping the second in mind during development.

2) While this seems like an exciting project to me I do not have the necessary skills/knowledge. The best I could offer would be as a serf in some corner implementing some component that I could handle.

having a clean image is the first step so removing any bad or simple code is at your hand :)

Actually I cannot really offer that because I have my own open source project that I am working on for the next several man years. if it is open-source did you announce it? because we did not see it passing around

Worse still I am going broke (Hi Keith) and desperately need to find a job. Unfortunately Smalltalk jobs in Canada are extremely rare so I can going to have to program in C or Java or learn Python or Php or Ruby (any recommendations as to which of those languages is the best) just so I can be employed.

Did you send a mail around stating that you were looking for job? You can send a CV to ESUG Pay attention to you this is the most important aspect. stef

I would really like to see this project go forward though. I also much prefer some current version of Smalltalk make this modification rather than forking a new Smalltalk (call it Orapple?). Can this project go forward in stages, say in 4 stages, with a stage completed with each major release? Could the project be a loadable package or are things too intimate for that?

3) I don't know much about stripping but I don't believe this idea removes it. Even if the application and IDE are separated the application image is likely to have many things not meant for the client such as application specific debugging tools (tools separate from the IDE), testing software such as SUnit tests, features meant for one client but not another, etc. As pointed out, however, stripping becomes much easier and effective if the IDE is separated from the image. An alternative to stripping is to have three images, The IDE image, the application development version image and the application release image. Then the application release image never has anything in it that needs to be stripped (ignoring the multiple client issue). Admittedly, having three imaging running at the same time seems a bit wild. To be honest, I really don't know if having a third image is a good idea. I wouldn't worry too much about this though until we actually have a much progressed real project on our hands.

Regards,

Ralph Boland (Available Smalltalker, will travel anywhere)

...

...

...

Nevertheless there are advantages to both and I wonder to what extent I can have my cake and eat it too (that would be an orapple cake I suppose). I should also point out that my discussion is abstract; I am asking if it would have been better if Smalltalk had been designed the way I propose and not should some Smalltalk, say Squeak, now be modified to be so.

Yes I think that we can the best of both world. A minimal (but not smaller) runtime with a good separation between the infrastructure for the runtime and the loadeble ide and others.

...

The thing, particular to this discussion, about Eclipse that I like is that the Eclipse (image I will call it) is separate from the application (image I will call it). In Smalltalk the two images are combined. This means that a lot of code is encorporated into your application image that you may not want.

I imagine that in Pharo in a couple of versions this will be different. At least this is where we want to go.

...

Of course, before you release your application you can run a stripping program or process that strips out code that is not used in your application. (I have never done this but my understanding is that this is possible.)

Stripping should not be the process.

...

Never having done this I assume that stripping is not that satisfactory because there is much code that cannot be stripped out; certainly unused methods and likely unused classes as well; the dynamic nature of Smalltalk means that much necessary information to determine that a method or class is not used is not available so such methods and classes must be included in the application release.

Stripping in VW was not satisfactory not really easy.

It was not easy precisely because it had to strip out its own tools. If the tools are remote then stripping is much easier. I'm not proposing stripping as a standard step in deploying applications. I;m proposing stripping via remote tools as a way to get down to a small kernel image. Part of the difficulty in producing the small kernel image is needing to keep tools around so that the development system keeps working. This is all much easier if that development system is remote.

...

...

Some may argue, that with all the memory processing power available today, who cares; but I care. For example, I would like to be able to put a Smalltalk application into a linux pipe as in:

a | b > c

where a or b or both are applications written in Smalltalk.

me too :)

...

Similarly I would like to use Smalltalk applications in bash (Linux) shells scripts or use it in other scripting languages. So it may be important that my image not be unnecessarily large as getting it started may take longer than what it does once it is started.

Yes we want that too. So any help in that direction is welcomed.

...

Of course I could use GNU Smalltalk for this but is it really necessary that I know/use two smalltalks?

I think the Eclipse model suggests an improvement to Smalltalk. It seems to me that when you run Smalltalk ideally you would run two images. One would be the development environment image and the other would be the application image. When you started a new project you would choose from a selection of start images each including some subset of the development classes you expect to need.

for minimal image to work, we need a way to manage parts this is why metacello is important. Because you want to have ways to build coherent images out of components with dependencies. This is just because of that that we did not started to heavily remove packages from the image.

...

I can't speak in detail to the Firewall work done at Digitalk by people like Steve Messick, but they did get stuff done in the late 90's, although I don't think anything was productised. I think they made the mistake of focussing on being able to produce very small executables rather than just separating the development and deployment images. I think Steve was able to produce a tiny executable that only contained SmallInteger and hence was able to eliminate tagged pointers and method lookup. But the focus on optimization was I think a step too far.

+ 1.

...

One thing I remember was that the communications stub in the deployment image was in the VM, not in Smalltalk code, again I think a mistake. Can anyone from Digitalk put flesh on these bones? ?

eliot what is the communications stub

Is is some code that sits on a socket in the target image and receives remote message send requests from the UI image, evaluates them and returns the results. So it is a minimal communications port through which the UI can send messages to objects in the target image. It should be written entirely in Smalltalk for flexibility and simplicity. We don't need to get to 100's of bytes of image (e.g. being able to implement the unix mv command) we need to get down below 1Mb.

I think Craig has such a remote messaging scheme in Spoon.

...

...

We spent a lot of time discussing doing this in the VisualWorks team about 10 years ago but as far as I'm aware none of it was realised. Splitting tools UIs from the image under development (the target image) was a major design goal behind OpenTalk, the distributed messaging framework that Xu Wang did. We had the idea of the Model in an MVC triad being the interface between the UI and target images, the natural point at which one would define an API narrow enough to be efficiently distributed. If the API is designed appropriately one would be exchanging symbols (class names, selectors etc) not complex objects (classes, compiled methods) and that because symbols and numbers are immutable they can be passed by value.

We had the idea of the model being split in two, one half being specific to a tool and the same whether one was in a single image or in the ui/target pair, and the other half being an adaptor to either the local image or the remote target. This adaptor would have sat between a model and some complex object graph (like the class hierarchy) and defined the entire API through which the object graph was accessed, narrowing the interface to make it suitable for distribution, and doumenting the interface to make it easier to understand. Vassili named these things between the model and an object graph TTIBs for "The Things In Between".

Was using distribution just one way to make sure that your two systems were decoupled? Because we could achieve the same using seaside? or

Exactly. The TTIB serves as a specification and a filter. But it is independent of the transport. Seaside is a valid transport. If things are done right one can have many different kinds of UI. But one is making the choice that the transport is Smalltalk messages over some non-Smalltalk stream. No one is contemplating using SOAP or CORBA or... (I hope).

...

...

We very much wanted to produce a set of tools in a tools UI image that could be targetted either at itself or at a remote target, and wanted to be able to attach the tools image to a remote target dynamically (which we called "dynamic capitation", for dynamically gaining a head) so that one could have full debuggability on a remote image deployed in the field.

:) I would really like to know what are the issue at the level of the Display.... VM code. What are the existing hypotheses that have to be broken. Like global access to sensor or

Yes, lots of things break potentally. Transcript, self confirm: etc. Part of the challenge is in coming up with "pluggable" replacements. So that for example Transcript in the target image is something that sends its output to stdout until one attaches a tools UI, at which it becomes something that sends its output to the Transcript in the UI image. This takes some thought :)

...

Alas we never managed to get the priority of the project high enough to actually commit major resources to it. There were always more pressing more mundane things to do. Frustration at not doing things like this was in part behind my leaving Cincom. Xu got quite far with a prototype, demonstrating things like a distributed inspector that allowed one to walk though an object graph spanning more than one image. The inspector would change colour to indicate which machine the current object was one.

...

Personally I think that Smalltalk is the ideal vehicle for this kind of

approach. We have lightweight remote messaging which is relatively easy to implement. We have UI architectures which are relatively well-decomposed and amenable to distribution. We have a reified exception system which is not pushed down into hidden execution machinery and which has resume semantics. We now have a much faster much more reliable internet. We could relatively easily add things like delegation which can enable interesting interleaving of tools and target images (see below). I think we could have something truly revolutionary, the ability to interact with a remote headless image that contains no development tools or UI frameworks and only a relatively small communications module (remote messaging interface). This would enable us to get to images of a few 10s or 100s of kilobytes. I fervently hope you work on this. Lots of people want this. I was reading Colin's Thin Air pages on wiresong yesterday and what he wants is realised by this.

Yes this would be good to have that. I think that cleaning the left over of years of experimentations (which we started with 3.6 Kernel Cleaning project and continue in 3.9 and now is taking a large part in current pharo) is the first phase so that we can make sure kind of system happening.

...

If you do work on this remember your goals. The most important thing is incremental deliverables and useful progress. Don't focus on whizzy optimisations or cool demos (such as the inspector changing colour). Dn't focus on arbitrarily complex topologies of remote images. Instead focus on building a robust, well-engineered remote tools framework that just allows one to do what one can do today in a browser and debugger, but remotely between two images.

We were thinking just about that with marcus and noury for mariano PhD. this could be used to debug remote image running on robots.

...

I would start with browser, debugger and stripper, where stripper would be something that allowed you to eliminate code from the target image using the UI image to discover what was there and what was needed or not, i.e. remote stripping, not the much trickier auto-stripping.

On delegation I think one really cool thing is to wrap remote objects with local development-only code. In a deployed image I would expect there to be no inspectorClass methods or debugPrintOn: messages etc. The target is lean and mean. Anything not needed for deployment except that which supports adequate remote messaging is absent. So one wants to be able to add methods that add debugging features to remote objects, and delegation is the way. If the local handle on a remote object is wrapped by a delegate, the delegate can implement the debugging methods. Catching doesNotUnderstand: in the target mage can allow one to intercept computations in the target and resume them in the development image. So when connecting a tools image to a target one would layer tools and UI code upon objects in the target but that code would reside only in the tools image.

Naming conventions in packaging, like splitting a Foo package into Foo-Deployment & Foo-Development could help in making this kind of split easy to navigate and conceptualise. So after producing the tools/target split I would next focus on packaging tools, including analysis of call graphs etc to help automating the deployment/development split.

You forget deliver often :)

Right!

best Eliot

...

_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project