[Pharo-project] Future "Modules system"
Hello. Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk. For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class. What it means in context of pharo? In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this. So my global question is how it can be solved in pharo? Do you have some plan for this? I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk? Little conclusion about such module system (if we can named it such way?) and what it is required from pharo: - ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system. Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly. So what you think about it? Best regards, Denis
Have a look at ClassBoxes: http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf I think they come pretty close to what you have in mind. On 2013-02-03, at 12:30, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hello.
Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk.
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
On 3 February 2013 11:36, Camillo Bruni <camillobruni@gmail.com> wrote:
Have a look at ClassBoxes:
http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf
I think they come pretty close to what you have in mind.
On 2013-02-03, at 12:30, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hello.
Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk.
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
No they won't, unless you have separate class loaders. What will happen is you will get one version of a class or another. frank
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
2013/2/3 Frank Shearar <frank.shearar@gmail.com>
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
No they won't, unless you have separate class loaders. What will happen is you will get one version of a class or another.
Of course to get my java example working you should load classes from libA and libB by separated class loaders. In pharo there is no way to make it work. And this is what I want to discuss here.
frank
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
On 3 February 2013 15:42, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2013/2/3 Frank Shearar <frank.shearar@gmail.com>
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
No they won't, unless you have separate class loaders. What will happen is you will get one version of a class or another.
Of course to get my java example working you should load classes from libA and libB by separated class loaders. In pharo there is no way to make it work. And this is what I want to discuss here.
But it's complicated and fragile to have separate classloaders, and a nightmare to debug. The Java people actively avoid this, for instance in JBoss installations. Take a look at Colin Putney's Environments work in Squeak 4.5. It provides a, well, environment in which you can load classes such that you can avoid the name clash problem. _Right now_ that's all it does, but it'll likely do more in the future. Gilad Bracha et al's work in Newspeak is the real way to go, I reckon. (Environments could be a first step in that direction.) Newspeak uses fully parameterised modules, which are first class entities. In fact they really are (stateless) classes. However, Newspeak supports inner classes.
frank
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/). Miguel Le 03/02/2013 18:03, Frank Shearar a écrit :
On 3 February 2013 15:42, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2013/2/3 Frank Shearar <frank.shearar@gmail.com>
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class. No they won't, unless you have separate class loaders. What will happen is you will get one version of a class or another.
Of course to get my java example working you should load classes from libA and libB by separated class loaders. In pharo there is no way to make it work. And this is what I want to discuss here. But it's complicated and fragile to have separate classloaders, and a nightmare to debug. The Java people actively avoid this, for instance in JBoss installations.
Take a look at Colin Putney's Environments work in Squeak 4.5. It provides a, well, environment in which you can load classes such that you can avoid the name clash problem. _Right now_ that's all it does, but it'll likely do more in the future.
Gilad Bracha et al's work in Newspeak is the real way to go, I reckon. (Environments could be a first step in that direction.) Newspeak uses fully parameterised modules, which are first class entities. In fact they really are (stateless) classes. However, Newspeak supports inner classes.
frank
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
--
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com>
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/ **projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>).
Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
Le 03/02/2013 19:27, Denis Kudriashov a écrit :
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com <mailto:miguel.moquillon@gmail.com>>
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>).
Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
IMHO, it is not important and should be avoided because it is error-prone (what is the actual version used by the objects?). This can even be a nightmare with a lot of dependencies when an error occurs. Don't forget: keep things simple (it is always more easily to build too more complex systems). Having a program depending on a package in two different versions at the same time in an image is in my opinion a design error. And if your program depends on some tiers-party packages that depend on a different version of a same package, then you should decide for the more recent one (or according to other heuristic) and fix, if needed, the dependency in the others packages. This is more easy to do with Pharo than with Java. Even in Java, we take care to avoid these situations by using a dependency exclusion with Maven. Miguel
On 3 February 2013 19:53, Miguel Moquillon <miguel.moquillon@gmail.com> wrote:
Le 03/02/2013 19:27, Denis Kudriashov a écrit :
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com <mailto:miguel.moquillon@gmail.com>>
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>).
Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
IMHO, it is not important and should be avoided because it is error-prone (what is the actual version used by the objects?). This can even be a nightmare with a lot of dependencies when an error occurs.
That means that you have to be _extremely_ careful about what libraries you use, because you might well end up having different parts of your application depending on different versions of some library used by various libraries you use. The Metacello docs call this the "diamond dependency" problem. The thing with selector namespaces is that, at least as far as I'm aware, it's an open research problem. The classboxes work, if I recall correctly, pointed out several shortcomings in their approaches. Unless one moved ownership of methods from classes to packages, and ended up with a system looking a lot like Clojure...? At any rate, it's clearly not a simple thing or we'd have already done it. frank
Don't forget: keep things simple (it is always more easily to build too more complex systems). Having a program depending on a package in two different versions at the same time in an image is in my opinion a design error. And if your program depends on some tiers-party packages that depend on a different version of a same package, then you should decide for the more recent one (or according to other heuristic) and fix, if needed, the dependency in the others packages. This is more easy to do with Pharo than with Java. Even in Java, we take care to avoid these situations by using a dependency exclusion with Maven.
Miguel
On 3 February 2013 20:53, Miguel Moquillon <miguel.moquillon@gmail.com> wrote:
Le 03/02/2013 19:27, Denis Kudriashov a écrit :
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com <mailto:miguel.moquillon@gmail.com>>
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>).
Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
IMHO, it is not important and should be avoided because it is error-prone (what is the actual version used by the objects?). This can even be a nightmare with a lot of dependencies when an error occurs. Don't forget: keep things simple (it is always more easily to build too more complex systems). Having a program depending on a package in two different versions at the same time in an image is in my opinion a design error. And if your program depends on some tiers-party packages that depend on a different version of a same package, then you should decide for the more recent one (or according to other heuristic) and fix, if needed, the dependency in the others packages. This is more easy to do with Pharo than with Java. Even in Java, we take care to avoid these situations by using a dependency exclusion with Maven.
+1. From a first sight, it looks like we are missing features to support such cases (like having two different versions of same class because of dependencies), but if we look at results of having this, it is apparent that it will serve as a source of confusion, and while in perfect cases this will work fine, i would hate knowing that my project, in order to function properly, requires two different versions of same class (or package) and avoid this at all costs. I think this problem stems from having a proprietary code, which you cannot control or change, but since in pharo we don't have such 'frozen & closed' code, all the problems with dependencies can be solved in more traditional manner: by fixing them :)
Miguel
-- Best regards, Igor Stasenko.
On 3 February 2013 20:42, Igor Stasenko <siguctua@gmail.com> wrote:
On 3 February 2013 20:53, Miguel Moquillon <miguel.moquillon@gmail.com> wrote:
Le 03/02/2013 19:27, Denis Kudriashov a écrit :
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com <mailto:miguel.moquillon@gmail.com>>
And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>).
Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
IMHO, it is not important and should be avoided because it is error-prone (what is the actual version used by the objects?). This can even be a nightmare with a lot of dependencies when an error occurs. Don't forget: keep things simple (it is always more easily to build too more complex systems). Having a program depending on a package in two different versions at the same time in an image is in my opinion a design error. And if your program depends on some tiers-party packages that depend on a different version of a same package, then you should decide for the more recent one (or according to other heuristic) and fix, if needed, the dependency in the others packages. This is more easy to do with Pharo than with Java. Even in Java, we take care to avoid these situations by using a dependency exclusion with Maven.
+1. From a first sight, it looks like we are missing features to support such cases (like having two different versions of same class because of dependencies), but if we look at results of having this, it is apparent that it will serve as a source of confusion, and while in perfect cases this will work fine, i would hate knowing that my project, in order to function properly, requires two different versions of same class (or package) and avoid this at all costs.
It's a nice idea, to avoid this kind of situation at all costs. The problem is, you can't.
I think this problem stems from having a proprietary code, which you cannot control or change, but since in pharo we don't have such 'frozen & closed' code, all the problems with dependencies can be solved in more traditional manner: by fixing them :)
Well, no, unless you write all your libraries yourself. Noone, especially not people trying to deliver software in a commercial context, can afford to write everything themselves. You'll have to use other people's libraries. Now sure, you can talk to the library authors, and encourage them to all use the same version of Foo or whatever, but it's not actually something you can avoid "at all costs". So we might as well think about how to actually deal with the problem. frank
Miguel
-- Best regards, Igor Stasenko.
dare I mention Craig Latta's Spoon project which explicitly is meant to solve this issue? L On 2/3/13 1:42 PM, Igor Stasenko wrote:
On 3 February 2013 20:53, Miguel Moquillon <miguel.moquillon@gmail.com> wrote:
Le 03/02/2013 19:27, Denis Kudriashov a écrit :
2013/2/3 Miguel Moquillon <miguel.moquillon@gmail.com <mailto:miguel.moquillon@gmail.com>> And the Java people are preparing to throw away the classloaders mechanism with Jigsaw that will be in Java 9. (The classloading is a brittle and error-prone mechanism.) The aim of jigswa is to design and implement a new module system for the Java SE platform and to apply that sysÂtem to the platÂform itÂself and to the JDK (more information in the project homepage http://openjdk.java.net/projects/jigsaw/ <http://openjdk.java.net/projects/jigsaw/>). Miguel
I know about this. But let's talk about pharo. Now pharo can't do this. You can't have two versions of same package at same time in image. So how we can implement it? And do this is really important?
IMHO, it is not important and should be avoided because it is error-prone (what is the actual version used by the objects?). This can even be a nightmare with a lot of dependencies when an error occurs. Don't forget: keep things simple (it is always more easily to build too more complex systems). Having a program depending on a package in two different versions at the same time in an image is in my opinion a design error. And if your program depends on some tiers-party packages that depend on a different version of a same package, then you should decide for the more recent one (or according to other heuristic) and fix, if needed, the dependency in the others packages. This is more easy to do with Pharo than with Java. Even in Java, we take care to avoid these situations by using a dependency exclusion with Maven.
+1. From a first sight, it looks like we are missing features to support such cases (like having two different versions of same class because of dependencies), but if we look at results of having this, it is apparent that it will serve as a source of confusion, and while in perfect cases this will work fine, i would hate knowing that my project, in order to function properly, requires two different versions of same class (or package) and avoid this at all costs.
I think this problem stems from having a proprietary code, which you cannot control or change, but since in pharo we don't have such 'frozen & closed' code, all the problems with dependencies can be solved in more traditional manner: by fixing them :)
Miguel
-- Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner). https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian Kernighan
On 4 February 2013 01:23, Lawson English <lenglish5@cox.net> wrote:
dare I mention Craig Latta's Spoon project which explicitly is meant to solve this issue?
in what way? being able to exchange data between two images ~~ being able to deal with two different versions of same class properly. Think about proper support for development tools. For example, look at traits: they are integrated since 2007? (i think) , but some tools still have problems with them.
L
-- Best regards, Igor Stasenko.
Well, the tools available are still relatively primitive, but bugs aside, the current version of Spoon has the capability of creating and editing methods in a "remote" spoon image using a class browser living in the main image which looks and behaves exactly like a normal Pharo/Squeak image except that it has support for Spoon and all the saving is done on the remote image. In principle, that remote image could live in your local computer or anywhere on the internet. The kind of use-cases that Craig and I have discussed include things like taking a Pharo-compatible Spoon image and "imprinting" (importing live versions of) the classes and objects from a Squeak-compatible Spoon image onto the Pharo image so that two programmers running Squeak and Pharo on their machines can collaborate on editing a specific running application in their own favorite environment. The tools and capabilities for this already exist, although there are no doubt still bugs to be squashed. A more sophisticated use-case would be to create a remote Spoon image running somewhere in the cloud that two or more programmers could manipulate at the same time. This is also doable using the current Spoon image and tools, but the traditional database issues concerning simultaneous access to the same resource apply and I don't think solving them in the Spoon Class Browser is a top priority for Craig right now. L On 2/4/13 12:41 AM, Igor Stasenko wrote:
On 4 February 2013 01:23, Lawson English <lenglish5@cox.net> wrote:
dare I mention Craig Latta's Spoon project which explicitly is meant to solve this issue?
in what way? being able to exchange data between two images ~~ being able to deal with two different versions of same class properly. Think about proper support for development tools. For example, look at traits: they are integrated since 2007? (i think) , but some tools still have problems with them.
L
-- Squeak from the very start (introduction to Squeak and Pharo Smalltalk for the (almost) complete and compleate beginner). https://www.youtube.com/playlist?list=PL6601A198DF14788D&feature=view_all "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian Kernighan
Hi-- Yes, Spoon[1] supports multiple active versions of the same class. Background: Spoon has no system dictionary, nor does it use "chunk-format" files such as sources or changes files, or "fileouts". It doesn't use files at all, apart from object memory snapshots and virtual machine executables, which are embedded into the Spoon "app" as far as the host platform is concerned. Instead of using source code in files, Spoon uses a distributed live-objects history system called Naiad (short for "Name And Identity Are Distinct"). Each normal object memory is paired with a special separate memory, connected by remote messaging, for recording history information. This special memory is called a "history memory", and the memory it describes is called a "subject memory". The history memory stores "edition" objects which describe various development events, including class and method installation. These editions are managed by an "edit history" object, which is exposed to its subject memory as a seamless proxy. Class installation happens whenever you define a new class, or change the definition of a class. The edition that describes a class installation includes all the information needed to replay that event, to undo it, and to provide context for people studying it later (for example, the author and the time of installation). Among that information is the name given to the class at the time. Each edition has a reference to the previous and next editions. The edit history stores the most recent edition for the class in a dictionary; the key is a UUID[2]. That UUID is also stored in a slot of the active class object being described, in the subject memory. When the compiler needs to find the class object for a name mentioned in source code, it asks the edit history for the IDs of all the active classes with that name, and finds all the active classes for those IDs, by traversing the class hierarchy. If there is more than one class for the same name, the user interface can help the developer make a choice, by showing some of the additional contextual information from the edit history. (The history memory is effectively an application-specific distributed object database. Multiple subject memories can use the same history memory, and history memories can federate. From each subject memory object's point of view, the edit history is just a normal local object.) With this organization, the name of each class can be anything. It doesn't matter that you already have a class called Array, you could install as many more as you like. They would each have a different ID, and you could tell them apart from their historical information. Since the name of every class is unconstrained, every class effectively has its own namespace, without having to reify the concept of "namespace" in the developer's mental model of the system, or in the implementation of the system. And now for your use cases... :) Since class names are unconstrained, supporting multiple classes with the same name, but with unrelated development histories, is trivial. For example, you and I both happened to choose the same name for classes which we have developed independently, have different message interfaces, and different slot layouts. Our classes coexist, the compiler can compile new methods for them (with occasional disambiguation from a person informed by historical information), and instantiation works as always. At the other extreme, you might want to install a previous version of an active class that evolved over many versions, without modifying the active class object. The system would create a class using information from the older class edition, but give the resulting class object a new ID, and record this with an appropriate class edition using that new ID. Personally, I would make some attempt to avoid both of these situations, through discussion with my fellow authors, but I recognize that this isn't always feasible. For more details about Naiad, please see my blog[3]. thanks again, -C [1] http://netjam.org/spoon [2] http://en.wikipedia.org/wiki/Universally_unique_identifier [3] http://tinyurl.com/8f5zq85 (thiscontext.wordpress.com) -- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
Oh, and I'm testing Naiad support for all the Smalltalk systems I know: Pharo, Squeak, VisualWorks, VAST, ObjectStudio, GNU Smalltalk, Amber, Redline Smalltalk, and Dolphin Smalltalk. Please let me know of any still in existence that I overlooked. I think there are possibilities for Self and JavaScript as well, but I'm starting with systems that are class-oriented, with single inheritance, and a stack-oriented virtual machine instruction set (or can provide a sufficient illusion). With Naiad support installed, any system is able to synchronize live editions (for methods, classes, authors, tags, etc.), with any other. Compiled methods are transferred directly; recompilation from source code is only done when the instruction sets of the VMs involved are irreconcilable for that method (or on demand). For most of the methods I've considered so far, no participating system actually needs a compiler installed. -C -- Craig Latta www.netjam.org/resume +31 6 2757 7177 (SMS ok) + 1 415 287 3547 (no SMS)
2013/2/3 Frank Shearar <frank.shearar@gmail.com>
On 3 February 2013 15:42, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2013/2/3 Frank Shearar <frank.shearar@gmail.com>
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as
libS-b.jar.
Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
No they won't, unless you have separate class loaders. What will happen is you will get one version of a class or another.
Of course to get my java example working you should load classes from libA and libB by separated class loaders. In pharo there is no way to make it work. And this is what I want to discuss here.
But it's complicated and fragile to have separate classloaders, and a nightmare to debug. The Java people actively avoid this, for instance in JBoss installations.
If your program interracts with multiple external systems by their provided libraries you can't avoid such dependencies problem. So in some level of your application you should work with class loaders. Maybe your "environment framework" hides it.
Take a look at Colin Putney's Environments work in Squeak 4.5. It provides a, well, environment in which you can load classes such that you can avoid the name clash problem. _Right now_ that's all it does, but it'll likely do more in the future.
I know about Colin Putney's Environments. And I like it very much. But initially his work solves competelly ortogonal problem. Interesting how it can be reused to load separated versions of same package. But as I know Environments not supports selector namespaces which I think required to totally support what I proposed: loading any version of project with no influence on rest system behaviour.
Gilad Bracha et al's work in Newspeak is the real way to go, I reckon. (Environments could be a first step in that direction.) Newspeak uses fully parameterised modules, which are first class entities. In fact they really are (stateless) classes. However, Newspeak supports inner classes.
As Environment's what they call modules is completelly ortogonal to my requirement.
frank
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
2013/2/3 Camillo Bruni <camillobruni@gmail.com>
Have a look at ClassBoxes:
http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf
I think they come pretty close to what you have in mind.
Thank's for link. I think classboxies can be good basis to implement my requirement. Separated classbox per package version can be solution. Interesting how performance can be improved here? 25% is big penalty
I arrived to almost the same problem some weeks ago, when I was thinking on how to have multiple versions of Fuel at the same time, so one can, for example, easily migrate her files to a newer version. Fuel has lots of extension methods, so we need also multiple versions of the same extension method. Then, I arrived to ClassBoxes paper, from which I would like to extract and reproduce the experiment in a Pharo2.0 with Opal compiler. I guess Opal can simplify the implementation. There is also a paper about a prototype named CoExist, based on squeak. I didn't try it but apparently they only have a little performance degradation, because they used a modified vm. Martin On Sun, Feb 3, 2013 at 12:36 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
Have a look at ClassBoxes:
http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf
I think they come pretty close to what you have in mind.
On 2013-02-03, at 12:30, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hello.
Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk.
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
On Mon, Feb 4, 2013 at 4:15 PM, Martin Dias <tinchodias@gmail.com> wrote:
Then, I arrived to ClassBoxes paper, from which I would like to
Hmmm, I misread. Actually the paper I was talking about is the ChangeBoxes one. So I wanted to play with that but without tracking the changes as they do...
Hi 2013/2/4 Martin Dias <tinchodias@gmail.com>
I arrived to almost the same problem some weeks ago, when I was thinking on how to have multiple versions of Fuel at the same time, so one can, for example, easily migrate her files to a newer version.
Fuel has lots of extension methods, so we need also multiple versions of the same extension method.
Then, I arrived to ClassBoxes paper, from which I would like to extract and reproduce the experiment in a Pharo2.0 with Opal compiler. I guess Opal can simplify the implementation. There is also a paper about a prototype named CoExist, based on squeak. I didn't try it but apparently they only have a little performance degradation, because they used a modified vm.
I think It can be implemented with method wrappers and without impact on performance of methods which has not everrided versions. When you load code from package with method extensions you can put this methods in target class with method wrappers which know about classboxies logic. But rest system will continue working as usuall. Do you have links on CoExist? Best regards, Denis
Martin
On Sun, Feb 3, 2013 at 12:36 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
Have a look at ClassBoxes:
http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf
I think they come pretty close to what you have in mind.
On 2013-02-03, at 12:30, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hello.
Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk.
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
On Mon, Feb 4, 2013 at 5:25 PM, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi
2013/2/4 Martin Dias <tinchodias@gmail.com>
I arrived to almost the same problem some weeks ago, when I was thinking on how to have multiple versions of Fuel at the same time, so one can, for example, easily migrate her files to a newer version.
Fuel has lots of extension methods, so we need also multiple versions of the same extension method.
Then, I arrived to ClassBoxes paper, from which I would like to extract and reproduce the experiment in a Pharo2.0 with Opal compiler. I guess Opal can simplify the implementation. There is also a paper about a prototype named CoExist, based on squeak. I didn't try it but apparently they only have a little performance degradation, because they used a modified vm.
I think It can be implemented with method wrappers and without impact on performance of methods which has not everrided versions. When you load code from package with method extensions you can put this methods in target class with method wrappers which know about classboxies logic. But rest system will continue working as usuall.
Do you have links on CoExist?
yes: http://www.bastiansteinert.org/coexist.html anyway it's a bit off-topic because their prototype is not about modules... best regards, Martin
Best regards, Denis
Martin
On Sun, Feb 3, 2013 at 12:36 PM, Camillo Bruni <camillobruni@gmail.com> wrote:
Have a look at ClassBoxes:
http://scg.unibe.ch/archive/papers/Berg03aClassboxes.pdf
I think they come pretty close to what you have in mind.
On 2013-02-03, at 12:30, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hello.
Some days ago I think about how java works with dependencies and how it can be implemented in smalltalk.
For java example libA.jar uses SharedClass.methodA and libB.jar uses SharedClass.methodB. But #methodA exists only in version deployed as libS-a.jar. And #methodB exists only in version deployed as libS-b.jar. Dependency from libA.jar on libS-a.jar and libB.jar on libS-b.jar are defined explicitly in their classpathes. So libA.jar knows nothing about libS-b.jar and v.v. And that's why your java programm can works at same time with libA.jar and libB.jar. They will use different versions of same class.
What it means in context of pharo?
In pharo we have configurations which manage dependencies of projects. But you can't use at same time projectA and projectB which depend on different versions of projectS (when they both use same SharedClass from projectS like in java example). If you load projectA last you will get SharedClass with only methodA. If you load projectB last you will get SharedClass with only methodB. So projectA and projectB can't works correctly at same time. My example very simple. It includes just single class differencies. It can be more complicated and java world is good example for this.
So my global question is how it can be solved in pharo? Do you have some plan for this?
I can't remember something similar in pharo vision docs. But maybe what Stephane called "modules system" is exactly about this? I mean what Stephane always opposes in namespaces/environment discussions. And I think that this problem is ortogonal to namespaces/environments support. Do you agree? Do you know about such reseach in smalltalk?
Little conclusion about such module system (if we can named it such way?) and what it is required from pharo:
- ability to get in image different versions of same package. - separated class (global) dictionaries (multiple Smalltalk's) per package version. Maybe it can be named Module. So let's think about Module as unique Package+Version pair which contains all globals specific to this version and all dependent other Modules. - each Module knows it dependencies from other Modules, This information can be retreived from configuration. - each method and class should know from which project version (module) it was loaded. - inside any methods compiler should resolve class names based on Module dependencies information. Compiler should lookup actual class objects from module dependencies hierarchy. So no global Smalltak for compiler. But Smalltalk can be stay in system as default core Module. - If you want to use class which presents in two loaded versions of same package you should resolve conflict manually. Such action should add dependency to Module of your code. - Module can't have dependencies of two Modules of same package. It means that code of your project can use only version of some dependant class. I think it is essential restriction of Modules system.
Main thing which Module system can get to us is ability to safelly load any versions of our projects and use it. It is not matter how old this code. System just loads old version of pharo core for example. Existed pharo core will continue works correctly.
So what you think about it?
Best regards, Denis
participants (8)
-
Camillo Bruni -
Craig Latta -
Denis Kudriashov -
Frank Shearar -
Igor Stasenko -
Lawson English -
Martin Dias -
Miguel Moquillon