On 1/23/16 8:59 AM, Stephan Eggermont
wrote:
We
want to use Metacello to manage smalltalk dependencies in a way
that results in working software using a minimum amount of manual
work. At the moment there are several ways in which Metacello is
used, some of which result in more work and instability than
others.
An inherent conflict in managing these dependencies is the level
of repeatability that is wanted and achievable, vs. the amount of
manual work needed. On the one hand we would like perfect
repeatability. That needs an exact description of the order in
which specific versions of packages were loaded in a specific
image. In Metacello, we can achieve this by making sure all
package versions are fixed and the dependencies are fixed to
numeric versions.
This is one of the reasons that I have pushed for the use of git ...
with git ALL of the packages are committed together so a single
commit SHA is sufficient to ensure "that all package versions are
fixed" without regard to numeric versions - the packages that work
together are committed together.
The Metacello registry records the SHA of the commit that was loaded
into an image so there is absolutely no ambiguity.
Dependencies are recorded in a BaselineOf that is also committed
with the packages, so the dependencies are "guaranteed" to be in
sync with the package versions ... a single SHA is all that is
needed to exactly duplicate the formula used to load a project into
an image ...
The problem with this is that that not necessarily results in
working software. There are several reasons for this:
- Metacello only manages the smalltalk level dependencies. It does
not take into account changes in the vm necessitating image-level
changes or operating system and library changes.
There is no reason that project attirbutes should be restricted to
only platform and platform versions .... the attribute list is
generated by the running image, so a list of attributes that
included additional information like vm type and version could be
included and then it would only be necessary to use these new
attributes in the baseline ...
- The
image itself is not fully managed with Metacello (yet). Bug fixes
made after the creation of the Metacello version might be needed
for the software to work.
Yes, this is a critical point ...
- Bugs
might have been found and fixed in dependencies.
In the situation of a top-level configuration, i.e. one that no
other configuration depends on, this might be acceptable.
Metacello dependencies are defined one-directional. A
configuration knows on which configuration it depends, not which
configurations depend on it. Configurations that are published
should therefore always be regarded as possibly being used by
others, and not only top-level.
If we consider the situation where others (B) are depending on
configurations (A) that are written like this, then the situation
gets worse:
- B might need an update because it is affected by an issue that
does not affect A.
- B might need a different load ordering, e.g. need to only
partially load dependencies of A, then something itself and then
the other dependencies of A.
Nevertheless, sometimes we need near-perfect repeatability. For
that, a snapshot of all loaded packages provides a reasonable
alternative to a Metacello configuration version.
... and a list of project/sha/branch like the following:

provides near-perfect repeatability on a project by project basis
--- there are ways using existing Metacello features to ensure that
the exact SHA associated with each of the projects will be used to
the exclusion of all others ...
On the
other hand the least amount of manual updates results from always
depending on #development or #stable versions of dependencies.
This results in the following problems:
- If the API of a dependency changes, the configuration might
break silently.
- Projects might be slow to promote versions from #bleedingEdge to
#development to #stable, resulting in a slow uptake of
improvements.
For configurations of simple packages that have an API that is
very stable or only expands, depending on #stable can be
appropriate.
The historical problem with generic symbolic versions like
#development and #stable has�� been that the semantics of the #stable
version changes over time in the beginning it meant the version that
was "stable on Pharo2.0", but when the same symbolic version to
imply "stable on Pharo3.0", all was lost:)...
From
a modularity point of view, a configuration should be specific on
the packages in its own span of control, and as generous as
possible in what it accepts from its dependencies. In that way a
high cohesion and low coupling is achieved. We can achieve that by
defining symbolic versions that reflect observable change in
behaviour. If we make changes that should not affect users of the
configuration, we create a new numeric version and update the
symbolic version in place to use this new version. By default we
only depend on these symbolic versions.
You are basically describing the principles behind semantic
versioning[1] and�� I've always been a fan of the symbolic version
scheme that the Seaside project has been using (#release3,
#release3.1, etc.), because it makes it possible to express a
project dependency at the appropriate semantic version level...
For git repositories, one can create a tag for a particular SHA
("specific on the packages in its own span of control") and then use
"tag pattern matching"[2], to provide a reasonable approximation of
symbolic versions with a "git" repo.
-----
4 years ago, I was very aware of the fragility of the��
ConfigurationOf ecosystem and at the time, was very glad to see that
responsibility for the implementation for versions, branches and
tags could be off-loaded onto a disk-based SCM - leaving Metacello
to continue to specify package load order and external project
dependencies...
In the subsequent years, I have been adding features to Metacello
that I found were needed to support a "git-based" workflow
(primarily the lock command[3])....
I have also been working on building a Smalltalk development
environment that embraces "git" from the ground up and I consider
that I've learned quite a few important lessons in this area ... I
am not going to say that tODE[4] should be ported to Pharo, but I
will say that tODE _is_ a proof of concept that a git-based
development environment for Smalltalk can be built that preserves
image-based Smalltalk development while leveraging "git" as the
SCM... I should note that tODE bridges the ConfigurationOf and
BaselineOf worlds with support for both ....
I have been "patiently waiting" for folks to come to accept/tolerate
"git" and it seems that that time is getting close enough to begin
trying to share the lessons that I've learned�� --- but I can't "push
on a rope" ---
Tool support is critical at this juncture and I am willing to share
my experience (and code if desired) with the folks who will be
building these next generation tools ---
As I mentioned in another post, the focus for the tools should shift
from being package-centric to being project-centric ... a "git"
repository contains all of the artifacts associated with a project,
and in the "tODE model" there is a one to one mapping between a
project and a git repository.
Of course there is more, but I will save that for sharing with folks
who are curious and interested in learning from my mistakes and my
successes:)
Dale
[1] http://semver.org
[2]
https://github.com/dalehenrich/metacello-work/issues/277#issuecomment-58970696
[3]
https://github.com/dalehenrich/metacello-work/blob/master/docs/LockCommandReference.md#lock-command-reference
[4]
https://github.com/dalehenrich/tode#tode-the-object-centric-development-environment-