Yes, you should be able to load an mcz in that image by doing: (MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load. Guille On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo' ]. spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk- smalltalk:v1.10/pharo-repository' ]. spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. ].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

...

Tim

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/ job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessfulBuild/ar tifact/Pharo-minimal-64.zip one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0 -Step-04-01-ConfigurationOfMinimalPharo/ one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bi t-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in:

Metacello new repository: 'filetree://../src'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

-- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all): Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 Then you will do something like: 2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works>:

Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

...

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo' ]. spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk- smalltalk:v1.10/pharo-repository' ]. spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. ].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

...

Tim

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/ job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessfulBuild/ar tifact/Pharo-minimal-64.zip one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0 -Step-04-01-ConfigurationOfMinimalPharo/ one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bi t-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in:

Metacello new repository: 'filetree://../src'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr/>

*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now… When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future. Tim

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <filetree://./PharoLambda/src>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <filetree://./PharoLambda/src>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core? I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?) Tim

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works> wrote:

I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello. Sometimes you can print the result of #record: Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core'). -> "linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06" ------ Anyway.... I guess I would try to load these packages (in this order): FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel And when you are down loading, then execute: (Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src. ? Just so I know for the future.

yes

-- Pavel

...

Tim

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

...

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek < pavel.krivanek@gmail.com> wrote:

...

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo' ]. spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk- smalltalk:v1.10/pharo-repository' ]. spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. ].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

...

Tim

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2- Minimal/lastSuccessfulBuild/artifact/Pharo-minimal-64.zip one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0 -Step-04-01-ConfigurationOfMinimalPharo/ one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bi t-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in:

Metacello new repository: 'filetree://../src'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr/>

*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works> wrote:

Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - *MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames*

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "*MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames*", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>> fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: # fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf: keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works> wrote:

...

I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

...

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src. ? Just so I know for the future.

yes

-- Pavel

...

Tim

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

...

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@ gmail.com> wrote:

...

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo' ]. spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk- smalltalk:v1.10/pharo-repository' ]. spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. ].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

...

Tim

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2- Minimal/lastSuccessfulBuild/artifact/Pharo-minimal-64.zip one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0 -Step-04-01-ConfigurationOfMinimalPharo/ one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bi t-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in:

Metacello new repository: 'filetree://../src'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr/>

*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works> wrote:

Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass: fuelAccept: aGeneralMapper ^self explicitRequirement. Rather than the correct one: fuelAccept: aGeneralMapper ^aGeneralMapper visitClass: self *Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.* If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ? Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works> wrote:

...

Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - *MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames*

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

...

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

...

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "*MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames*", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBu cket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works> wrote:

...

I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

...

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src. ? Just so I know for the future.

yes

-- Pavel

...

Tim

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: ' filetree://./PharoLambda/src'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

...

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail .com> wrote:

...

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo' ]. spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk- smalltalk:v1.10/pharo-repository' ]. spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. ].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

...

Tim

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2- Minimal/lastSuccessfulBuild/artifact/Pharo-minimal-64.zip one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0 -Step-04-01-ConfigurationOfMinimalPharo/ one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bi t-Conversion/lastSuccessfulBuild/artifact/latest-minimal-64.zip one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>*doesNotUnderstand: #addTo:* 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in:

Metacello new repository: 'filetree://../src'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: ' filetree://./pharo-core/src'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr/>

*Web:* *http://guillep.github.io* <http://guillep.github.io/> *Phone: *+33 06 52 70 66 13 <+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am). I need to double check this - as I must be making some trivial mistake. Tim

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel? However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian. Tim

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <filetree:///builds/macta/PharoLambda/bootstrap> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <filetree:///builds/macta/PharoLambda/bootstrap> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

HURRAH - I got it to work, and Fuel out an error to S3 and read it back into a fat 6.1 image with a debug stack (which is so cool!). My footprint size is still a bit larger than I would like (31mb - image of 22.7mb - of which 1mb if fuel and 7mb is S3 and XMLParser support) - so I guess I need to find ways to trim down some of the addons (and any additional image size reductions would be helpful - if there are any to be had). Is there any easy tools/scripts to explore a headless image and figure out what takes the most size (maybe at a package level?). But definitely this has got me into a good place for experimentation. For the record I had to do the following to get things working: Define a new baseline: baseline: spec <baseline> spec for: #common do: [ spec package: 'Metacello-GitBasedRepository'; package: 'Metacello-GitHub'; package: 'FuelPlatform-Core'; package: 'FuelPlatform-Pharo-Core'; package: 'FuelPlatform-Pharo-06'; package: 'Fuel'. ]. Run the following script: Metacello new baseline: 'MetacelloGitBasedRepository'; repository: 'filetree://../bootstrap'; load. “Replace missing traits" Behavior compiledMethodAt: #fuelIgnoredInstanceVariableNames ifPresent: [ :m | logger cr; nextPutAll: 'Trait method is present' ] ifAbsent: [ logger cr; nextPutAll: 'Recompile missing traits ...'. Behavior compile: 'fuelIgnoredInstanceVariableNames "Indicates which variables have to be ignored during serialization." ^#()'; compile: 'fuelNew "Answer an instance of mine in which serialized references will be injected." ^ self basicNew'; compile: 'fuelNew: sizeRequested "Answer an instance of mine in which serialized references will be injected." ^ self basicNew: sizeRequested'. "Ignore this one" "Class compile: 'fuelAccept: aGeneralMapper ^self explicitRequirement.'." ClassDescription compile: 'instanceVariableNamesDo: anUnaryBlock "This is part of the interface between the compiler and a class''s instance or field names. The class should enumerate anUnaryBlock with the instance variable name strings. The order is important. Names evaluated later will override the same names occurring earlier." | superInstSize | (superInstSize := self superclass notNil ifTrue: [self superclass instSize] ifFalse: [0]) > 0 ifTrue: [self superclass instanceVariableNamesDo: anUnaryBlock]. 1 to: self instSize - superInstSize do: [:i| anUnaryBlock value: (self instVarNames at: i)]'. ]. “Add a tacky Pharo7Platform" Smalltalk at: #FLPharo7Platform ifAbsent: [ logger cr; nextPutAll: '>Creating FLPharo7Platform'. (Smalltalk at: #FLPharo6Platform) subclass: #FLPharo7Platform. (Smalltalk at: #FLPharo6Platform) class compile: 'isResponsibleForCurrentPlatform ^true'. ]. logger cr; nextPutAll: 'Evaluating post load initialization...'. (Smalltalk at: #FLPlatform) current addHacks.

On 4 Aug 2017, at 12:48, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Fri, Aug 4, 2017 at 8:42 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Max, should we do something to the Fuel Platform for Pharo 7?

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

Hi - I was actually reading those slides the other day, but hadn’t fully appreciated the consequences… that is very cool - and definitely something for me to put on the stack of things to try out I was going to say I could put the Fuel and S3 access into such a library - but then I would need them to load it from S3 - so catch 22 for that, but I’ll bet I come up with another usage. Thanks to everyone for their help with this. Is there anything I need follow up on from this? I saw you asked Max about the Platform7 piece. I also don’t understand why I have problems with those missing fuel trait methods (so I’ll see what Pavel says about that). Tim

On 4 Aug 2017, at 14:24, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Fri, Aug 4, 2017 at 9:51 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: HURRAH - I got it to work, and Fuel out an error to S3 and read it back into a fat 6.1 image with a debug stack (which is so cool!).

COOOL!!! I want to see your blogpost once you make it!!! Congratulations, this was a very cool thread to follow :)

My footprint size is still a bit larger than I would like (31mb - image of 22.7mb - of which 1mb if fuel and 7mb is S3 and XMLParser support) - so I guess I need to find ways to trim down some of the addons (and any additional image size reductions would be helpful - if there are any to be had).

Is there any easy tools/scripts to explore a headless image and figure out what takes the most size (maybe at a package level?). But definitely this has got me into a good place for experimentation.

I know this is a bit offtopic... but maybe to give you some perspective... For my PhD, one of the things I did (lets say the closest to the PhD topic) was called Marea [1] (I can also send you the slides and the PhD defense video on youtube if you want). In Marea what I did was to modify the VM to implement a very basic (with lots of limitations) object usage tracking...I simply flagged when an object was used and cleared the flag every in a while. Then, at image side, I would find graphs of unused objects, and replaced the boundary objects with proxies using Ghost proxies (note that an anused object is not the same as unreferenced object,...GC does nothing here). The graphs where then serialized with Fuel. Finally, if those graphs happened to be needed, then the proxy would intercept the message, materialize from Fuel, and plug back the original graph.

Anyway... all of what was to said that I some point I did an experiment. I was able to already proxify and serialize classes, methods, etc. So I took the whole image and I swapped out all classes and their instances (each class with its instances in a different graph) but only a really small core. This image was a Seaside image running DBX Pier website. So I swapped out everything, and then I lazily started to navigate DBX website, causing the needed graphs to be swapped in. I was able to naviagat all webapp and use it perfectly. I even saved the image. And all this email is to say that such an image was 3MB.

Cheers,

[1] https://www.slideshare.net/MarianoMartinezPeck/thesis-presentation-14987645 <https://www.slideshare.net/MarianoMartinezPeck/thesis-presentation-14987645>

For the record I had to do the following to get things working:

Define a new baseline: baseline: spec <baseline>

spec for: #common do: [ spec package: 'Metacello-GitBasedRepository'; package: 'Metacello-GitHub'; package: 'FuelPlatform-Core'; package: 'FuelPlatform-Pharo-Core'; package: 'FuelPlatform-Pharo-06'; package: 'Fuel'. ].

Run the following script: Metacello new baseline: 'MetacelloGitBasedRepository'; repository: 'filetree://../bootstrap <>'; load.

“Replace missing traits" Behavior compiledMethodAt: #fuelIgnoredInstanceVariableNames ifPresent: [ :m | logger cr; nextPutAll: 'Trait method is present' ] ifAbsent: [ logger cr; nextPutAll: 'Recompile missing traits ...'.

Behavior compile: 'fuelIgnoredInstanceVariableNames "Indicates which variables have to be ignored during serialization."

^#()'; compile: 'fuelNew "Answer an instance of mine in which serialized references will be injected."

^ self basicNew'; compile: 'fuelNew: sizeRequested "Answer an instance of mine in which serialized references will be injected."

^ self basicNew: sizeRequested'.

"Ignore this one" "Class compile: 'fuelAccept: aGeneralMapper ^self explicitRequirement.'."

ClassDescription compile: 'instanceVariableNamesDo: anUnaryBlock "This is part of the interface between the compiler and a class''s instance or field names. The class should enumerate anUnaryBlock with the instance variable name strings. The order is important. Names evaluated later will override the same names occurring earlier."

| superInstSize | (superInstSize := self superclass notNil ifTrue: [self superclass instSize] ifFalse: [0]) > 0 ifTrue: [self superclass instanceVariableNamesDo: anUnaryBlock]. 1 to: self instSize - superInstSize do: [:i| anUnaryBlock value: (self instVarNames at: i)]'. ].

“Add a tacky Pharo7Platform" Smalltalk at: #FLPharo7Platform ifAbsent: [ logger cr; nextPutAll: '>Creating FLPharo7Platform'. (Smalltalk at: #FLPharo6Platform) subclass: #FLPharo7Platform. (Smalltalk at: #FLPharo6Platform) class compile: 'isResponsibleForCurrentPlatform ^true'. ].

logger cr; nextPutAll: 'Evaluating post load initialization...'. (Smalltalk at: #FLPlatform) current addHacks.

...

On 4 Aug 2017, at 12:48, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Fri, Aug 4, 2017 at 8:42 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Max, should we do something to the Fuel Platform for Pharo 7?

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple). So I should have the flattened versions then? (I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part) Tim

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Ah - I checked out master. So should I have checked out dev? Would this explain it? I've appreciated the help a lot by the way. Tim Sent from my iPhone

On 4 Aug 2017, at 17:28, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

It it were packages from the development branch then you do not need to flatten them.

-- Pavel

2017-08-04 15:52 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple).

So I should have the flattened versions then?

(I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part)

Tim

...

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

...

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

...

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works> wrote:

...

Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

...

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

...

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works> wrote:

...

I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

...

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src. ? Just so I know for the future.

yes

-- Pavel

...

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com>:

...

The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

...

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works>:

...

Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

...

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works>: > Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline. > > The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this: > > baseline: spec > <baseline> > > spec for: #common do: [ > spec configuration: 'ZTimestamp' with: [ > spec > versionString: #stable; > repository: 'http://mc.stfx.eu/Neo' ]. > > spec baseline: 'AWS' with: [ > spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository' ]. > > spec > package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}]. > > ]. > > > The “spec configuration: ….” Specifications cause the error: > > 25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: > 26 MCRepositoryGroup>>addRepository: > > If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages. > > So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

> > Tim > >> On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com> wrote: >> >> >> >> 2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works>: >>> I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... one still shows as being last updated 7 days ago. >>> >>> The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521). >>> >>> The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... one gives me a walkback when trying to run my install script : >>> >>> 25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: >>> 26 MCRepositoryGroup>>addRepository: >>> 27 createRepository >>> | repo | >>> repo := self project createRepository: self. >>> ^ MCRepositoryGroup default repositories >>> detect: [ :each | each = repo ] >>> ifNone: [ >>> MCRepositoryGroup default addRepository: repo. >>> repo ] in MetacelloRepositorySpec>>createRepository >>> >>> I think this is because the image doesn’t support Metacello? As in: >>> Metacello new >>> repository: 'filetree://../src'; >>> baseline: 'Lambda'; >>> load. >>> >>> How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?) >>> >>> I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this. >>> >>> Tim >>> >> >> I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works: >> >> ./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src'; load." >> >> ./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run" >> >> Can you test it too? >> >> -- Pavel >

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr

Web: http://guillep.github.io Phone: +33 06 52 70 66 13

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

-- Mariano http://marianopeck.wordpress.com

I’ve just switched branches and diffed - and yep, those methods were the difference. Sorry about the extra noise on that - I should have read your instructions a bit better - still I’ve learned a lot doing this, and am really chuffed that I’ve managed to get something working. I’m looking at how to get my image size down a bit more - and have another thread on how to properly remove packages. How stable do you think the minimal image will be? Should I try and snapshot it as a baseline - or do you think I can keep pulling safely from Jenkins for a while? Tim

On 4 Aug 2017, at 21:19, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

yes, because master branch contains code of Pharo 6. For Pharo 7 we have flattened the traits in the kernel, but this is probably the only difference that affects Fuel packages

-- Pavel

2017-08-04 20:32 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah - I checked out master. So should I have checked out dev? Would this explain it?

I've appreciated the help a lot by the way.

Tim

Sent from my iPhone

On 4 Aug 2017, at 17:28, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

...

It it were packages from the development branch then you do not need to flatten them.

-- Pavel

2017-08-04 15:52 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple).

So I should have the flattened versions then?

(I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part)

Tim

...

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Thanks Pavel - as I’m downloading: curl -L -o min.zip https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu... <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu...> I noted that the last successful build of that asset was: Last stable build (#29), 2 days 23 hr ago <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastStableBuild/> Should I peg myself to a specific version that is more or less compatible with Pharo 6.1? I can see builds 28 (Aug 4 11am) or 27 (July 31?) - or should I go earlier? Can I rely on those builds sticking around - or should I cache one somewhere? Tim

On 7 Aug 2017, at 12:03, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-07 12:56 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I’ve just switched branches and diffed - and yep, those methods were the difference. Sorry about the extra noise on that - I should have read your instructions a bit better - still I’ve learned a lot doing this, and am really chuffed that I’ve managed to get something working.

I’m looking at how to get my image size down a bit more - and have another thread on how to properly remove packages.

How stable do you think the minimal image will be? Should I try and snapshot it as a baseline - or do you think I can keep pulling safely from Jenkins for a while?

Pharo 7 is not stable in principle. You should not depend on the latest version but of course you can take a particular build that is supposed to be quite solid. Now you should rather take builds before Hermes integration and related refactorings.

-- Pavel

Tim

...

On 4 Aug 2017, at 21:19, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

yes, because master branch contains code of Pharo 6. For Pharo 7 we have flattened the traits in the kernel, but this is probably the only difference that affects Fuel packages

-- Pavel

2017-08-04 20:32 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah - I checked out master. So should I have checked out dev? Would this explain it?

I've appreciated the help a lot by the way.

Tim

Sent from my iPhone

On 4 Aug 2017, at 17:28, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

...

It it were packages from the development branch then you do not need to flatten them.

-- Pavel

2017-08-04 15:52 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple).

So I should have the flattened versions then?

(I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part)

Tim

...

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Hey thanks again - it looks like that file is from Build #25 (I think Jenkins might cache the last 5 builds from the looks of it). Interestingly that image is 2mb smaller than the later ones (which is helpful) - however when I try to use it, I get a walkback because Metacello wasn’t available in that build. (I then tried with build #29 the latest? and #28) - and then I get an error as it seems that the Locale class is not defined in that one? 'Errors in script loaded from minimal.st' MessageNotUnderstood: receiver of "currentPlatform" is nil UndefinedObject(Object)>>doesNotUnderstand: #currentPlatform LanguageEnvironment class>>currentPlatform LanguageEnvironment class>>defaultSystemConverter TextConverter class>>defaultSystemConverter MultiByteFileStream>>converter MultiByteFileStream>>nextPut: MultiByteFileStream(WriteStream)>>cr UndefinedObject>>DoIt So I guess the timing of my question was spot on as it looks like I’ve been running with build #27 for a few days (without realising I had cached it) and that one works. I would be interesting in working out how to get metacello in build #25 as its smaller size is attractive (assuming its not metacello support that adds 3mb to the image size), Do you think its worth it? Can I use the command line package loader to shoe-horn it in? Tim

On 7 Aug 2017, at 13:47, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

Unfortunately we store minimal versions only for the latest build and you should use older build with hash b1625bf32c6b2b6c6f8babb789e377af5132d740.

You can download this image here (I have found a random copy of it): https://drive.google.com/open?id=0BzSsmZhqtUTeclFEbDhtd0JvdEk <https://drive.google.com/open?id=0BzSsmZhqtUTeclFEbDhtd0JvdEk>

Copy it to some public location you can reach.

-- Pavel

2017-08-07 14:25 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Thanks Pavel - as I’m downloading:

curl -L -o min.zip https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu... <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu...>

I noted that the last successful build of that asset was: Last stable build (#29), 2 days 23 hr ago <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastStableBuild/> Should I peg myself to a specific version that is more or less compatible with Pharo 6.1? I can see builds 28 (Aug 4 11am) or 27 (July 31?) - or should I go earlier?

Can I rely on those builds sticking around - or should I cache one somewhere?

Tim

...

On 7 Aug 2017, at 12:03, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-07 12:56 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I’ve just switched branches and diffed - and yep, those methods were the difference. Sorry about the extra noise on that - I should have read your instructions a bit better - still I’ve learned a lot doing this, and am really chuffed that I’ve managed to get something working.

I’m looking at how to get my image size down a bit more - and have another thread on how to properly remove packages.

How stable do you think the minimal image will be? Should I try and snapshot it as a baseline - or do you think I can keep pulling safely from Jenkins for a while?

Pharo 7 is not stable in principle. You should not depend on the latest version but of course you can take a particular build that is supposed to be quite solid. Now you should rather take builds before Hermes integration and related refactorings.

-- Pavel

Tim

...

On 4 Aug 2017, at 21:19, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

yes, because master branch contains code of Pharo 6. For Pharo 7 we have flattened the traits in the kernel, but this is probably the only difference that affects Fuel packages

-- Pavel

2017-08-04 20:32 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah - I checked out master. So should I have checked out dev? Would this explain it?

I've appreciated the help a lot by the way.

Tim

Sent from my iPhone

On 4 Aug 2017, at 17:28, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

...

It it were packages from the development branch then you do not need to flatten them.

-- Pavel

2017-08-04 15:52 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple).

So I should have the flattened versions then?

(I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part)

Tim

...

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Hi Guillermo - its 64bit I’m after for running AWS Lambda - the metacello version at 10mb looks very interesting (as the one I’m using from build #27 is 15mb). However, given I’m trying to find a stable place where I can develop on Pharo 6.1 and get a bit more experience - it looks like the latest post #27 images start breaking things for me with as I get weird errors (see below). But maybe your cleanup might resolve all of this. I really appreciate the work you guys are putting into have some minimal images - its very cool seeing all of this come together. Tim

On 7 Aug 2017, at 15:41, Guillermo Polito <guillermopolito@gmail.com> wrote:

Hi Tim,

in the file server:

http://files.pharo.org/image/70/ <http://files.pharo.org/image/70/>

You can get now latest 32 bit builds in many flavours: - compiler (basic pharo image with just a compiler) - core (compiler + package initialization) - monticello bootstrap (core + basic monticello packages -- no network here!) - monticello (monticello bootstrap + network support) - metacello (monticello + metacello) - default/full (image with everything loaded)

If you check the built images you'll see that:

- compiler is 4.7MB - core is 5.6MB - monticello bootstrap is 8.3MB - monticello is 9MB - metacello is 10MB - full pharo is 30MB

However, metacello and monticello images could be cleaned up a bit, as they still have some monticello cache to flush.

I'm working on uploading 64 bit images too now.

On Mon, Aug 7, 2017 at 4:26 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hey thanks again - it looks like that file is from Build #25 (I think Jenkins might cache the last 5 builds from the looks of it).

Interestingly that image is 2mb smaller than the later ones (which is helpful) - however when I try to use it, I get a walkback because Metacello wasn’t available in that build.

(I then tried with build #29 the latest? and #28) - and then I get an error as it seems that the Locale class is not defined in that one?

'Errors in script loaded from minimal.st <http://minimal.st/>' MessageNotUnderstood: receiver of "currentPlatform" is nil UndefinedObject(Object)>>doesNotUnderstand: #currentPlatform LanguageEnvironment class>>currentPlatform LanguageEnvironment class>>defaultSystemConverter TextConverter class>>defaultSystemConverter MultiByteFileStream>>converter MultiByteFileStream>>nextPut: MultiByteFileStream(WriteStream)>>cr UndefinedObject>>DoIt

So I guess the timing of my question was spot on as it looks like I’ve been running with build #27 for a few days (without realising I had cached it) and that one works.

I would be interesting in working out how to get metacello in build #25 as its smaller size is attractive (assuming its not metacello support that adds 3mb to the image size), Do you think its worth it? Can I use the command line package loader to shoe-horn it in?

Tim

...

On 7 Aug 2017, at 13:47, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Unfortunately we store minimal versions only for the latest build and you should use older build with hash b1625bf32c6b2b6c6f8babb789e377af5132d740.

You can download this image here (I have found a random copy of it): https://drive.google.com/open?id=0BzSsmZhqtUTeclFEbDhtd0JvdEk <https://drive.google.com/open?id=0BzSsmZhqtUTeclFEbDhtd0JvdEk>

Copy it to some public location you can reach.

-- Pavel

2017-08-07 14:25 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Thanks Pavel - as I’m downloading:

curl -L -o min.zip https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu... <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastSuccessfulBu...>

I noted that the last successful build of that asset was: Last stable build (#29), 2 days 23 hr ago <https://ci.inria.fr/pharo/job/70-Bootstrap-32bit-Conversion/lastStableBuild/> Should I peg myself to a specific version that is more or less compatible with Pharo 6.1? I can see builds 28 (Aug 4 11am) or 27 (July 31?) - or should I go earlier?

Can I rely on those builds sticking around - or should I cache one somewhere?

Tim

...

On 7 Aug 2017, at 12:03, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-07 12:56 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I’ve just switched branches and diffed - and yep, those methods were the difference. Sorry about the extra noise on that - I should have read your instructions a bit better - still I’ve learned a lot doing this, and am really chuffed that I’ve managed to get something working.

I’m looking at how to get my image size down a bit more - and have another thread on how to properly remove packages.

How stable do you think the minimal image will be? Should I try and snapshot it as a baseline - or do you think I can keep pulling safely from Jenkins for a while?

Pharo 7 is not stable in principle. You should not depend on the latest version but of course you can take a particular build that is supposed to be quite solid. Now you should rather take builds before Hermes integration and related refactorings.

-- Pavel

Tim

...

On 4 Aug 2017, at 21:19, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

yes, because master branch contains code of Pharo 6. For Pharo 7 we have flattened the traits in the kernel, but this is probably the only difference that affects Fuel packages

-- Pavel

2017-08-04 20:32 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah - I checked out master. So should I have checked out dev? Would this explain it?

I've appreciated the help a lot by the way.

Tim

Sent from my iPhone

On 4 Aug 2017, at 17:28, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

...

It it were packages from the development branch then you do not need to flatten them.

-- Pavel

2017-08-04 15:52 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - When you say “packages from the Pharo repo” - I think I’m doing that - yesterday I cloned the Pharo repo and then I copied the Fuel packages you guys suggested over into my branch (not elegant, but simple).

So I should have the flattened versions then?

(I have now managed to get the P7 image working - so thanks for all your help, but I’m curious about this part)

Tim

...

On 4 Aug 2017, at 12:49, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

Hmm, Pharo 7 has flattened traits in the kernel, but it still supports them. So probably because you are using vanilla Fuel packages, they extend traits that are not used for real. You should use packages from the Pharo repository.

-- Pavel

2017-08-04 13:42 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Sadly - false call, that was my logging message - so I’m stumped why these traits aren’t being compiled when I load in Fuel?

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn’t implement #isBigEndian.

Tim

...

On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

I’ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Compiling missing traits... Evaluating post load initialization... Finished Script.

However when I remove all the compiles I don’t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap <> --- cache ...finished baseline Evaluating post load initialization... Finished Script.

So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn’t seem to compile properly (in that case, it was loading a missing class that didn’t seem to cause recompilation).

Tim

...

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Actually there is something weird going on - as now I’m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian”, and that’s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

...

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote:

Hi Mariano - I’ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper ^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I’m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I’ve misunderstood what I should do (and I guess from Pavel’s comment - I should really look at why that initial method was missing as it sounds like it should have worked - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

...

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi Mariano - I compiled in the trait methods manually and it got further - but now I’m seeing another error - Error: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement”,

mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper ^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self

Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?

Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---", "Error: Explicitly required method", "", "\u001b[0m\u001b[31mError: Explicitly required method", "\u001b[0mLambda class(Object)>>error:", "explicitRequirement", " self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...", "False>>ifFalse:", "Lambda class(Object)>>explicitRequirement", "Lambda class(Class)>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "FLSerializer>>serialize:on:", "NonInteractiveUIManager>>writeFuelContext:", "Lambda class>>processDebug:", "Lambda class>>processJSON:", "Lambda class>>process:", "activate",

...

On 3 Aug 2017, at 17:26, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 1:17 PM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

Uff...that's a good question hahahhah I don't know if the minimal image supports traits, but what I am almost sure, is the Behavior DOES need the methods defined in TBehavior. So maybe the minimal image flattens the traits. In this case you could do the same for Fuel (flattenDownAllTraits)

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

TBehavior, TClass and TClassDescription are the only one that comes to my mind right now... there are only a few methods from Fuel. So could compile those directly at least on a first step...at least you keep you going.

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13

If Traits are supported then I must be doing something else wrong - maybe load order? (To be honest, I don’t fully understand how traits work - but I will keep digging). It could be that my latest error (having just done Class compile: ‘…. Missing method code … ‘) is something deeper related to these traits methods not applying somehow… I’ll try a few things this morning. It’s amazing how easily the image size jumps up when you add things though. With the original minimal v6 64bit image you sorted out for me I had a footprint of 21.6mb (image was 12mb). Now with the v7 image - and adding fuel and some S3 handling - it jumps up to 32.4mb (image is 15mb). I know that the S3 support has a lot of XML crud - but it seems quite astounding that this takes almost 8mb of code/objects. AWS loading (and hence response time) seems quite sensitive to the size of the package it has to load - although equally I wonder if there is also more #startUp/initalisation cost that is going on as well as when you get a warm lambda instance (so its not just loading your package) I’m still not seeing the 400-500ms times but more 800-1000ms times. Its all quite interesting - and shows why the work behind a minimal image is very useful. Of course there is tiny-minimal (which I know you guys are after) and then there is small-server- minimal (which is realistically what I’m after, so I can just focus on creating my app and just carefully load code and dependencies - but still have core server services like Fuel, and Git loading). Thanks for doing this work - as it brings a lot of useful things together. Tim

On 4 Aug 2017, at 06:48, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:

2017-08-03 18:17 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi guys - I think I’m getting close - I’m able to load remote packages, and I’ve added fuel to my repo and its loading - however I am getting an error when I run and test writing out a context - MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames

I can see that this is a trait - is trait support missing in the minimal image?

No, this image supports them

(I also seemed to get this far just loading the Fuel package - but just to be sure I added the other platform ones like you suggested and still get the same error). Given how simple that method is, I could just compile it in - but maybe there are many more?

Tim

"errorMessage": "Command failed: ./pharo Pharo.image --no-default-preferences process Lambda '{\"debug\":true,\"data\":5}'\n "errorType": "Error", "stackTrace": [ "Writing fuel context to S3...", "Using S3 bucket morethan-technology-projects", "\u001b[31m", "--- RUNNING ERROR HANDLER ---", "MessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "", "\u001b[0m\u001b[31mMessageNotUnderstood: Context class>>fuelIgnoredInstanceVariableNames", "\u001b[0mContext class(Object)>>doesNotUnderstand: #fuelIgnoredInstanceVariableNames", "FLVariablesMapping>>instanceVariableNamesToSerialize", "FLVariablesMapping>>initializeAnalyzing", "FLVariablesMapping class>>newAnalyzing:references:", "FLContextCluster(FLPointerObjectCluster)>>initializeAnalyzing:", "FLContextCluster class(FLObjectCluster class)>>newAnalyzing:", "clusterKeyedByObjectClass: t1 class: t2", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class: in Block: clusterKeyedByObjectClass: t1 class: t2...", "clusterInstanceOf: t1 keyInBucket: t2 factory: t3", " self error: 'No decompiler available' in FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory: in Block: clusterInstanceOf: t1 keyInBucket: t2 factory: t3...", "at: t1 ifAbsentPut: t2", " self error: 'No decompiler available' in IdentityDictionary(Dictionary)>>at:ifAbsentPut: in Block: at: t1 ifAbsentPut: t2...", "IdentityDictionary(Dictionary)>>at:ifAbsent:", "IdentityDictionary(Dictionary)>>at:ifAbsentPut:", "FLLightGeneralMapper(FLMapper)>>clusterInstanceOf:keyInBucket:factory:", "FLLightGeneralMapper(FLMapper)>>clusterKeyedByObjectClass:class:", "FLLightGeneralMapper(FLMapper)>>mapAndTraceByObjectClass:to:", "FLLightGeneralMapper>>visitMethodContext:", "Context>>fuelAccept:", "FLLightGeneralMapper>>mapAndTrace:", "FLLightGlobalMapper>>mapAndTrace:", "FLPluggableSubstitutionMapper>>mapAndTrace:", "FLAnalysis>>mapAndTrace:", "FLAnalysis>>run", "setDefaultAnalysis", " self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...", "FLAnalyzer>>analysisFor:", "FLSerialization>>analysisStep", "FLSerialization>>run", "setDefaultSerialization", " self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...", "serialize: t1 on: t2", " self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...", "on: t1 globalEnvironment: t2 do: t3", " self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...", "BlockClosure>>ensure:", "FLEncoder class>>on:globalEnvironment:do:", "\u001b[0m"

...

On 3 Aug 2017, at 13:16, Mariano Martinez Peck <marianopeck@gmail.com <mailto:marianopeck@gmail.com>> wrote:

On Thu, Aug 3, 2017 at 8:58 AM, Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>> wrote: I’m so close now (bit of a diversion with gitlab yml not liking the eval string) - that did load and it got me really close - I’m just missing FlSerializer now (I have progressed my work to fuel out the context for debugging to S3)… so looking at the repo, there are quite a few fuel packages - but in a big image I can see that FLSerialiser is in Core - so do I just need core? But then there are a few fuel packages with Core?

The easiest way to understand that is either checking ConfigurationOfFuel directly or some of the related Metacello tool. Anyway, for this case, I think you need the packages (in this order): FuelPlatform and Fuel. Bug again, Metacello allows some postLoad actions (which we use in Fuel)... so...it's not always easy to load a project without Metacello.

Sometimes you can print the result of #record:

Metacello new configuration: 'Fuel'; smalltalkhubUser: 'Pharo' project: 'Fuel'; version: #stable; record: #('Core').

->

"linear load : linear load : 2.1.10 [ConfigurationOfFuel] linear load : baseline [ConfigurationOfFuelPlatform] load : FuelPlatform-Core load : FuelPlatform-Pharo-Core load : FuelPlatform-Pharo-06"

------

Anyway.... I guess I would try to load these packages (in this order):

FuelPlatform-Core FuelPlatform-Pharo-Core FuelPlatform-Pharo-06 Fuel

And when you are down loading, then execute:

(Smalltalk at: #FLPlatform) current addHacks

I’m also not quite clear on how I would load packages without a Baseline (would I create my own baseline for the fuel packages I need?)

Tim

...

On 3 Aug 2017, at 12:07, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 13:02 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I really appreciate your patience and help on this - it looks very promising and I am giving it a spin now…

When you say the pharo repository - are you referring to this: https://github.com/pharo-project/pharo/tree/master/src <https://github.com/pharo-project/pharo/tree/master/src>. ? Just so I know for the future.

yes

-- Pavel

Tim

...

On 3 Aug 2017, at 09:16, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-03 10:14 GMT+02:00 Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>>: The easiest think you can do is to copy to your repository (or some other) related packages from the Pharo repository (to do not have to clone it all):

Metacello-GitBasedRepository.package Metacello-GitHub.package SUnit-Core.package

and create baseline to load them. I already tried it so you can test it: https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8 <https://drive.google.com/open?id=0BzSsmZhqtUTeMVJacTZtdW5UQW8>

Then you will do something like:

(accidental message send ;-) )

BASELINE=MetacelloGitBasedRepository pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

BASELINE=Lambda pharo "$IMAGE_NAME.image" --no-default-preferences eval --save \ "Metacello new baseline: '$BASELINE'; repository: 'filetree://./PharoLambda/src <>'; load."

As I tried that. The baseline of Lambda is then able to be loaded.

-- Pavel

2017-08-02 15:18 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Ah, I think I’m starting to get closer to what I need…

So if I want to load in that last piece to enable more general remote loading - how do I figure out how to do that? I’m trying to work out where the build steps for building up the image (can I see them in Jenkins? It wasn’t clear to me if I can look it up? Or the BaselineOfIDE was mentioned - looking there I can see a few metacello and gofer packages - but then I guess I’m looking for an easy Mcz I can use with the example below?

Or do I just load Metacello as a git submodule and then it will be on my local filesystem to then bootstrap up?

I guess I’m trying to work out the best sustainable approach to getting to a good server based image that has minimal tools and the ability to easily load remote code for pre-req projects.

Tim

...

On 2 Aug 2017, at 07:05, Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>> wrote:

Yes, you should be able to load an mcz in that image by doing:

(MCDirectoryRepository new directory: 'where-your-mcz-is') loadVersionFromFileNamed: 'Metacello-GitBasedRepository-Author.1.mcz') load.

Guille

On Wed, Aug 2, 2017 at 7:57 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-08-01 23:13 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: Hi Pavel - I tried it again and the problem is do with Metacello dependencies in your baseline.

The SUnit baseline doesn’t specify any additional dependencies to load (it just loads local packages), whereas my baseline that fails looks like this:

baseline: spec <baseline>

spec for: #common do: [ spec configuration: 'ZTimestamp' with: [ spec versionString: #stable; repository: 'http://mc.stfx.eu/Neo <http://mc.stfx.eu/Neo>' ].

spec baseline: 'AWS' with: [ spec repository: 'github://newapplesho/aws-sdk-smalltalk:v1.10/pharo-repository <>' ].

spec package: 'Lambda' with: [ spec requires: {'ZTimestamp'. 'AWS'}].

].

The “spec configuration: ….” Specifications cause the error:

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository:

If I remove those lines from my baseline above (as well the requires: reference to them) I can then successfully load my packages.

So is this something the minimal image should support (otherwise how do you load external packages into your image without checking them in locally?) OR is there something simple I can checkin and load locally to return that behaviour? (Personally I think it would make sense to allow remote loading - I’m guessing Pharo itself as a core has everything it needs - but if you want to do any experimentation on the core and need remote packages, you would hit this too - so it feels a bit limiting).

The dependencies in th baseline are supported but the support for most of external repositories (like package Metacello-GitBasedRepository) is loaded at the end of bootstrapping process in BaselineOfIDE. We should check/solve dependencies and move it into the minimal Pharo. For now you can try to load it by yourself or work with already prepared local clones of required repositories.

-- Pavel

Tim

...

On 1 Aug 2017, at 10:06, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:

2017-07-31 22:51 GMT+02:00 Tim Mackinnon <tim@testit.works <mailto:tim@testit.works>>: I wasn’t clear on which image to retry - the https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf... <https://ci.inria.fr/pharo/job/Pharo-6.0-Update-Step-3.2-Minimal/lastSuccessf...> one still shows as being last updated 7 days ago.

The https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur... <https://ci.inria.fr/pharo/view/6.0-SysConf/job/Pharo-6.0-Step-04-01-Configur...> one gives me a mismatch error: This interpreter (vers. 68021) cannot read image file (vers. 6521).

The https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc... <https://ci.inria.fr/pharo/view/7.0/job/70-Bootstrap-32bit-Conversion/lastSuc...> one gives me a walkback when trying to run my install script :

25 UndefinedObject(Object)>>doesNotUnderstand: #addTo: 26 MCRepositoryGroup>>addRepository: 27 createRepository | repo | repo := self project createRepository: self. ^ MCRepositoryGroup default repositories detect: [ :each | each = repo ] ifNone: [ MCRepositoryGroup default addRepository: repo. repo ] in MetacelloRepositorySpec>>createRepository

I think this is because the image doesn’t support Metacello? As in: Metacello new repository: 'filetree://../src <>'; baseline: 'Lambda'; load.

How do you guys load things into the minimal images (I thought you used Metacello - but maybe you do it some other way?)

I can use a big 6.1 image fine (as it has Metacello loaded) but I’d really like a minimal solution - that can load in libraries like AWS S3, or XML parsing etc. and it seems like I should be a good customer for kicking the tires on all of this.

Tim

I checked the 64-bit Pharo 7 minimal image and loading of baseline (of SUnit from pharo-project/pharo) works:

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval --save "Metacello new baseline: 'SUnit'; repository: 'filetree://./pharo-core/src <>'; load."

./pharo Pharo7.0-minimal-64bit-b1625bf.image eval "TestCase suite run"

Can you test it too?

-- Pavel

--

Guille Polito

Research Engineer French National Center for Scientific Research - http://www.cnrs.fr <http://www.cnrs.fr/>

Web: http://guillep.github.io <http://guillep.github.io/> Phone: +33 06 52 70 66 13 <tel:+33%206%2052%2070%2066%2013>

-- Mariano http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>