Hi - I was actually reading those slides the other day, but hadn���t fully appreciated the consequences��� that is very cool - and definitely something for me to put on the stack of things to try out I was going to say I could put the Fuel and S3 access into such a library - but then I would need them to load it from S3 - so catch 22 for that, but I���ll bet I come up with another usage.

Thanks to everyone for their help with this.

Is there anything I need follow up on from this? I saw you asked Max about the Platform7 piece. I also don���t understand why I have problems with those missing fuel trait methods (so I���ll see what Pavel says about that).


Tim

On 4 Aug 2017, at 14:24, Mariano Martinez Peck <marianopeck@gmail.com> wrote:



On Fri, Aug 4, 2017 at 9:51 AM, Tim Mackinnon <tim@testit.works> wrote:
HURRAH - I got it to work, and Fuel out an error to S3 and read it back into a fat 6.1 image with a debug stack (which is so cool!).



COOOL!!! I want to see your blogpost once you make it!!! 
Congratulations, this was a very cool thread to follow :)
 
My footprint size is still a bit larger than I would like (31mb - image of 22.7mb - of which 1mb if fuel and 7mb is S3 and XMLParser support) - so I guess I need to find ways to trim down some of the addons (and any additional image size reductions would be helpful - if there are any to be had).

Is there any easy tools/scripts to explore a headless image and figure out what takes the most size (maybe at a package level?). But definitely this has got me into a good place for experimentation.



I know this is a bit offtopic... but maybe to give you some perspective... For my PhD, one of the things I did (lets say the closest to the PhD topic) was called Marea [1]   (I can also send you the slides and the PhD defense video on youtube if you want). In Marea what I did was to modify the VM to implement a very basic (with lots of limitations) object usage tracking...I simply flagged when an object was used and cleared the flag every in a while. Then, at image side, I would find graphs of unused objects, and replaced the boundary objects with proxies using Ghost proxies (note that an anused object is not the same as unreferenced object,...GC does nothing here). The graphs where then serialized with Fuel. Finally, if those graphs happened to be needed, then the proxy would intercept the message, materialize from Fuel, and plug back the original graph.

Anyway... all of what was to said that I some point I did an experiment. I was able to already proxify and serialize classes, methods, etc. So I took the whole image and I swapped out all classes and their instances (each class with its instances in a different graph) but only a really small core. This image was a Seaside image running DBX Pier website.  So I swapped out everything, and then I lazily started to navigate DBX website, causing the needed graphs to be swapped in. I was able to naviagat all webapp and use it perfectly. I even saved the image.  And all this email is to say that such an image was 3MB.  


Cheers,





 
 
 
For the record I had to do the following to get things working:

Define a new baseline:
baseline: spec
   <baseline>

   spec for: #common do: [
      spec 
         package: 'Metacello-GitBasedRepository';
         package: 'Metacello-GitHub';
          package: 'FuelPlatform-Core';
            package: 'FuelPlatform-Pharo-Core';
            package: 'FuelPlatform-Pharo-06';
            package: 'Fuel'.      
   ].    

Run the following script:
Metacello new
    baseline: 'MetacelloGitBasedRepository';
    repository: 'filetree://../bootstrap';
    load.

���Replace missing traits"
Behavior compiledMethodAt: #fuelIgnoredInstanceVariableNames
    ifPresent: [ :m | logger cr; nextPutAll: 'Trait method is present' ]
    ifAbsent: [
        logger cr; nextPutAll: 'Recompile missing traits ...'.

        Behavior
            compile: 'fuelIgnoredInstanceVariableNames
                       "Indicates which variables have to be ignored during serialization."

                       ^#()';
            compile: 'fuelNew
                       "Answer an instance of mine in which serialized references will be injected."

                       ^ self basicNew';
            compile: 'fuelNew: sizeRequested
                       "Answer an instance of mine in which serialized references will be injected."

                       ^ self basicNew: sizeRequested'.

        "Ignore this one"
        "Class
            compile: 'fuelAccept: aGeneralMapper
                       ^self explicitRequirement.'."

        ClassDescription
            compile: 'instanceVariableNamesDo: anUnaryBlock
                       "This is part of the interface between the compiler and a class''s instance or field names.
                        The class should enumerate anUnaryBlock with the instance variable name strings.  The order is important. Names evaluated later will override the
                        same names occurring earlier."

                       | superInstSize |
                       (superInstSize := self superclass notNil ifTrue: [self superclass instSize] ifFalse: [0]) > 0 ifTrue:
                          [self superclass instanceVariableNamesDo: anUnaryBlock].
                       1 to: self instSize - superInstSize do:
                          [:i| anUnaryBlock value: (self instVarNames at: i)]'.
    ].

���Add a tacky Pharo7Platform"
Smalltalk at: #FLPharo7Platform ifAbsent: [
    logger cr; nextPutAll: '>Creating FLPharo7Platform'.
    (Smalltalk at: #FLPharo6Platform) subclass: #FLPharo7Platform.
    (Smalltalk at: #FLPharo6Platform) class compile: 'isResponsibleForCurrentPlatform
            ^true'.
].


logger cr; nextPutAll: 'Evaluating post load initialization...'.
(Smalltalk at: #FLPlatform) current addHacks.


On 4 Aug 2017, at 12:48, Mariano Martinez Peck <marianopeck@gmail.com> wrote:



On Fri, Aug 4, 2017 at 8:42 AM, Tim Mackinnon <tim@testit.works> wrote:
Sadly - false call, that was my logging message - so I���m stumped why these traits aren���t being compiled when I load in Fuel? 

However I have figured out why I get the BigEndian error - this is because the platform is now 7 and so the FlPharo06Platform is not being identified and the more generic FlPharoPlatform is then the default and it doesn���t implement #isBigEndian.



Max, should we do something to the Fuel Platform for Pharo 7?


 
Tim



On 4 Aug 2017, at 12:20, Tim Mackinnon <tim@testit.works> wrote:

I���ve spotted something interesting in my logs - when I added one of the missing methods to ClassDescription - I see in the log the message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap --- cache
...finished baseline
Compiling missing traits...
Evaluating post load initialization...
Finished Script.

However when I remove all the compiles I don���t see this message:

Loaded -> Fuel-cypress.1 --- filetree:///builds/macta/PharoLambda/bootstrap --- cache
...finished baseline
Evaluating post load initialization...
Finished Script.


So I wonder if this is similar to the problem I had in the previous 6.0 image - where some things didn���t seem to compile properly (in that case, it was loading a missing class that didn���t seem to cause recompilation).

Tim

On 4 Aug 2017, at 11:10, Tim Mackinnon <tim@testit.works> wrote:

Actually there is something weird going on - as now I���m getting a complaint about "FLPharoPlatform had the subclass responsibility to implement #isBigEndian���, and that���s in the FuelPlatform-Pharo-06 which you said to include (and it looks like I am).

I need to double check this - as I must be making some trivial mistake.

Tim

On 4 Aug 2017, at 11:01, Tim Mackinnon <tim@testit.works> wrote:

Hi Mariano - I���ve looked at this again, and it was my fault - I was compiling in :

fuelAccept: aGeneralMapper
^self explicitRequirement.

Because when I looked at the Traits for fuel - this is what it had for TClass? - as I���m not so clear on traits, what does this mean - as I had just assumed that I should flatten the Fuel methods for TCLass, TBehaviour and TClassDescription - but I think I���ve misunderstood what I should do (and I guess from Pavel���s comment - I should really look at why that initial method was missing as it sounds like it should have worked  - so maybe its an order loading thing?).

Thanks, for helping by the way - this is proving very instructional.

Tim

On 3 Aug 2017, at 18:26, Mariano Martinez Peck <marianopeck@gmail.com> wrote:


On Thu, Aug 3, 2017 at 2:14 PM, Tim Mackinnon <tim@testit.works> wrote:
Hi Mariano - I compiled in the trait methods manually and it got further - but now I���m seeing another error - Error: Explicitly required method",
    "\u001b[0mLambda class(Object)>>error:",
    "explicitRequirement���,


mmmm the stack is weird... it looks like if Class >> fuelAccept: would have the wrong implementation, that is, having the one from TClass:

fuelAccept: aGeneralMapper
^self explicitRequirement.

Rather than the correct one:

fuelAccept: aGeneralMapper

^aGeneralMapper visitClass: self


Maybe the solution is to compile all those methods from traits BUT NOT those that are "^self explicitRequirement." because those will override the good methods.

If that doesn't work, then maybe can you print to console the each impl (source) of each implementor of #fuelAccept: ?

Thoughts?



 
Any ideas?

Tim

"--- RUNNING ERROR HANDLER ---",
    "Error: Explicitly required method",
    "",
    "\u001b[0m\u001b[31mError: Explicitly required method",
    "\u001b[0mLambda class(Object)>>error:",
    "explicitRequirement",
    "  self error: 'No decompiler available' in Lambda class(Object)>>explicitRequirement in Block: explicitRequirement...",
    "False>>ifFalse:",
    "Lambda class(Object)>>explicitRequirement",
    "Lambda class(Class)>>fuelAccept:",
    "FLLightGeneralMapper>>mapAndTrace:",
    "FLLightGlobalMapper>>mapAndTrace:",
    "FLPluggableSubstitutionMapper>>mapAndTrace:",
    "FLAnalysis>>mapAndTrace:",
    "FLAnalysis>>run",
    "setDefaultAnalysis",
    "  self error: 'No decompiler available' in FLAnalyzer>>setDefaultAnalysis in Block: setDefaultAnalysis...",
    "FLAnalyzer>>analysisFor:",
    "FLSerialization>>analysisStep",
    "FLSerialization>>run",
    "setDefaultSerialization",
    "  self error: 'No decompiler available' in FLSerializer>>setDefaultSerialization in Block: setDefaultSerialization...",
    "serialize: t1 on: t2",
    "  self error: 'No decompiler available' in FLSerializer>>serialize:on: in Block: serialize: t1 on: t2...",
    "on: t1 globalEnvironment: t2 do: t3",
    "  self error: 'No decompiler available' in FLEncoder class>>on:globalEnvironment:do: in Block: on: t1 globalEnvironment: t2 do: t3...",
    "BlockClosure>>ensure:",
    "FLEncoder class>>on:globalEnvironment:do:",
    "FLSerializer>>serialize:on:",
    "NonInteractiveUIManager>>writeFuelContext:",
    "Lambda class>>processDebug:",
    "Lambda class>>processJSON:",
    "Lambda class>>process:",
    "activate",





--
Mariano
http://marianopeck.wordpress.com