On 17 Jan 2012, at 22:40, Mariano Martinez Peck wrote:

1) From a design point of view, I think that CompiledMethodTrailer is the correct place where source code should be stored.

Why?

From my point of view, the source fits into the image, but does not belong into the CompiledMethod. CompiledMethod is the representation for execution purpose. Encoding it into the bytearray along the the bytecode just asks for trouble.

CompiledMethod should have a counter part, which can also carry all kind of other meta data. I bet there is something in the ring framework to represent methods that is not based on CompiledMethod. That is where I would put the code.

3) I think (I should measure it) that having source code in the image is faster, even thought that's the least I care about.

The finder is obviously faster... Other tools wont benefit as much, because the cost of reading a single method is insignificant. Refactorings which might need to read code, or recompile everything might benefit.

4) Ease the logic of pointers in the trailer... 5) Easy of versioning for a binary serializer ;) so for example in Fuel we can now be able to serialize compiled methods with trailer that have the source code. Hence, versioning it (Monticello) is easier.

I still do not see why you would want the extra complexity (with encoding and all) to put it into the ByteArray of compiled method. That makes everything harder, and one argument you might want to use, no, it does not guarantee consistency between the two parts. Putting the code into the image, yes, but not into the compiled method. At least I do not see any clear benefits. Best regards Stefan -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525

On 19 February 2012 00:54, Craig Latta <craig@netjam.org> wrote:

Hi--

    If you have some way of uniquely identifying a method, then you can put the source wherever you like (e.g., in some other live object memory), and look it up when you need it. For an example, see the MethodID class in Spoon[1].

indeed. it is really unimportant where to store the method(s). They can be stored in file, remote DB or remote image.. But first we need to change the system to work in that way, by removing hardcoded SourceFiles everywhere and replacing with nice polymorphic source manager(s).

-C

[1] http://dl.dropbox.com/u/15188004/spoon/Spoon%203%20alpha%203.zip

-- Craig Latta www.netjam.org/resume +31 6 2757 7177 + 1 415 287 3547

-- Best regards, Igor Stasenko.

On 19 February 2012 13:37, Igor Stasenko <siguctua@gmail.com> wrote:

On 19 February 2012 00:54, Craig Latta <craig@netjam.org> wrote:

...

Hi--

    If you have some way of uniquely identifying a method, then you can put the source wherever you like (e.g., in some other live object memory), and look it up when you need it. For an example, see the MethodID class in Spoon[1].

indeed. it is really unimportant where to store the method(s). They can be stored in file, remote DB or remote image.. But first we need to change the system to work in that way, by removing hardcoded SourceFiles everywhere and replacing with nice polymorphic source manager(s).

It's unimportant right up until you're not online. Please don't write systems that assume a reliable network! (Reply: "You can always cache source, and possibly supply a prepopulated cache, called a .sources file.") frank

...

-C

[1] http://dl.dropbox.com/u/15188004/spoon/Spoon%203%20alpha%203.zip

-- Craig Latta www.netjam.org/resume +31 6 2757 7177 + 1 415 287 3547

-- Best regards, Igor Stasenko.

On Tue, Jan 17, 2012 at 1:40 PM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:

Well, with the following script it seems I am able to embed all source

...

code in trailers:

| trailer source key dict classes | dict := IdentityDictionary new. CompiledMethod allInstances do: [:aMethod | trailer := aMethod trailer. source := aMethod getSource. trailer sourceCode: source. dict at: aMethod put: (aMethod copyWithTrailerBytes: trailer). ]. dict keys asArray elementsExchangeIdentityWith: dict values asArray.

Problem is that after executing the script, when a method is recompiled from the system browser, it puts back a trailer with a .changes pointer :( I will see if I find where I can change that.

In another thread, Stefan Marr asked "Is there any general use for such an embedding, beside making the finder usable?"

I would just give my answer:

1) From a design point of view, I think that CompiledMethodTrailer is the correct place where source code should be stored.

I disagree. The changes file is synergistic: - provides crash recovery - removes significant space overhead from the image - changes file plus sources file provide method source history

2) I can understand to put source code in files would make sense 20 years ago. But I don't think so anymore... a normal notebook has 8gb of RAM these days.

it is not just a space trade-off. see above.

3) I think (I should measure it) that having source code in the image is faster, even thought that's the least I care about.

certainly. but doing a better job with readOnly copies of the changes file speeds up things a lot. there is no need to keep more than one read-only copy of the changes file. I have old Newspeak code for 3.9 if you're interested.

4) Ease the logic of pointers in the trailer... 5) Easy of versioning for a binary serializer ;) so for example in Fuel we can now be able to serialize compiled methods with trailer that have the source code. Hence, versioning it (Monticello) is easier.

that's a fair reason. but with parcels having two files, objects (code) and source, worked well, e.g. deploying without source means not including the sources file.

So...what I would do (that's just my opinion) is:

1) always store source code in trailer 2) do not use .sources anymore 3) Use .changes ONLY for recovery but not for getting source code (#getSource) 4) Provide a way to shrink an image for production cases where people is interested in image size. Can can very easily replace compiled methods trailers with empty trailers (zap) and then if it just happens that you need the sources..you decompile it.

Maybe there are obvious drawbacks I am not seeing...

Cheers

-- Mariano http://marianopeck.wordpress.com

-- best, Eliot