- Pharo-dev - lists.pharo.org

Re: [Pharo-project] detect if image was saved with Cog

by Sven Van Caekenberghe

OK, thanks guys, I learrned new things today. -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill On 01 Oct 2012, at 17:08, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > On 01.10.2012 16:56, Sven Van Caekenberghe wrote: >> On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> On 01.10.2012 14:43, Sven Van Caekenberghe wrote: >>>> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>>> >>>>> Endianness in header depends on the platform the image was saved on. >>>>> Which for all practical purposes, means little-endian these days. >>>> How can an image be cross-platform, but the header not ? >>>> >>> IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. >> Thanks for the answer, I suspected something like that. >> >> But that means that the image format on disk, being binary, is native only to one endianess and not the other, > Yes >> making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ? >> > Swapping happens at startup, so penalty is only when loading a big-endian image on little-endian platform or vice versa. > Without saving, that happens every time, it is not automatically saved if a conversion takes place when loading. > > When saving, it saves in the platform endianness (which is the same as is in memory), and incur no extra penalty. > > Incidentally, that's the same approach I suggested Mariano use in fuel. > IMHO, it really is most optimal approach when platform endianness changes are rare, and the penalty can easily be avoided by simply saving the image on the platform it is intended to be used. > > Cheers, > Henry > >

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 17:08, Henrik Sperre Johansen wrote: > > IMHO, it really is most optimal approach when platform endianness > changes are rare, and the penalty can easily be avoided by simply > saving the image on the platform it is intended to be used. > > Cheers, > Henry > In other words; it's not really something you usually need to think about unless you are developing closed applications shipping with a fixed image. Obviously, one such case is phone applications, John mentioned its impact on iPhone apps here; http://forum.world.st/Where-is-the-heap-tp2195336p2195639.html Interestingly, (at least the way I read it) the real cost was memory access, and endianness swap was really just one of multiple items that fell under the category "Accesses the entire object memory at startup". Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by David T. Lewis

On Mon, Oct 01, 2012 at 05:07:48PM +0200, Henrik Sperre Johansen wrote: > On 01.10.2012 15:44, Igor Stasenko wrote: > >On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >>On 01 Oct 2012, at 14:01, Henrik Sperre Johansen > >><henrik.s.johansen(a)veloxit.no> wrote: > >> > >>>Endianness in header depends on the platform the image was saved on. > >>>Which for all practical purposes, means little-endian these days. > >>How can an image be cross-platform, but the header not ? > >> > >Just another artifact of incremental development, i guess. > >I don't know what reasoning was behind this, but IMO, defining header > >to have fixed byte order > >would be much better. Of course, it is always easier to think backwards. > >However, things like IP header, were existed before squeak image > >format. So, again, > >i am clueless why guys decided that doing like it's done is a Good Idea > >(tm). > > > IIRC, the code that writes the image header is in Slang. > Using uints for fields there would be a logical choice, I would guess. Yes, it is done in #checkImageVersionFrom:startingAt: > > TBH, I don't think it would be much better if a byte order were enforced > for a single header field only. > > Though, I personally would think it'd be nice if the format field were > to be expanded and kept symmetrical. > That way, deciphering the image formats numerical value could be done > independently of endianness (and knowing what #'s are valid formats). > > As nice as backwards compatability though, considering there would be no > breaking image format changes per se? Probably not. It would be straightforward to design a new object file header, so long as the VM retains the ability to read the old one. The resulting image could not be read by older VMs but otherwise there would be no problem. Dave

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by David T. Lewis

On Mon, Oct 01, 2012 at 04:56:39PM +0200, Sven Van Caekenberghe wrote: > > On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > > > On 01.10.2012 14:43, Sven Van Caekenberghe wrote: > >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> > >>> Endianness in header depends on the platform the image was saved on. > >>> Which for all practical purposes, means little-endian these days. > >> How can an image be cross-platform, but the header not ? > >> > > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. > > Thanks for the answer, I suspected something like that. > > But that means that the image format on disk, being binary, is native only to one endianess and not the other, making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ? > > Normally the conversion does not need to be done, because the common use case is for starting an image on the same (or similar) platform. That means that in the common case, object memory can be directly loaded without word swapping, which is a bit performance win when loading the image. Of course the header part of the file is different from the object memory itself, so Igor is correct: The real reason it got done that way is ... that's how it got done ;) Dave

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 16:56, Sven Van Caekenberghe wrote: > On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> On 01.10.2012 14:43, Sven Van Caekenberghe wrote: >>> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>> >>>> Endianness in header depends on the platform the image was saved on. >>>> Which for all practical purposes, means little-endian these days. >>> How can an image be cross-platform, but the header not ? >>> >> IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. > Thanks for the answer, I suspected something like that. > > But that means that the image format on disk, being binary, is native only to one endianess and not the other, Yes > making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ? > Swapping happens at startup, so penalty is only when loading a big-endian image on little-endian platform or vice versa. Without saving, that happens every time, it is not automatically saved if a conversion takes place when loading. When saving, it saves in the platform endianness (which is the same as is in memory), and incur no extra penalty. Incidentally, that's the same approach I suggested Mariano use in fuel. IMHO, it really is most optimal approach when platform endianness changes are rare, and the penalty can easily be avoided by simply saving the image on the platform it is intended to be used. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 15:44, Igor Stasenko wrote: > On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> Endianness in header depends on the platform the image was saved on. >>> Which for all practical purposes, means little-endian these days. >> How can an image be cross-platform, but the header not ? >> > Just another artifact of incremental development, i guess. > I don't know what reasoning was behind this, but IMO, defining header > to have fixed byte order > would be much better. Of course, it is always easier to think backwards. > However, things like IP header, were existed before squeak image > format. So, again, > i am clueless why guys decided that doing like it's done is a Good Idea (tm). > IIRC, the code that writes the image header is in Slang. Using uints for fields there would be a logical choice, I would guess. TBH, I don't think it would be much better if a byte order were enforced for a single header field only. Though, I personally would think it'd be nice if the format field were to be expanded and kept symmetrical. That way, deciphering the image formats numerical value could be done independently of endianness (and knowing what #'s are valid formats). As nice as backwards compatability though, considering there would be no breaking image format changes per se? Probably not. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Sven Van Caekenberghe

On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > On 01.10.2012 14:43, Sven Van Caekenberghe wrote: >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> Endianness in header depends on the platform the image was saved on. >>> Which for all practical purposes, means little-endian these days. >> How can an image be cross-platform, but the header not ? >> > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. Thanks for the answer, I suspected something like that. But that means that the image format on disk, being binary, is native only to one endianess and not the other, making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ?

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 14:43, Sven Van Caekenberghe wrote: > On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> Endianness in header depends on the platform the image was saved on. >> Which for all practical purposes, means little-endian these days. > How can an image be cross-platform, but the header not ? > > > -- > Sven Van Caekenberghe > http://stfx.eu > Smalltalk is the Red Pill > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Igor Stasenko

On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> Endianness in header depends on the platform the image was saved on. >> Which for all practical purposes, means little-endian these days. > > How can an image be cross-platform, but the header not ? > Just another artifact of incremental development, i guess. I don't know what reasoning was behind this, but IMO, defining header to have fixed byte order would be much better. Of course, it is always easier to think backwards. However, things like IP header, were existed before squeak image format. So, again, i am clueless why guys decided that doing like it's done is a Good Idea (tm). -- Best regards, Igor Stasenko.

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Sven Van Caekenberghe

On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > Endianness in header depends on the platform the image was saved on. > Which for all practical purposes, means little-endian these days. How can an image be cross-platform, but the header not ? -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Oct. 1, 2012