October 2012 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 17:08, Henrik Sperre Johansen wrote: > > IMHO, it really is most optimal approach when platform endianness > changes are rare, and the penalty can easily be avoided by simply > saving the image on the platform it is intended to be used. > > Cheers, > Henry > In other words; it's not really something you usually need to think about unless you are developing closed applications shipping with a fixed image. Obviously, one such case is phone applications, John mentioned its impact on iPhone apps here; http://forum.world.st/Where-is-the-heap-tp2195336p2195639.html Interestingly, (at least the way I read it) the real cost was memory access, and endianness swap was really just one of multiple items that fell under the category "Accesses the entire object memory at startup". Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by David T. Lewis

On Mon, Oct 01, 2012 at 05:07:48PM +0200, Henrik Sperre Johansen wrote: > On 01.10.2012 15:44, Igor Stasenko wrote: > >On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > >>On 01 Oct 2012, at 14:01, Henrik Sperre Johansen > >><henrik.s.johansen(a)veloxit.no> wrote: > >> > >>>Endianness in header depends on the platform the image was saved on. > >>>Which for all practical purposes, means little-endian these days. > >>How can an image be cross-platform, but the header not ? > >> > >Just another artifact of incremental development, i guess. > >I don't know what reasoning was behind this, but IMO, defining header > >to have fixed byte order > >would be much better. Of course, it is always easier to think backwards. > >However, things like IP header, were existed before squeak image > >format. So, again, > >i am clueless why guys decided that doing like it's done is a Good Idea > >(tm). > > > IIRC, the code that writes the image header is in Slang. > Using uints for fields there would be a logical choice, I would guess. Yes, it is done in #checkImageVersionFrom:startingAt: > > TBH, I don't think it would be much better if a byte order were enforced > for a single header field only. > > Though, I personally would think it'd be nice if the format field were > to be expanded and kept symmetrical. > That way, deciphering the image formats numerical value could be done > independently of endianness (and knowing what #'s are valid formats). > > As nice as backwards compatability though, considering there would be no > breaking image format changes per se? Probably not. It would be straightforward to design a new object file header, so long as the VM retains the ability to read the old one. The resulting image could not be read by older VMs but otherwise there would be no problem. Dave

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by David T. Lewis

On Mon, Oct 01, 2012 at 04:56:39PM +0200, Sven Van Caekenberghe wrote: > > On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > > > On 01.10.2012 14:43, Sven Van Caekenberghe wrote: > >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> > >>> Endianness in header depends on the platform the image was saved on. > >>> Which for all practical purposes, means little-endian these days. > >> How can an image be cross-platform, but the header not ? > >> > > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. > > Thanks for the answer, I suspected something like that. > > But that means that the image format on disk, being binary, is native only to one endianess and not the other, making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ? > > Normally the conversion does not need to be done, because the common use case is for starting an image on the same (or similar) platform. That means that in the common case, object memory can be directly loaded without word swapping, which is a bit performance win when loading the image. Of course the header part of the file is different from the object memory itself, so Igor is correct: The real reason it got done that way is ... that's how it got done ;) Dave

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 16:56, Sven Van Caekenberghe wrote: > On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> On 01.10.2012 14:43, Sven Van Caekenberghe wrote: >>> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >>> >>>> Endianness in header depends on the platform the image was saved on. >>>> Which for all practical purposes, means little-endian these days. >>> How can an image be cross-platform, but the header not ? >>> >> IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. > Thanks for the answer, I suspected something like that. > > But that means that the image format on disk, being binary, is native only to one endianess and not the other, Yes > making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ? > Swapping happens at startup, so penalty is only when loading a big-endian image on little-endian platform or vice versa. Without saving, that happens every time, it is not automatically saved if a conversion takes place when loading. When saving, it saves in the platform endianness (which is the same as is in memory), and incur no extra penalty. Incidentally, that's the same approach I suggested Mariano use in fuel. IMHO, it really is most optimal approach when platform endianness changes are rare, and the penalty can easily be avoided by simply saving the image on the platform it is intended to be used. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 15:44, Igor Stasenko wrote: > On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> Endianness in header depends on the platform the image was saved on. >>> Which for all practical purposes, means little-endian these days. >> How can an image be cross-platform, but the header not ? >> > Just another artifact of incremental development, i guess. > I don't know what reasoning was behind this, but IMO, defining header > to have fixed byte order > would be much better. Of course, it is always easier to think backwards. > However, things like IP header, were existed before squeak image > format. So, again, > i am clueless why guys decided that doing like it's done is a Good Idea (tm). > IIRC, the code that writes the image header is in Slang. Using uints for fields there would be a logical choice, I would guess. TBH, I don't think it would be much better if a byte order were enforced for a single header field only. Though, I personally would think it'd be nice if the format field were to be expanded and kept symmetrical. That way, deciphering the image formats numerical value could be done independently of endianness (and knowing what #'s are valid formats). As nice as backwards compatability though, considering there would be no breaking image format changes per se? Probably not. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Sven Van Caekenberghe

On 01 Oct 2012, at 16:33, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > On 01.10.2012 14:43, Sven Van Caekenberghe wrote: >> On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: >> >>> Endianness in header depends on the platform the image was saved on. >>> Which for all practical purposes, means little-endian these days. >> How can an image be cross-platform, but the header not ? >> > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. Thanks for the answer, I suspected something like that. But that means that the image format on disk, being binary, is native only to one endianess and not the other, making startup or saving a image faster/slower depending on whether the conversion needs to be done, right ? Or is this automagically fixed with the first save ?

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Henrik Sperre Johansen

On 01.10.2012 14:43, Sven Van Caekenberghe wrote: > On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> Endianness in header depends on the platform the image was saved on. >> Which for all practical purposes, means little-endian these days. > How can an image be cross-platform, but the header not ? > > > -- > Sven Van Caekenberghe > http://stfx.eu > Smalltalk is the Red Pill > IIRC, because the VM checks which endianness the header is written in (by comparing to known image formats in big and little-endian format), and then triggering a swap of all pointers/variableWord objects when loading the image on a differing platform. Cheers, Henry

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Igor Stasenko

On 1 October 2012 14:43, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > >> Endianness in header depends on the platform the image was saved on. >> Which for all practical purposes, means little-endian these days. > > How can an image be cross-platform, but the header not ? > Just another artifact of incremental development, i guess. I don't know what reasoning was behind this, but IMO, defining header to have fixed byte order would be much better. Of course, it is always easier to think backwards. However, things like IP header, were existed before squeak image format. So, again, i am clueless why guys decided that doing like it's done is a Good Idea (tm). -- Best regards, Igor Stasenko.

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Sven Van Caekenberghe

On 01 Oct 2012, at 14:01, Henrik Sperre Johansen <henrik.s.johansen(a)veloxit.no> wrote: > Endianness in header depends on the platform the image was saved on. > Which for all practical purposes, means little-endian these days. How can an image be cross-platform, but the header not ? -- Sven Van Caekenberghe http://stfx.eu Smalltalk is the Red Pill

Oct. 1, 2012

Re: [Pharo-project] detect if image was saved with Cog

by Max Leske

Thanks for the help guys, I think I have what I need. @Stef Christoph will get in touch with you. Cheers, Max On 01.10.2012, at 14:03, Igor Stasenko <siguctua(a)gmail.com> wrote: > On 1 October 2012 13:52, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >> Hi max >> >> I do not remember (may be doru) told me that he talked with christophe about the update of the SeasideHosting VM. >> I passed the mail to the esug-board because ESUG would certainly support action from this side. >> There is the work done by laurent laffont around smallHarbor is certainly to be considered as well as the hosting of the source >> modification in our git hub repository ( we could imagine that the hostingVM is part of our builds). >> > Is that was work about restricting file access/ports? > Sure it costs nothing in keeping a branch with little changes. > > >> may be we should see how we can help/what we can do about seaside hosting. >> I like the idea behind SmallHarbor: that people could deploy a way to manage their seaside app. >> >> Do you have more information from netstyle point of view? >> >> >> Stef >> >> >> On Oct 1, 2012, at 11:09 AM, Max Leske wrote: >> >>> Hi guys >>> >>> Is there a way to find out (from outside of the image), if an image has been saved with Cog? What I would like to do is something like this: >>> >>> $ head -20 myImage.image | grep <some expression> >>> >>> The use case for this is seaside hosting, where we only have squeak VMs to run images, so when an image is uploaded that was saved with Cog that image cause trouble. >>> Any ideas? >>> >>> Cheers, >>> Max >> > > > > -- > Best regards, > Igor Stasenko.

Oct. 1, 2012