- Pharo-users - lists.pharo.org

Re: [Pharo-users] Unicode support in Pharo

by Sven Van Caekenberghe

On 26 Sep 2014, at 23:22, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: > Pasting a Greek string in a workspace shows hieroglyphs (editors? morph?), but GT Inspector display is ok. That is a font issue. Check your font settings in the Settings Browser. Like I mentioned in the chapter, you could try to use 'Arial Unicode MS'. I would guess you use Source Code Pro for code panes (like the workspace) and Open Sans for the rest. Source Code Pro has limited Unicode support. GT probably uses only Open Sans.

Sept. 26, 2014

Re: [Pharo-users] Unicode support in Pharo

by Alain Rastoul

Le 26/09/2014 23:22, Alain Rastoul a Ã©crit : > Le 26/09/2014 20:47, stepharo a Ã©crit : >> I'm not expert and I would like to know what people think. >> But I think that we should consider >> >> - the impact of spur new object format. I would like to have >> unicode and clean the leadChar >> >> Stef >> >> > Just to start a new thread about that, because it deserves it > and also some people might appreciate :) > (was in: "Ridiculous we are"). > > I may not be the most qualified to state on that but I give my 2c > (this is a community isn't it ? please do not flame...) > > As Sven said, the situation is not so bad (quite good in fact): Pharo > has true unicode built-in support with WideString and encoders (great > job, IMHO this part of Zn components should be in the base system > or in an internationalization package not hidden in an htpp components > package). > - encoding support has to be in the system : ZnEncoders(?) for utf8, > unicode and other encoders for locale support because of existing system > basis, web (utf8) and internationalization. > - this will be transparent: Smalltalk is a typeless langage, one do not > have to specify String type (ByteString or WideString) > - Spur new character encoding is very interesting (in alpha stage now) > and will be transparent for the same reason (though I'm still wondering > of the 32bits/64 bits encoding on a 64 bits vm?). > - ByteString is needed for most european/occidental people who don't > care about internationalization and should stay because Pharo is an > european/english based system , also not to break existing code > (again will be transparent to most users for same reason). > - some parts of the system seems to be not "WideString/unicode aware" : > Pasting a Greek string in a workspace shows hieroglyphs (editors? > morph?), but GT Inspector display is ok. > Font plugin seems to require little work : uses fopen instead of _wfopen > (utf16 version) on windows and needs utf8 convertion on unix. > may be a check with File plugin (and other system related plugins) is > also needed? > > In short, some little additionnal work but a very good basis. > (but what is leadChar about?) > > Regards, > > Alain > > > This was not a new thread indeed, I did "respond to group" I am ridiculous ... hahaha :)

Sept. 26, 2014

Unicode support in Pharo

by Alain Rastoul

Le 26/09/2014 20:47, stepharo a Ã©crit : > I'm not expert and I would like to know what people think. > But I think that we should consider > > - the impact of spur new object format. I would like to have > unicode and clean the leadChar > > Stef > > Just to start a new thread about that, because it deserves it and also some people might appreciate :) (was in: "Ridiculous we are"). I may not be the most qualified to state on that but I give my 2c (this is a community isn't it ? please do not flame...) As Sven said, the situation is not so bad (quite good in fact): Pharo has true unicode built-in support with WideString and encoders (great job, IMHO this part of Zn components should be in the base system or in an internationalization package not hidden in an htpp components package). - encoding support has to be in the system : ZnEncoders(?) for utf8, unicode and other encoders for locale support because of existing system basis, web (utf8) and internationalization. - this will be transparent: Smalltalk is a typeless langage, one do not have to specify String type (ByteString or WideString) - Spur new character encoding is very interesting (in alpha stage now) and will be transparent for the same reason (though I'm still wondering of the 32bits/64 bits encoding on a 64 bits vm?). - ByteString is needed for most european/occidental people who don't care about internationalization and should stay because Pharo is an european/english based system , also not to break existing code (again will be transparent to most users for same reason). - some parts of the system seems to be not "WideString/unicode aware" : Pasting a Greek string in a workspace shows hieroglyphs (editors? morph?), but GT Inspector display is ok. Font plugin seems to require little work : uses fopen instead of _wfopen (utf16 version) on windows and needs utf8 convertion on unix. may be a check with File plugin (and other system related plugins) is also needed? In short, some little additionnal work but a very good basis. (but what is leadChar about?) Regards, Alain

Sept. 26, 2014

Re: [Pharo-users] Hash values

by Alexandre Bergel

Yeah!!! That would be really cool! Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. On Sep 26, 2014, at 4:02 PM, stepharo <stepharo(a)free.fr> wrote: > >> Hi! >> >> Alejandro and I are working on a new profiling technique to identify redundant objects. >> >> Weâve heard there are some effort on increasing the range of identity hash values. Currently, it goes up to 4096, which is not much. Can someone tell us about this? > Check the spur format on clement's blog. If I remember we will have 12 bits for hashes soon. >> >> Also, I discovered that -100 hash = -100. Does this make sense? >> >> Cheers, >> Alexandre >> -- >> _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: >> Alexandre Bergel http://www.bergel.eu >> ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. >> >> >> >

Sept. 26, 2014

Re: [Pharo-users] Ridiculous we are

by Hilaire

Le 26/09/2014 21:00, phil(a)highoctane.be a Ã©crit : > I'd love another title for this thread. > > It depresses me. Yes, me too. Hilaire -- Dr. Geo - http://drgeo.eu iStoa - http://istoa.drgeo.eu

Sept. 26, 2014

Re: [Pharo-users] Hash values

by stepharo

> Hi! > > Alejandro and I are working on a new profiling technique to identify > redundant objects. > > Weâve heard there are some effort on increasing the range of identity > hash values. Currently, it goes up to 4096, which is not much. Can > someone tell us about this? Check the spur format on clement's blog. If I remember we will have 12 bits for hashes soon. > > Also, I discovered that -100 hash = -100. Does this make sense? > > Cheers, > Alexandre > -- > _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: > Alexandre Bergel http://www.bergel.eu > ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > > >

Sept. 26, 2014

Re: [Pharo-users] Ridiculous we are

by phil＠highoctane.be

I'd love another title for this thread. It depresses me. Phil On Fri, Sep 26, 2014 at 8:48 PM, stepharo <stepharo(a)free.fr> wrote: > Sven I love this chapter. > I will read it calmly now. > > Stef > > On 25/9/14 07:23, Sven Van Caekenberghe wrote: > >> On 25 Sep 2014, at 01:04, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: >> >> Le 25/09/2014 00:06, Sven Van Caekenberghe a Ã©crit : >>> >>>> Alain, >>>> The character encoding situation in Pharo is pretty good actually. The >>>> only problem is that there is some old school code left that encodes >>>> strings into strings, but today you can easily write much better and >>>> conceptually correct code. >>>> >>>> You could have a look at this draft chapter of the upcoming 'Enterprise >>>> Pharo' book that I am currently writing: >>>> >>>> http://stfx.eu/EnterprisePharo/Zinc-Encoding-Meta/ >>>> >>>> Concerning file system paths, FilePathEncoder and FilePluginPrimitives >>>> already do the right thing. >>>> >>>> Now, your idea about using UTF-8 to represent internal Strings is >>>> something that has been discussed before and in many other languages as >>>> well. The short answer is that due to it being variable length, the >>>> inefficiency is (probably) just too high. Simple indexed access becomes a >>>> problem, let alone more complex string manipulations. I am not saying that >>>> it cannot be done, I think it is just not worth the trouble. The current >>>> solution in Pharo with ByteString and WideString is quite nice (check the >>>> chapter I mentioned before). >>>> >>>> Sven >>>> >>>> Very interesting ! >>> It seems that most of what I was saying is already here :) >>> I was not saying that Pharo should use utf8 (I mentionned utf8 because >>> it is a standard, but I find the variable length encoding very weird), I >>> was rather talking of using WideString in UTF 16 or 32 and that's done. >>> I saw asWideString but didn't know about automatic convertion or >>> codepoint selector and internal wide string support. >>> Does it means that Pharo Greek users (for example) use WideString for >>> Strings without having to specify it or make explicit convertions (except >>> of course when dealing with bytes if they want to) ? >>> If yes, very good, job is almost done :) >>> (personnally I would also deprecate ByteString, and get rid of it, just >>> my opinion). >>> Thanks for the link, another good chapter . >>> >>> Regards, >>> >>> Alain >>> >> Yes, the Greek users won't notice a difference, it is all transparent. >> ByteString is important because it is an optimalization of the most common >> case. As a normal user you should only think of abstract Strings and never >> use #asByteString (but use proper encoding). >> >> Feedback on the chapter is always welcome. >> >> Sven >> >> > > >

Sept. 26, 2014

Re: [Pharo-users] Ridiculous we are

by stepharo

Sven I love this chapter. I will read it calmly now. Stef On 25/9/14 07:23, Sven Van Caekenberghe wrote: > On 25 Sep 2014, at 01:04, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: > >> Le 25/09/2014 00:06, Sven Van Caekenberghe a Ã©crit : >>> Alain, >>> The character encoding situation in Pharo is pretty good actually. The only problem is that there is some old school code left that encodes strings into strings, but today you can easily write much better and conceptually correct code. >>> >>> You could have a look at this draft chapter of the upcoming 'Enterprise Pharo' book that I am currently writing: >>> >>> http://stfx.eu/EnterprisePharo/Zinc-Encoding-Meta/ >>> >>> Concerning file system paths, FilePathEncoder and FilePluginPrimitives already do the right thing. >>> >>> Now, your idea about using UTF-8 to represent internal Strings is something that has been discussed before and in many other languages as well. The short answer is that due to it being variable length, the inefficiency is (probably) just too high. Simple indexed access becomes a problem, let alone more complex string manipulations. I am not saying that it cannot be done, I think it is just not worth the trouble. The current solution in Pharo with ByteString and WideString is quite nice (check the chapter I mentioned before). >>> >>> Sven >>> >> Very interesting ! >> It seems that most of what I was saying is already here :) >> I was not saying that Pharo should use utf8 (I mentionned utf8 because it is a standard, but I find the variable length encoding very weird), I was rather talking of using WideString in UTF 16 or 32 and that's done. >> I saw asWideString but didn't know about automatic convertion or codepoint selector and internal wide string support. >> Does it means that Pharo Greek users (for example) use WideString for Strings without having to specify it or make explicit convertions (except of course when dealing with bytes if they want to) ? >> If yes, very good, job is almost done :) >> (personnally I would also deprecate ByteString, and get rid of it, just my opinion). >> Thanks for the link, another good chapter . >> >> Regards, >> >> Alain > Yes, the Greek users won't notice a difference, it is all transparent. ByteString is important because it is an optimalization of the most common case. As a normal user you should only think of abstract Strings and never use #asByteString (but use proper encoding). > > Feedback on the chapter is always welcome. > > Sven >

Sept. 26, 2014

Re: [Pharo-users] Ridiculous we are

by stepharo

I'm not expert and I would like to know what people think. But I think that we should consider - the impact of spur new object format. I would like to have unicode and clean the leadChar Stef

Sept. 26, 2014

Hash values

by Alexandre Bergel

Hi! Alejandro and I are working on a new profiling technique to identify redundant objects. Weâve heard there are some effort on increasing the range of identity hash values. Currently, it goes up to 4096, which is not much. Can someone tell us about this? Also, I discovered that -100 hash = -100. Does this make sense? Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

Sept. 26, 2014