[Pharo-project] UTF support in Linux
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
UTF input (in VM and image) support in Pharo (or Squeak) is a must ! 2008/7/9 Stéphane Ducasse <stephane.ducasse@inria.fr>:
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Serge Stinckwich Smalltalkers do: [:it | All with: Class, (And love: it)] http://blog.doesnotunderstand.org/
On Wed, 2008-07-09 at 10:44 +0200, Serge Stinckwich wrote:
UTF input (in VM and image) support in Pharo (or Squeak) is a must !
What do you mean? There is support for UTF-8 and UTF-16 in the image. The image itself changed the internal character format to a modified Unicode (4 Bytes) Character. I assume that you don't need any conversion for UTF-32. The problem is not the support but to know what encoding the unicode has. If you have a Byte-Order-Mark this can be easy. Otherwise there is no way around than to transmit a character encoding along with the data or to use a convention for it. The etoys stuff seems like it can changes the input to something unicode but this is a convention that is manually adjusted. Do I understand something wrong? Norbert
2008/7/9 Stéphane Ducasse <stephane.ducasse@inria.fr>:
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
On Wed, 2008-07-09 at 11:18 +0200, Norbert Hartl wrote:
On Wed, 2008-07-09 at 10:44 +0200, Serge Stinckwich wrote:
UTF input (in VM and image) support in Pharo (or Squeak) is a must !
What do you mean? There is support for UTF-8 and UTF-16 in the image. The image itself changed the internal character format to a modified Unicode (4 Bytes) Character. I assume that you don't need any conversion for UTF-32.
The problem is not the support but to know what encoding the unicode s/unicode/input/
Norbert
has. If you have a Byte-Order-Mark this can be easy. Otherwise there is no way around than to transmit a character encoding along with the data or to use a convention for it. The etoys stuff seems like it can changes the input to something unicode but this is a convention that is manually adjusted.
Do I understand something wrong?
Norbert
2008/7/9 Stéphane Ducasse <stephane.ducasse@inria.fr>:
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Hi Norbert, i'm aware of the UTF infrastructure inside Squeak. I think i see recently some messages on squeak-dev about various problems with keyboard inputs. I have to dig in the squeak-dev mailing list in order to find the messages ... 2008/7/9 Norbert Hartl <norbert@hartl.name>:
On Wed, 2008-07-09 at 10:44 +0200, Serge Stinckwich wrote:
UTF input (in VM and image) support in Pharo (or Squeak) is a must !
What do you mean? There is support for UTF-8 and UTF-16 in the image. The image itself changed the internal character format to a modified Unicode (4 Bytes) Character. I assume that you don't need any conversion for UTF-32.
The problem is not the support but to know what encoding the unicode has. If you have a Byte-Order-Mark this can be easy. Otherwise there is no way around than to transmit a character encoding along with the data or to use a convention for it. The etoys stuff seems like it can changes the input to something unicode but this is a convention that is manually adjusted.
Do I understand something wrong?
Norbert
2008/7/9 Stéphane Ducasse <stephane.ducasse@inria.fr>:
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Serge Stinckwich Smalltalkers do: [:it | All with: Class, (And love: it)] http://blog.doesnotunderstand.org/
I think that fixing encodings is also part of mike push. I just saw these changesets and may be we should open tickets to see if we want to have a look. But encodings is a hell. Stef On Jul 9, 2008, at 11:29 AM, Serge Stinckwich wrote:
Hi Norbert, i'm aware of the UTF infrastructure inside Squeak. I think i see recently some messages on squeak-dev about various problems with keyboard inputs. I have to dig in the squeak-dev mailing list in order to find the messages ...
2008/7/9 Norbert Hartl <norbert@hartl.name>:
On Wed, 2008-07-09 at 10:44 +0200, Serge Stinckwich wrote:
UTF input (in VM and image) support in Pharo (or Squeak) is a must !
What do you mean? There is support for UTF-8 and UTF-16 in the image. The image itself changed the internal character format to a modified Unicode (4 Bytes) Character. I assume that you don't need any conversion for UTF-32.
The problem is not the support but to know what encoding the unicode has. If you have a Byte-Order-Mark this can be easy. Otherwise there is no way around than to transmit a character encoding along with the data or to use a convention for it. The etoys stuff seems like it can changes the input to something unicode but this is a convention that is manually adjusted.
Do I understand something wrong?
Norbert
2008/7/9 Stéphane Ducasse <stephane.ducasse@inria.fr>:
Begin forwarded message:
From: Bert Freudenberg <bert@freudenbergs.de> Date: July 9, 2008 12:20:52 AM CEDT To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org> Subject: Re: [Newbies] UTF support in Linux? Reply-To: "A friendly place to get answers to even the most basic questions about Squeak." <beginners@lists.squeakfoundation.org>
Am 08.07.2008 um 23:09 schrieb smith02243:
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
-- Serge Stinckwich Smalltalkers do: [:it | All with: Class, (And love: it)] http://blog.doesnotunderstand.org/
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
Stéphane Ducasse wrote:
I think that fixing encodings is also part of mike push. I just saw these changesets and may be we should open tickets to see if we want to have a look. But encodings is a hell.
Started looking at it. It is pretty scary in how many places raw characters are created without honoring any encodings etc... big question is: should everything be bakcward compatible with older, non-unicode supporting VMs? Right now we check against slot 7 if there is unicode available in the key event buffer, otherwise we go through the default encoder. Michael
On Jul 9, 2008, at 11:27 PM, Michael Rueger wrote:
Stéphane Ducasse wrote:
I think that fixing encodings is also part of mike push. I just saw these changesets and may be we should open tickets to see if we want to have a look. But encodings is a hell.
Started looking at it. It is pretty scary in how many places raw characters are created without honoring any encodings etc...
big question is: should everything be bakcward compatible with older, non-unicode supporting VMs? Right now we check against slot 7 if there is unicode available in the key event buffer, otherwise we go through the default encoder.
My gut feeling would say no. Else we will have again patches over patches. But now we can be sensible to argument that you can have. Stef
Michael
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
I have the impression that this is an important todo for the future pharo milestones. Stef
Hi all,
In the Linux VM, when I try to input, via paste or the right alt combination, non-ascii characters, like 'ö' or 'é' for example, Squeak merely shows 'o?' or 'e?' I researched into this and accordingly chose a font that supports UTF-8 and set the locale and VM command line options appropriately:
export LC_CTYPE="en_IE.UTF-8" locale-gen en_IE.UTF-8 (...which claims that locale is upto date...) sudo dpkg-reconfigure -plow locales (...which outputs en_IE.UTF-8... done among other things....) squeak -encoding UTF-8 -textenc UTF-8 -pathenc UTF-8 Squeak3.10.2-7179-basic.image
Now when I try to input the aforementioned characters I encounter a debug message: "Error subscript is out of bound."
However, I have set Squeak's language to Spanish and then German, and the characters display perfectly well in the menus. Although, the same cannot be said for the any files with non-ascii characters in the file dialogue. So, it seems it's an input problem.
Anyone with any ideas? Thanks.
This has only been addressed recently, and may not have found its way into a release yet.
You need the very latest Squeak VM (3.10-1 at least). It supports Unicode keyboard events (which are passed to the image as UTF32).
To make use of these events your image needs to be using UTF32InputInterpreter. Evaluate
ActiveHand keyboardInterpreter
to see the currently used one. It can be changed by patching Latin1Environment class>>inputInterpreterClass.
I don't know if this is in any official release yet, or scheduled for one.
Some relevant change sets from the Etoys image seem to be
http://tinlizzie.org/updates/etoys/updates/1713UnicodeInput1-yo.cs http://tinlizzie.org/updates/etoys/updates/1714UnicodeInput2-yo.cs http://tinlizzie.org/updates/etoys/updates/1925unixLatin1Input-yo.cs http://tinlizzie.org/updates/etoys/updates/1935unicodeFallback-bf.cs
- Bert -
_______________________________________________ Beginners mailing list Beginners@lists.squeakfoundation.org http://lists.squeakfoundation.org/mailman/listinfo/beginners
_______________________________________________ Pharo-project mailing list Pharo-project@lists.gforge.inria.fr http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
participants (4)
-
Michael Rueger -
Norbert Hartl -
Serge Stinckwich -
Stéphane Ducasse