[Pharo-project] Encodings, Charsets, converters... streams?
Hi guys, I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So, - We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together. What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it. - MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )... What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package? - Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says: *"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain. On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode. In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server.*" So, does someone use it and see value on keeping it? Guille
BTW, in case people keep quiet, I'll start opening issues :) On Mon, Jun 25, 2012 at 4:02 PM, Guillermo Polito <guillermopolito@gmail.com
wrote:
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
*"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server.*"
So, does someone use it and see value on keeping it?
Guille
http://code.google.com/p/pharo/issues/detail?id=6154 http://code.google.com/p/pharo/issues/detail?id=6155 On Mon, Jun 25, 2012 at 4:08 PM, Guillermo Polito <guillermopolito@gmail.com
wrote:
BTW, in case people keep quiet, I'll start opening issues :)
On Mon, Jun 25, 2012 at 4:02 PM, Guillermo Polito < guillermopolito@gmail.com> wrote:
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
*"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server.*"
So, does someone use it and see value on keeping it?
Guille
Hello. What is responsibility of environments (russian, japanese) classes. Why its needed? Why converter classes not enough? For example, I can write russian text at workspace without setting any specific russian environment. I only need specific fonts for this. 2012/6/25 Guillermo Polito <guillermopolito@gmail.com>
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
*"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server.*"
So, does someone use it and see value on keeping it?
Guille
Hi Denis, On Mon, Jun 25, 2012 at 6:06 PM, Denis Kudriashov <dionisiydk@gmail.com>wrote:
Hello.
What is responsibility of environments (russian, japanese) classes. Why its needed? Why converter classes not enough?
For example, I can write russian text at workspace without setting any specific russian environment. I only need specific fonts for this.
Without being an expert, what I can see is that LanguageEnvironments are a kind of facade/factory of Charset, Encoding, Locales, Fonts... So, Greek environment should provide you all the correct configuration to do greek stuff...
2012/6/25 Guillermo Polito <guillermopolito@gmail.com>
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
*"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server.*"
So, does someone use it and see value on keeping it?
Guille
On 25.06.2012 18:06, Denis Kudriashov wrote:
Hello.
What is responsibility of environments (russian, japanese) classes. Why its needed? Why converter classes not enough?
For example, I can write russian text at workspace without setting any specific russian environment. I only need specific fonts for this.
Novadays, they can be used to read number/date printing policies, decide which translation package to use, etc. Basically what you'd expect from an OS locale, but more limited in capabilities. The part linked to charsets and encodings seem to me an anachronism from when the OS's used different encodings based on the selected environment though. (so files would be read and written automatically with the correct (non-unicode) encoding, and potentially displayed using a different bitmap font through selection using leadingChar)
2012/6/25 Guillermo Polito <guillermopolito@gmail.com <mailto:guillermopolito@gmail.com>>
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
They are not neatly composed. Trying to merge them so the methods work with a file and bytearray/string backend equivalently would probably only make things even worse. Using a decorator would be hard, since the MultiByteFileStream method implementations use alot of its inherited behaviour. As for putting them in a different package, sure you can peddle crap around all you want, but that doesn't make it stink any less.
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
/"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server./"
So, does someone use it and see value on keeping it?
Sure, there is value in neatly integrating OS input methods for composed characters. Whether that belongs in a plugin of it's own, and whether what the plugin currently offers justifies its existence, is a different question, which I guess the removal was the answer to. :) To find any potential actual users, I guess you'd have to find a pharo app localized to the eastern hemisphere and have it ask their linux users if they appreciate it/if it works. That, or a developer from the same area who codes in his native language (which will be hard I think, given arbitrary tool constraints wrt message names etc.) Cheers, Henry
ok but how can we build a concrete roadmap. What are the steps that we can take and perform one by one. Stef On Jun 27, 2012, at 3:11 PM, Henrik Sperre Johansen wrote:
On 25.06.2012 18:06, Denis Kudriashov wrote:
Hello.
What is responsibility of environments (russian, japanese) classes. Why its needed? Why converter classes not enough?
For example, I can write russian text at workspace without setting any specific russian environment. I only need specific fonts for this.
Novadays, they can be used to read number/date printing policies, decide which translation package to use, etc. Basically what you'd expect from an OS locale, but more limited in capabilities.
The part linked to charsets and encodings seem to me an anachronism from when the OS's used different encodings based on the selected environment though. (so files would be read and written automatically with the correct (non-unicode) encoding, and potentially displayed using a different bitmap font through selection using leadingChar)
2012/6/25 Guillermo Polito <guillermopolito@gmail.com> Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package? They are not neatly composed. Trying to merge them so the methods work with a file and bytearray/string backend equivalently would probably only make things even worse. Using a decorator would be hard, since the MultiByteFileStream method implementations use alot of its inherited behaviour.
As for putting them in a different package, sure you can peddle crap around all you want, but that doesn't make it stink any less.
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server."
So, does someone use it and see value on keeping it?
Sure, there is value in neatly integrating OS input methods for composed characters.
Whether that belongs in a plugin of it's own, and whether what the plugin currently offers justifies its existence, is a different question, which I guess the removal was the answer to. :)
To find any potential actual users, I guess you'd have to find a pharo app localized to the eastern hemisphere and have it ask their linux users if they appreciate it/if it works. That, or a developer from the same area who codes in his native language (which will be hard I think, given arbitrary tool constraints wrt message names etc.)
Cheers, Henry
I've spent 3 hours on trying to understand deeply the Multilingual mess, but I'm dying in the process... It mixes environments with charsets... Some charsets and converters are for really old OS versions... Sometimes it uses unicode, some times not... argggdsdsa On Wed, Jun 27, 2012 at 5:25 PM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
ok but how can we build a concrete roadmap. What are the steps that we can take and perform one by one.
Stef
On Jun 27, 2012, at 3:11 PM, Henrik Sperre Johansen wrote:
On 25.06.2012 18:06, Denis Kudriashov wrote:
Hello.
What is responsibility of environments (russian, japanese) classes. Why its needed? Why converter classes not enough?
For example, I can write russian text at workspace without setting any specific russian environment. I only need specific fonts for this.
Novadays, they can be used to read number/date printing policies, decide which translation package to use, etc. Basically what you'd expect from an OS locale, but more limited in capabilities.
The part linked to charsets and encodings seem to me an anachronism from when the OS's used different encodings based on the selected environment though. (so files would be read and written automatically with the correct (non-unicode) encoding, and potentially displayed using a different bitmap font through selection using leadingChar)
2012/6/25 Guillermo Polito <guillermopolito@gmail.com> Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package? They are not neatly composed. Trying to merge them so the methods work with a file and bytearray/string backend equivalently would probably only make things even worse. Using a decorator would be hard, since the MultiByteFileStream method implementations use alot of its inherited behaviour.
As for putting them in a different package, sure you can peddle crap around all you want, but that doesn't make it stink any less.
- Just curious. ImmPlugin is working or shipped with vms? Not for
macos, because there is no macos implementation. Then, the class comment says:
"On Windows, the plugin is not provided in the shipped VM's, and its
current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when
over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode
returned by the X Server, so whether this is actually relevant at all depends on the X Server."
So, does someone use it and see value on keeping it?
Sure, there is value in neatly integrating OS input methods for composed characters.
Whether that belongs in a plugin of it's own, and whether what the plugin currently offers justifies its existence, is a different question, which I guess the removal was the answer to. :)
To find any potential actual users, I guess you'd have to find a pharo app localized to the eastern hemisphere and have it ask their linux users if they appreciate it/if it works. That, or a developer from the same area who codes in his native language (which will be hard I think, given arbitrary tool constraints wrt message names etc.)
Cheers, Henry
On Jun 25, 2012, at 4:02 PM, Guillermo Polito wrote:
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
I would group per language.
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
I should say that I did not spend enough time on stream mess
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server."
So, does someone use it and see value on keeping it?
:) So lovely. What does Imm mean?
Guille
On Mon, Jun 25, 2012 at 7:28 PM, Stéphane Ducasse <stephane.ducasse@inria.fr
wrote:
On Jun 25, 2012, at 4:02 PM, Guillermo Polito wrote:
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
I would group per language.
In a first step, of course :). If we go deep, I could say that the triplicated hierarchy is smelling...
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
I should say that I did not spend enough time on stream mess
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server."
So, does someone use it and see value on keeping it?
:) So lovely. What does Imm mean?
Dunno :/
Guille
On Mon, Jun 25, 2012 at 7:58 PM, Guillermo Polito <guillermopolito@gmail.com
wrote:
On Mon, Jun 25, 2012 at 7:28 PM, Stéphane Ducasse < stephane.ducasse@inria.fr> wrote:
On Jun 25, 2012, at 4:02 PM, Guillermo Polito wrote:
Hi guys,
I'm trying to understand the Multilingual packages to get things a bit more reorganized. This way we can start to have a simpler and understandable kernel. So,
- We have all environments(greek, chinese, japanese, korean, russian...) in the same package. Each environment has it's associated charset, and text converter, which are in different packages. Also, each text converter has a table of conversions between unicode and the encoding. But all converters are together, and all charsets are together.
What about putting related things together? I mean, japanese converter with japanese environment with japanese charset in one package. Same with russian, korean, chinese... Does it make sense? This way we should be able to unload one of them if we do not really need it.
I would group per language.
In a first step, of course :). If we go deep, I could say that the triplicated hierarchy is smelling...
- MultiByteFileStream and MultiByteBinaryOrTextStream are inside Multilingual-TextConversion? Even, they do the same on top of a file or a memory stream of bytes (or they say so in the comment :^D )...
What about merging them? using a decorator? They have a lot in common... :/ Put them in a different package?
I should say that I did not spend enough time on stream mess
- Just curious. ImmPlugin is working or shipped with vms? Not for macos, because there is no macos implementation. Then, the class comment says:
"On Windows, the plugin is not provided in the shipped VM's, and its current whereabouts are uncertain.
On Unix, the only implemented behaviour is setting the position when over-the-spot precomposition of characters is the current mode.
In the VM, the mode is chosen to the first available valid mode returned by the X Server, so whether this is actually relevant at all depends on the X Server."
So, does someone use it and see value on keeping it?
:) So lovely. What does Imm mean?
Dunno :/
In VMMaker-Plugins there is no such plugin... I'm tempted :P
Guille
participants (4)
-
Denis Kudriashov -
Guillermo Polito -
Henrik Sperre Johansen -
Stéphane Ducasse