Hi Christophe,

On Thu, Sep 17, 2015 at 7:09 AM, Christophe Demarey <Christophe.Demarey@inria.fr> wrote:

Hi again,

Does anyone know the rationale behind this?

declareEncodedCharSet: anEncodedCharSetOrLanguageEnvironmentClass atIndex: aNumber

        EncodedCharSets at: aNumber put: anEncodedCharSetOrLanguageEnvironmentClass

        "this method is used to modularize the old initialize method:
        EncodedCharSets at: 0+1 put: Unicode.
        EncodedCharSets at: 1+1 put: JISX0208.
        EncodedCharSets at: 2+1 put: GB2312.
        EncodedCharSets at: 3+1 put: KSX1001.
        EncodedCharSets at: 4+1 put: JISX0208.
        EncodedCharSets at: 5+1 put: JapaneseEnvironment.
        EncodedCharSets at: 6+1 put: SimplifiedChineseEnvironment.
        EncodedCharSets at: 7+1 put: KoreanEnvironment.
        EncodedCharSets at: 8+1 put: GB2312.
        EncodedCharSets at: 12+1 put: KSX1001.
        EncodedCharSets at: 13+1 put: GreekEnvironment.
        EncodedCharSets at: 14+1 put: Latin2Environment.
        EncodedCharSets at: 15+1 put: RussianEnvironment.
        EncodedCharSets at: 17+1 put: Latin9Environment.
        EncodedCharSets at: 256 put: Unicode.

        "

what Henrik says is correct.  Here's the relevant definition in Character:

Character>>leadingChar

"Answer the value of the 8 highest bits which is used to identify the language.

This is mostly used for east asian languages CJKV as a workaround against unicode han-unification."

^ self asInteger bitShift: -22

i.e. the top 8 bytes of the leading character in a string is (was?) used to index EncodedCharSets to determine what language the string is in.

_,,,^..^,,,_

best, Eliot