On 12 Sep 2014, at 15:10, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Hi,
ZnCharacterEncoder and its subclasses in the package Zinc-Character-Encoding-Core (a standard part of Pharo) have the following features:
- proper Unicode Character/String to/from byte(s) encoding/decoding - calculation of encoding size per Character/String - proper handling of holes in tables with exceptions (strict/lenient) - properly backing up one character on a binary stream - simple, efficient, documented - convenience API - optimisations for some encodings
Together, they support the following encodings:
- utf-8 - utf-16 (big/little endian, bom detection) - null - iso-8859-1 (latin-1) - iso-8859-2 (latin-2) - mac-roman - cp1250 - cp1252 - cp1253 - koi8r - iso-8859-7 - iso-8859-15
The question is, what other encodings do you want/need ?
There is already a request to add CP1251.
I decided to add a comprehensive set of encodings, based on http://encoding.spec.whatwg.org/#legacy-single-byte-encodings (a list which more or less corresponds to the encodings list supported by popular browsers). The mappings themselves are still taken from http://unicode.org/Public/MAPPINGS So now Pharo (Zinc-Character-Encoding-Core) supports the following byte encodings: ISO/IEC 8859 1, 2, 3, 4, 5, 6, 7, 8, 10, 13, 14, 15 and 16, Windows Codepages 866, 874, 1250, 1251, 1252, 1253, 1253, 1254, 1255, 1256, 1257, 1258, KOI8 R & U as well as Mac Roman & Cyrillic - each of these with a number of aliases like latin1, latin2, latin3, latin4, latin5, latin6, cyrillic, arabic, greek and hebrew. Furthermore, identifier mapping has been made more liberal (case and separator insensitive). For now, this is in Zinc #bleedingEdge
New byte encoders are easy to add, based on the official Unicode mappings that can be found here: http://unicode.org/Public/MAPPINGS (these are automagically parsed).
Please help us make Pharo more international friendly.
Sven