On 28 August 2012 14:45, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 28.08.2012 14:35, Igor Stasenko wrote:
or it was like that from the birth??
arrrrgghhhh...
| stream | stream := WriteStream on: (ByteArray new: 100).
UTF8TextConverter new nextPut: (Character value: 129 ) toStream: stream.
stream contents #[129]
This is WRONG! RTFM, about utf8 encoding, please! :)
---------
nextPut: aCharacter toStream: aStream | leadingChar nBytes mask shift ucs2code | aStream isBinary ifTrue: [^aCharacter storeBinaryOn: aStream].
in my case, stream is binary, so it goes directly to #storeBinaryOn:
storeBinaryOn: aStream "Store the receiver on a binary (file) stream" value < 256 ifTrue:[aStream basicNextPut: value] ifFalse:[aStream nextInt32Put: value].
This is not even close to UTF8. If character code is less than 256, it will store a single byte (wtf?), and if more than that, it will store 32-bit integer value in big-endian order (wtf raisedToPower: 2)..
i wonder , for what purpose we actually having this code path? this stuff is completely useless. according to implementation of storeBinaryOn: there's no way how you can read the same character value back. because it can be 1 byte or 4 bytes.. but you simply cannot determine which one. this is one of the reasons we using utf8 encoding, btw ;)
It's been this way from the start.
so, it was broken from the start? the point is, that i am talking specifically about UTF8Converter... i don't care about any grade of polymorphism/megamorphism/whatevermorphism inherited from TextConverters.. but by simplest logic, my ignorant expectation was that UTF8Converter should convert its input into utf8 form or die trying.. but not something which not even remotely related to utf8.. wtf? Because if i don't need it to be converted to utf8, then why i would bother using it in a first place?
You want to sanely encode to in-memory bytearrays, use the Zn converters. You want to conveniently switch between putting binary and encoded contents to a filestream (and do so efficiently), use the TextConverters.
conveniently? are you joking? :) conveniently is when i say things once, either: newstream := converter on: stream. x timesRepeat: [ newstream nextPut: oneElement ]. or stream addConverter: myconverter. x timesRepeat: [ stream nextPut: oneElement ]. but instead i forced to use it like: x timesRepeat: [ converter nextPut: oneElement onStream: stream ] it is neither convenient, nor optimal, because you operating with three entities in a loop: - stream, converter and element, while you really need only two. But aside of that, is there a reason why we must have 2 implementations of very same thing in our 'clean' system? If we have that , i think, we should stop calling it clean :) Sven told, why he implemented his own - just because the existing one is inherently broken (well, it was more polite form if i remember). Maybe Sven is too polite to put it straight or don't wants to shake the very foundations of our beautiful text conversion ecosystem.. But i don't mind to say it: TextConverters is piece of crap and we need to throw it out and replaced with something better or review/refactor/(choose your path).. So if some of you feel offended, i am sorry. You're a good guy/girl, who produced this code. The problem is that code doesn't follows "The principle of a least wtf". :) -- Best regards, Igor Stasenko.