On 28.08.2012 23:11, Igor Stasenko wrote:
On 28 August 2012 14:45, Henrik Sperre Johansen <henrik.s.johansen@veloxit.no> wrote:
On 28.08.2012 14:35, Igor Stasenko wrote:
or it was like that from the birth??
arrrrgghhhh...
| stream | stream := WriteStream on: (ByteArray new: 100).
UTF8TextConverter new nextPut: (Character value: 129 ) toStream: stream.
stream contents #[129]
This is WRONG! RTFM, about utf8 encoding, please! :)
---------
nextPut: aCharacter toStream: aStream | leadingChar nBytes mask shift ucs2code | aStream isBinary ifTrue: [^aCharacter storeBinaryOn: aStream].
in my case, stream is binary, so it goes directly to #storeBinaryOn:
storeBinaryOn: aStream "Store the receiver on a binary (file) stream" value < 256 ifTrue:[aStream basicNextPut: value] ifFalse:[aStream nextInt32Put: value].
This is not even close to UTF8. If character code is less than 256, it will store a single byte (wtf?), and if more than that, it will store 32-bit integer value in big-endian order (wtf raisedToPower: 2)..
i wonder , for what purpose we actually having this code path? this stuff is completely useless. according to implementation of storeBinaryOn: there's no way how you can read the same character value back. because it can be 1 byte or 4 bytes.. but you simply cannot determine which one. this is one of the reasons we using utf8 encoding, btw ;)
It's been this way from the start. so, it was broken from the start? the point is, that i am talking specifically about UTF8Converter... i don't care about any grade of polymorphism/megamorphism/whatevermorphism inherited from TextConverters.. but by simplest logic, my ignorant expectation was that UTF8Converter should convert its input into utf8 form or die trying.. but not something which not even remotely related to utf8.. wtf? Because if i don't need it to be converted to utf8, then why i would bother using it in a first place?
You want to sanely encode to in-memory bytearrays, use the Zn converters. You want to conveniently switch between putting binary and encoded contents to a filestream (and do so efficiently), use the TextConverters.
conveniently? are you joking? :) conveniently is when i say things once, either:
newstream := converter on: stream. x timesRepeat: [ newstream nextPut: oneElement ]. or
stream addConverter: myconverter. x timesRepeat: [ stream nextPut: oneElement ].
but instead i forced to use it like:
x timesRepeat: [ converter nextPut: oneElement onStream: stream ]
it is neither convenient, nor optimal, because you operating with three entities in a loop: - stream, converter and element, while you really need only two.
But aside of that, is there a reason why we must have 2 implementations of very same thing in our 'clean' system? If we have that , i think, we should stop calling it clean :)
Sven told, why he implemented his own - just because the existing one is inherently broken (well, it was more polite form if i remember).
Maybe Sven is too polite to put it straight or don't wants to shake the very foundations of our beautiful text conversion ecosystem.. But i don't mind to say it: TextConverters is piece of crap and we need to throw it out and replaced with something better or review/refactor/(choose your path)..
So if some of you feel offended, i am sorry. You're a good guy/girl, who produced this code. The problem is that code doesn't follows "The principle of a least wtf". :)
I originally wrote a detailed response, but gave up after awhile. Let me just say that, at it's core, I don't think it's so much a problem of believing the current situation is good, but that the inextricable link between converters and filestreams, and how their extensive usage limits the space for actual improvements within the space of backwards compatability. (for a related example, there are multiple codepaths that rely on #binary not actually doing any encoding) For a definition of better that means "less wtf's per line of code read/spent trying to use these things", I for one don't see a way for big improvements that does not involve breaking huge amounts of old code. At that point, a clean break to a more sensible architecture like XTreams, with clear separation of streaming/encoding responsibilites becomes a more attractive alternative in the first place. Creating backwards-compatability protols to a new framework might help somewhat, but it's still a massive undertaking which would cause huge amounts of pain to users. Sort of like the Settings-switch really, but at a much larger scale. And so we keep on encountering a few wtf's once in a while, learn to deal with them, and then move on with our lives while hoping someone else picks up the torch :) Cheers, Henry