[Pharo-project] String>>#base64Encoded
Hi, It took me quite some time today to figure this out and it might be relevant for others too: it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length. This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64: Sven
On Mar 21, 2011, at 9:42 PM, Sven Van Caekenberghe wrote:
Hi,
It took me quite some time today to figure this out and it might be relevant for others too:
it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length.
when would it makes sense?
This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false
Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64:
Sven
On 21 Mar 2011, at 21:52, Stéphane Ducasse wrote:
On Mar 21, 2011, at 9:42 PM, Sven Van Caekenberghe wrote:
Hi,
It took me quite some time today to figure this out and it might be relevant for others too:
it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length.
when would it makes sense?
Base64 encoding ( http://en.wikipedia.org/wiki/Base64 ) is a techique to encode binary data in ASCII strings using a radix 64 representation. It came from the early days of the internet where is was used to encode binary stuff in mail messages (part of MIME encoding). For these usages, line breaks make sense, since some email client break lines themselves, and by preformatting you prevent that, for example: (Base64MimeConverter mimeEncode: (ZnClient get: 'http://www.pharo-project.org/favicon.ico') entity readStream multiLine: true) contents 'AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAABMLAAATCwAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0K1+/+2tF//lpRj/0J1F/86x kP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQso7/ 1Zw1/+ekF//koBf/zZdF/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAADLrIr/yI05/9SLG//UvKP/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAA3s2//8qrjf8AAAAAAAAAAAAAAADClmr/28m4/wAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACpayz/rH1S/9K9q/8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxKeP/4lWLv9+Tiz/oXtc/97P xf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADYxrj/ oX1j/2RBNP+QbFT/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAC6m4H/1MGz/+7m4v/Ltqb/tJR3/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAsJWF/1tHVf+qkIH/4tTL/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAANG8rf9GPGj/IiJn/19MYP/Mt6n/AAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANXEuP9NQGj/IiiB/youf/8NFnb/Qztx/9PC t/8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADczcL/mIGA/y8ziP8nMZj/ GSGF/4l2hP/Tw7n/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMy4 q/8pMZj/JzKi/yUxov+znJb/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAADezsT/d3Om/1dt0f9cZrz/v6mf/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJqapP+i3f//iazZ/9/Pwv8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADBqIv/lYps/7Caev8AAAAAAAAAAAAA AAAAAAAA/g8AAP8H8L//wwAA/nMAAP8fNcL/BwAA/4cAAP8HAAD/DwAA/wc4FP4DAAD+AwAA /wf///8H////hzp0/48AAA==' Base64 encoding is also used a lot in cryptography to encode binary data like hashes or even for silly obfuscation. For example, in HTTP, Basic Authentication is a defined as a HTTP header called 'Authorization' with as value 'Basic' <space> <credentials> where credentials is obtained by Base64 encoding <username>:<password>. ZnRequest>>#setBasicAuthenticationUsername: username password: password self headers at: 'Authorization' put: 'Basic ', (ZnUtils encodeBase64: (username, ':', password)) ZnUtils class>>#encodeBase64: string ^ (Base64MimeConverter mimeEncode: string readStream multiLine: false) contents Here you don't want line breaks since it would break the common assumption that a header is a single line (technically, multiline headers do exist, but that is another story). Now, as long as the input string is below 55 characters in length, no line breaks will be introduced. In code (assuming no line line breaks will be introduced by our email clients or the mailing list software inbetween; re-execute in a workspace if necessary): (Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: true) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB' (Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: false) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB' (Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: true) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB QQ==' (Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: false) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQQ==' Please note, this is not a bug, just a gotcha that can go unnoticed for quite some time. Sven PS: Base64 decoding is clever enough to transparently deal with line breaks.
This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false
Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64:
Sven
Thanks may be we should record this nice explanation in the help or something like that. On Mar 22, 2011, at 7:45 AM, Sven Van Caekenberghe wrote:
On 21 Mar 2011, at 21:52, Stéphane Ducasse wrote:
On Mar 21, 2011, at 9:42 PM, Sven Van Caekenberghe wrote:
Hi,
It took me quite some time today to figure this out and it might be relevant for others too:
it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length.
when would it makes sense?
Base64 encoding ( http://en.wikipedia.org/wiki/Base64 ) is a techique to encode binary data in ASCII strings using a radix 64 representation. It came from the early days of the internet where is was used to encode binary stuff in mail messages (part of MIME encoding). For these usages, line breaks make sense, since some email client break lines themselves, and by preformatting you prevent that, for example:
(Base64MimeConverter mimeEncode: (ZnClient get: 'http://www.pharo-project.org/favicon.ico') entity readStream multiLine: true) contents
'AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAABMLAAATCwAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0K1+/+2tF//lpRj/0J1F/86x kP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQso7/ 1Zw1/+ekF//koBf/zZdF/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAADLrIr/yI05/9SLG//UvKP/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAA3s2//8qrjf8AAAAAAAAAAAAAAADClmr/28m4/wAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACpayz/rH1S/9K9q/8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxKeP/4lWLv9+Tiz/oXtc/97P xf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADYxrj/ oX1j/2RBNP+QbFT/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAC6m4H/1MGz/+7m4v/Ltqb/tJR3/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAsJWF/1tHVf+qkIH/4tTL/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAANG8rf9GPGj/IiJn/19MYP/Mt6n/AAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANXEuP9NQGj/IiiB/youf/8NFnb/Qztx/9PC t/8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADczcL/mIGA/y8ziP8nMZj/ GSGF/4l2hP/Tw7n/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMy4 q/8pMZj/JzKi/yUxov+znJb/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAADezsT/d3Om/1dt0f9cZrz/v6mf/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJqapP+i3f//iazZ/9/Pwv8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADBqIv/lYps/7Caev8AAAAAAAAAAAAA AAAAAAAA/g8AAP8H8L//wwAA/nMAAP8fNcL/BwAA/4cAAP8HAAD/DwAA/wc4FP4DAAD+AwAA /wf///8H////hzp0/48AAA=='
Base64 encoding is also used a lot in cryptography to encode binary data like hashes or even for silly obfuscation. For example, in HTTP, Basic Authentication is a defined as a HTTP header called 'Authorization' with as value 'Basic' <space> <credentials> where credentials is obtained by Base64 encoding <username>:<password>.
ZnRequest>>#setBasicAuthenticationUsername: username password: password self headers at: 'Authorization' put: 'Basic ', (ZnUtils encodeBase64: (username, ':', password))
ZnUtils class>>#encodeBase64: string ^ (Base64MimeConverter mimeEncode: string readStream multiLine: false) contents
Here you don't want line breaks since it would break the common assumption that a header is a single line (technically, multiline headers do exist, but that is another story).
Now, as long as the input string is below 55 characters in length, no line breaks will be introduced. In code (assuming no line line breaks will be introduced by our email clients or the mailing list software inbetween; re-execute in a workspace if necessary):
(Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: true) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB'
(Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: false) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB'
(Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: true) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB QQ=='
(Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: false) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQQ=='
Please note, this is not a bug, just a gotcha that can go unnoticed for quite some time.
Sven
PS: Base64 decoding is clever enough to transparently deal with line breaks.
This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false
Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64:
Sven
In other language environments I have seen base64 encoders where the line break is optional, or where a maximum line length can be specified. In Smalltalk it might look line: String>>#base64Encoded String>>#base64EncodedWithLineBreaks or String>>#base64EncodedMaxLineLength: TF On Mon, Mar 21, 2011 at 11:45 PM, Sven Van Caekenberghe <sven@beta9.be>wrote:
On 21 Mar 2011, at 21:52, Stéphane Ducasse wrote:
On Mar 21, 2011, at 9:42 PM, Sven Van Caekenberghe wrote:
Hi,
It took me quite some time today to figure this out and it might be relevant for others too:
it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length.
when would it makes sense?
Base64 encoding ( http://en.wikipedia.org/wiki/Base64 ) is a techique to encode binary data in ASCII strings using a radix 64 representation. It came from the early days of the internet where is was used to encode binary stuff in mail messages (part of MIME encoding). For these usages, line breaks make sense, since some email client break lines themselves, and by preformatting you prevent that, for example:
(Base64MimeConverter mimeEncode: (ZnClient get: ' http://www.pharo-project.org/favicon.ico') entity readStream multiLine: true) contents
'AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAABMLAAATCwAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0K1+/+2tF//lpRj/0J1F/86x kP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQso7/ 1Zw1/+ekF//koBf/zZdF/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAADLrIr/yI05/9SLG//UvKP/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAA3s2//8qrjf8AAAAAAAAAAAAAAADClmr/28m4/wAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACpayz/rH1S/9K9q/8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxKeP/4lWLv9+Tiz/oXtc/97P xf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADYxrj/ oX1j/2RBNP+QbFT/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAC6m4H/1MGz/+7m4v/Ltqb/tJR3/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAsJWF/1tHVf+qkIH/4tTL/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAANG8rf9GPGj/IiJn/19MYP/Mt6n/AAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANXEuP9NQGj/IiiB/youf/8NFnb/Qztx/9PC t/8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADczcL/mIGA/y8ziP8nMZj/ GSGF/4l2hP/Tw7n/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMy4 q/8pMZj/JzKi/yUxov+znJb/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAADezsT/d3Om/1dt0f9cZrz/v6mf/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJqapP+i3f//iazZ/9/Pwv8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADBqIv/lYps/7Caev8AAAAAAAAAAAAA AAAAAAAA/g8AAP8H8L//wwAA/nMAAP8fNcL/BwAA/4cAAP8HAAD/DwAA/wc4FP4DAAD+AwAA /wf///8H////hzp0/48AAA=='
Base64 encoding is also used a lot in cryptography to encode binary data like hashes or even for silly obfuscation. For example, in HTTP, Basic Authentication is a defined as a HTTP header called 'Authorization' with as value 'Basic' <space> <credentials> where credentials is obtained by Base64 encoding <username>:<password>.
ZnRequest>>#setBasicAuthenticationUsername: username password: password self headers at: 'Authorization' put: 'Basic ', (ZnUtils encodeBase64: (username, ':', password))
ZnUtils class>>#encodeBase64: string ^ (Base64MimeConverter mimeEncode: string readStream multiLine: false) contents
Here you don't want line breaks since it would break the common assumption that a header is a single line (technically, multiline headers do exist, but that is another story).
Now, as long as the input string is below 55 characters in length, no line breaks will be introduced. In code (assuming no line line breaks will be introduced by our email clients or the mailing list software inbetween; re-execute in a workspace if necessary):
(Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: true) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB'
(Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: false) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB'
(Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: true) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB QQ=='
(Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: false) contents
'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQQ=='
Please note, this is not a bug, just a gotcha that can go unnoticed for quite some time.
Sven
PS: Base64 decoding is clever enough to transparently deal with line breaks.
This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false
Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64:
Sven
participants (3)
-
Stéphane Ducasse -
Sven Van Caekenberghe -
Tony Fleig