Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ? cheers -ben
On 10 Dec 2017, at 10:01, Ben Coman <btc@openinworld.com> wrote:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
Well Pharo itself of course ! (HMAC on: SHA256) key: (ByteArray new: 32); digestMessage: #[1 2 3]. SHA256 new hmac key: (ByteArray new: 32); digestMessage: #[1 2 3]. Sven PS: You might like this one too https://medium.com/concerning-pharo/the-code-behind-google-authenticator-9c5...
On 10 December 2017 at 18:23, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 10 Dec 2017, at 10:01, Ben Coman <btc@openinworld.com> wrote:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
Well Pharo itself of course !
(HMAC on: SHA256) key: (ByteArray new: 32); digestMessage: #[1 2 3].
SHA256 new hmac key: (ByteArray new: 32); digestMessage: #[1 2 3].
Thanks Sven. Its interesting to trace that through to put other stuff I've read about HMAC into perspective. However SHA256 != SHA512 which is a defined requirement of the site I'm accessing. What I understand from the trace is that the HMAC is generic regardless of size of SHA function and could remain in-Image while the SHA512 part could be supplied from outside the image. I could perhaps use the one from the OpenSSL library already included with Pharo. $ readelf -a ./pharo-vm/lib/pharo/5.0-201707201942/libssl.so.1.0.0 | grep 512 EVP_sha512 SHA512_Init SHA512_Update SHA512_Transform SHA512_Final where the "EVP function provide a high level interface to OpenSSL cryptographic functions." and I guess could be used similar to... ftp://188.44.46.157/Augustus/blatSrc/lib/hmac.c except I'm not sure how I'd use proceed without it taking any parameters... const EVP_MD *EVP_sha512(void); // include/openssl/evp.h The lower level functions could be used like... http://www.askyb.com/cpp/openssl-sha512-hashing-example-in-cpp/ https://github.com/openssl/openssl/blob/OpenSSL_1_1_0-stable/crypto/sha/sha5... unsigned char *SHA512(const unsigned char *d, size_t n, unsigned char *md); int SHA512_Init(SHA512_CTX *c); // include/openssl/sha.h int SHA512_Update(SHA512_CTX *c, const void *data, size_t len); int SHA512_Final(unsigned char *md, SHA512_CTX *c); void SHA512_Transform(SHA512_CTX *c, const unsigned char *data); Actually the lower level functions look easier from an FFI perspective. Other options I found... * http://forum.world.st/How-to-encrypt-a-password-td3933585.html#a3933778 but it seems to be NativeBoost rather than UFFI, and also is Linux only (which might not be an issue) * https://github.com/mygityf/cipher/blob/master/cipher/sha512.h https://github.com/mygityf/cipher/blob/master/cipher/sha512.c to compile into a (hopefully) cross platform shared library cheers -ben P.S. I learnt today that "SHA-512 is faster than SHA-256 on 64 bit machines (as they use 64 bit arithmetic internally)" https://stackoverflow.com/a/18083633
On 10 Dec 2017, at 17:46, Ben Coman <btc@openinworld.com> wrote:
Thanks Sven. Its interesting to trace that through to put other stuff I've read about HMAC into perspective. However SHA256 != SHA512 which is a defined requirement of the site I'm accessing.
I was too quick. There is also http://www.samadhiweb.com/blog/2017.02.18.shacrypt.html I prefer code written in Pharo, but if you need real performance, then native code will be needed. Are you sure SHA512 is not in the Cryptography package ?
-------- Original Message -------- Subject: Re: [Pharo-users] HMAC-SHA512 Local Time: December 10, 2017 1:20 PM UTC Time: December 10, 2017 6:20 PM From: sven@stfx.eu To: Any question about pharo is welcome <pharo-users@lists.pharo.org>
On 10 Dec 2017, at 17:46, Ben Coman btc@openinworld.com wrote: Thanks Sven. Its interesting to trace that through to put other stuff I've read about HMAC into perspective. However SHA256 != SHA512 which is a defined requirement of the site I'm accessing.
I was too quick.
There is also http://www.samadhiweb.com/blog/2017.02.18.shacrypt.html
I prefer code written in Pharo, but if you need real performance, then native code will be needed. Are you sure SHA512 is not in the Cryptography package ?
SHA512 is not in the Cryptography package, but it would be great to see it there, with an appropriate plugin, of course, for performance. I am toying with the idea of extending SSL to include TLS 1.3, and that would require SHA512, plus it would be great to keep the Cryptography package current. Adding TLS 1.3 would be a fair amount of work requiring Diffie-Hellman group extensions to ephemeral elliptic curves, in addition. I am unsure what symmetric ciphers are used by TLS 1.3 also. The advantage is that it is automatically cross-platform, even with plugin generation, such that Cryptography could be used on the big 3 as well as on ARM, Android and iOS. I'll keep dreaming about it.
On 11 December 2017 at 03:08, henry <henry@callistohouse.club> wrote:
-------- Original Message -------- Subject: Re: [Pharo-users] HMAC-SHA512 Local Time: December 10, 2017 1:20 PM UTC Time: December 10, 2017 6:20 PM From: sven@stfx.eu To: Any question about pharo is welcome <pharo-users@lists.pharo.org>
On 10 Dec 2017, at 17:46, Ben Coman btc@openinworld.com wrote: Thanks Sven. Its interesting to trace that through to put other stuff I've read about HMAC into perspective. However SHA256 != SHA512 which is a defined requirement of the site I'm accessing.
I was too quick.
There is also http://www.samadhiweb.com/blog/2017.02.18.shacrypt.html
I prefer code written in Pharo, but if you need real performance, then native code will be needed. Are you sure SHA512 is not in the Cryptography package ?
SHA512 is not in the Cryptography package, but it would be great to see it there, with an appropriate plugin, of course, for performance. I am toying with the idea of extending SSL to include TLS 1.3, and that would require SHA512, plus it would be great to keep the Cryptography package current. Adding TLS 1.3 would be a fair amount of work requiring Diffie-Hellman group extensions to ephemeral elliptic curves, in addition. I am unsure what symmetric ciphers are used by TLS 1.3 also. The advantage is that it is automatically cross-platform, even with plugin generation, such that Cryptography could be used on the big 3 as well as on ARM, Android and iOS. I'll keep dreaming about it.
Given that SSL is "so last century" [ https://www.polyglotdeveloper.com/timeline/2015-07-01-ssl-tls-timeline/] with security issues [Section 2.2 https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practic... ] and latency improvements in upcoming TLS1.3 [ https://www.keycdn.com/blog/tls-1-3] perhaps this would make a good bounty to be done outside the current planned work for engineering resources. There seem several potential resources available [ http://www.squeaksource.com/Cryptography.html] Could a plan be made to address TLS? cheers -ben
On Mon, Dec 11, 2017 at 12:46:59AM +0800, Ben Coman wrote:
where the "EVP function provide a high level interface to OpenSSL cryptographic functions."
Hi Ben, As it happened, over the weekend I implemented the Pharo wrappers for the EVP_DigestSign* and EVP_DigestVerify* APIs. It is straightforward to add the non-public key-using message digest functions, so I've just done it. Code snippets: | msg st c | msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := SHA256 hashMessage: msg asByteArray. c := LcEvpSHA256 new hashMessage: msg asByteArray. st = c => true Using Sven's example: | msg st c | key := ByteArray new: 32. msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := (HMAC on: SHA256) key: key; digestMessage: msg asByteArray. c := (HMAC on: LcEvpSHA256) key: key; digestMessage: msg asByteArray. st = c => true Code at http://smalltalkhub.com/#!/~PierceNg/OpenSSL-Pharo. I've also added tests for SHA256 similar to the snippets above. I will add SHA512 tests when I find the test vectors. In the next two weeks when I have some time off from work, I will attempt to move the repo to GH. Meanwhile, I've added you as a contributor to the STH repo if you'd like to hack on it. Pierce
On 11 December 2017 at 20:28, Pierce Ng <pierce@samadhiweb.com> wrote:
On Mon, Dec 11, 2017 at 12:46:59AM +0800, Ben Coman wrote:
where the "EVP function provide a high level interface to OpenSSL cryptographic functions."
Hi Ben,
As it happened, over the weekend I implemented the Pharo wrappers for the EVP_DigestSign* and EVP_DigestVerify* APIs. It is straightforward to add the non-public key-using message digest functions, so I've just done it. Code snippets:
| msg st c | msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := SHA256 hashMessage: msg asByteArray. c := LcEvpSHA256 new hashMessage: msg asByteArray. st = c => true
Using Sven's example:
| msg st c | key := ByteArray new: 32. msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := (HMAC on: SHA256) key: key; digestMessage: msg asByteArray. c := (HMAC on: LcEvpSHA256) key: key; digestMessage: msg asByteArray. st = c => true
Code at http://smalltalkhub.com/#!/~PierceNg/OpenSSL-Pharo.
I've also added tests for SHA256 similar to the snippets above. I will add SHA512 tests when I find the test vectors.
I don't about SHA512-only, but I found some HMACSHA512 test vectors at https://tools.ietf.org/html/rfc4231. I've attached the first five tests I got working against LibSodium.
In the next two weeks when I have some time off from work, I will attempt to move the repo to GH. Meanwhile, I've added you as a contributor to the STH repo if you'd like to hack on it.
thx. cheers -ben
On 11 December 2017 at 20:28, Pierce Ng <pierce@samadhiweb.com> wrote:
On Mon, Dec 11, 2017 at 12:46:59AM +0800, Ben Coman wrote:
where the "EVP function provide a high level interface to OpenSSL cryptographic functions."
Hi Ben,
As it happened, over the weekend I implemented the Pharo wrappers for the EVP_DigestSign* and EVP_DigestVerify* APIs. It is straightforward to add the non-public key-using message digest functions, so I've just done it. Code snippets:
| msg st c | msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := SHA256 hashMessage: msg asByteArray. c := LcEvpSHA256 new hashMessage: msg asByteArray. st = c => true
Using Sven's example:
| msg st c | key := ByteArray new: 32. msg := 'Grumpy wizards make toxic brew for the evil Queen and Jack.'. st := (HMAC on: SHA256) key: key; digestMessage: msg asByteArray. c := (HMAC on: LcEvpSHA256) key: key; digestMessage: msg asByteArray. st = c => true
Great! Looks like you've been busy! I was going to ask why these updates aren't being pushed an OpenSSL repo shared with Squeak, since things should not be much different at this low level, but actually it was hard to determine which was the original repo. I see... http://www.squeaksource.com/SqueakSSL.html seems to have not beenupdated since 2012. cheers -ben I've also added tests for SHA256 similar to the snippets above. I will add
SHA512 tests when I find the test vectors.
In the next two weeks when I have some time off from work, I will attempt to move the repo to GH. Meanwhile, I've added you as a contributor to the STH repo if you'd like to hack on it.
On Tue, Dec 12, 2017 at 12:52:11AM +0800, Ben Coman wrote:
I was going to ask why these updates aren't being pushed an OpenSSL repo shared with Squeak, since things should not be much different at this low level, but actually it was hard to determine which was the original repo. I see... http://www.squeaksource.com/SqueakSSL.html seems to have not beenupdated since 2012.
I think this might be the SSL plugin which is now part of the VM and is still called SqueakSSL. My library is for access to the crypto/X509 functions in OpenSSL's libcrypto. For green field applications I recommend using NaCl aka libsodium with its modern algorithms and crypto-safe programming practices. Pierce
Good to have extensions to Cryptography to include SHA-512, along with SHA-384. For that matter there seems to be other extensions could be made to Cryptography to support the proximate TLS 1.3, see references below for algorithms/groups specifiable. Cryptography would love to be extended for these, although OpenSSL should pick up TLS 1.3 for standardized wrapped implementation of TLS. https://tools.ietf.org/html/draft-ietf-tls-tls13-22#section-4.4.1 https://tools.ietf.org/html/draft-ietf-tls-tls13-22#section-4.2.3 https://tools.ietf.org/html/draft-ietf-tls-tls13-22#section-4.2.7 https://tools.ietf.org/html/rfc7919 - HH
-------- Original Message -------- Subject: [Pharo-users] HMAC-SHA512 Local Time: December 10, 2017 4:01 AM UTC Time: December 10, 2017 9:01 AM From: btc@openinworld.com To: Any question about pharo is welcome <pharo-users@lists.pharo.org>
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
Hi Ben, I would use NaCl: http://cr.yp.to/highspeed/coolnacl-20120725.pdf which you can install from the Pharo Project Catalog in Pharo 6.x Cheers, Hernán 2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
On 11 December 2017 at 01:30, Hernán Morales Durand < hernan.morales@gmail.com> wrote:
Hi Ben,
I would use NaCl: http://cr.yp.to/highspeed/coolnacl-20120725.pdf which you can install from the Pharo Project Catalog in Pharo 6.x
Thanks Henry. That looks like path of least resistance. I hit a stumbling block that took a short while to understand. The library binary downloaded specified by ConfigruationOfNacl>>platformLibraryUrl is libsodium: ELF 32-bit LSB shared object whereas I'm using 64bit Linux. I have the following system library preinstalled... /usr/lib/x86_64-linux-gnu/libsodium.so.18.0.1: ELF 64-bit LSB shared object However its api is slightly different. Pharo Nacl makes this call out... crypto_hash_sha512_ref() but the system library provides... 56: 0000000000014dc0 175 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_final 81: 0000000000014c40 384 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_update 154: 0000000000014bd0 106 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_init 233: 0000000000014e70 114 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512 342: 0000000000012330 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_bytes 351: 0000000000012340 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_statebytes The canonical source doesn't seem to have the function https://github.com/jedisct1/libsodium/search?utf8=%E2%9C%93&q=+crypto_hash_s... so I'm curious where the "_ref" comes from. My options seem... * Compile a 64-bit libsodium from source which includes "_ref" functions. Where is such source? * Use the system libsodium and define an FFI callout just for the one function I need. cheers -ben
On 11 December 2017 at 13:09, Ben Coman <btc@openinworld.com> wrote:
2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
On 11 December 2017 at 01:30, Hernán Morales Durand < hernan.morales@gmail.com> wrote:
Hi Ben,
I would use NaCl: http://cr.yp.to/highspeed/coolnacl-20120725.pdf which you can install from the Pharo Project Catalog in Pharo 6.x
Thanks Henry. That looks like path of least resistance.
Whoops, I meant thanks Hernán.
I hit a stumbling block that took a short while to understand. The library binary downloaded specified by ConfigruationOfNacl>> platformLibraryUrl is libsodium: ELF 32-bit LSB shared object whereas I'm using 64bit Linux.
btw, I discovered just now that FFILibrary>>unix64ModuleName was introduced this year. So presumably with the following you could include both bit'ness libraries in the libsodium.so.gz file specified in ConfigurationOfNacl >> platformLibraryUrl. FFILbrary subclass: #Nacl instanceVariableNames: '' classVariableNames: '' package: 'Crypto-Nacl' Nacl >> unixModuleName ^ 'libsodium.so' Nacl >> unix64ModuleName ^ 'libsodium64.so' Nacl class >> apiXXX ^ self ffiCall: #(void XXX()) module: Nacl
I have the following system library preinstalled... /usr/lib/x86_64-linux-gnu/libsodium.so.18.0.1: ELF 64-bit LSB shared object
However its api is slightly different. Pharo Nacl makes this call out... crypto_hash_sha512_ref()
but the system library provides... 56: 0000000000014dc0 175 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_final 81: 0000000000014c40 384 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_update 154: 0000000000014bd0 106 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_init 233: 0000000000014e70 114 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512 342: 0000000000012330 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_bytes 351: 0000000000012340 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_statebytes
The canonical source doesn't seem to have the function https://github.com/jedisct1/libsodium/search?utf8=%E2%9C% 93&q=+crypto_hash_sha512_ref so I'm curious where the "_ref" comes from.
My options seem... * Compile a 64-bit libsodium from source which includes "_ref" functions. Where is such source?
* Use the system libsodium and define an FFI callout just for the one
function I need.
btw, I'm doing the latter for now as the immediate path of least resistance. Thanks Hernan, for the tip on libsodium. cheers -ben
2017-12-11 5:28 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 11 December 2017 at 13:09, Ben Coman <btc@openinworld.com> wrote:
2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
On 11 December 2017 at 01:30, Hernán Morales Durand <hernan.morales@gmail.com> wrote:
Hi Ben,
I would use NaCl: http://cr.yp.to/highspeed/coolnacl-20120725.pdf which you can install from the Pharo Project Catalog in Pharo 6.x
Thanks Henry. That looks like path of least resistance.
Whoops, I meant thanks Hernán.
I hit a stumbling block that took a short while to understand. The library binary downloaded specified by ConfigruationOfNacl>>platformLibraryUrl is libsodium: ELF 32-bit LSB shared object whereas I'm using 64bit Linux.
btw, I discovered just now that FFILibrary>>unix64ModuleName was introduced this year. So presumably with the following you could include both bit'ness libraries in the libsodium.so.gz file specified in ConfigurationOfNacl >> platformLibraryUrl.
FFILbrary subclass: #Nacl instanceVariableNames: '' classVariableNames: '' package: 'Crypto-Nacl'
Nacl >> unixModuleName ^ 'libsodium.so'
Nacl >> unix64ModuleName ^ 'libsodium64.so'
Nacl class >> apiXXX ^ self ffiCall: #(void XXX()) module: Nacl
Done. Thank you Ben. All tests passes in Windows, so I uploaded a new Configuration with updated dev version. Let me know any issues.
I have the following system library preinstalled... /usr/lib/x86_64-linux-gnu/libsodium.so.18.0.1: ELF 64-bit LSB shared object
However its api is slightly different. Pharo Nacl makes this call out... crypto_hash_sha512_ref()
but the system library provides... 56: 0000000000014dc0 175 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_final 81: 0000000000014c40 384 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_update 154: 0000000000014bd0 106 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_init 233: 0000000000014e70 114 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512 342: 0000000000012330 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_bytes 351: 0000000000012340 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_statebytes
The canonical source doesn't seem to have the function
https://github.com/jedisct1/libsodium/search?utf8=%E2%9C%93&q=+crypto_hash_s... so I'm curious where the "_ref" comes from.
My options seem... * Compile a 64-bit libsodium from source which includes "_ref" functions. Where is such source?
The official sources seem to be here: https://download.libsodium.org/libsodium/releases/ I discovered that the version we use in the Pharo binding is old, so I will update to the latest version as next step.
* Use the system libsodium and define an FFI callout just for the one function I need.
btw, I'm doing the latter for now as the immediate path of least resistance. Thanks Hernan, for the tip on libsodium.
cheers -ben
Cheers Hernán
2017-12-11 10:13 GMT-03:00 Hernán Morales Durand <hernan.morales@gmail.com>:
2017-12-11 5:28 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 11 December 2017 at 13:09, Ben Coman <btc@openinworld.com> wrote:
2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
On 11 December 2017 at 01:30, Hernán Morales Durand <hernan.morales@gmail.com> wrote:
Hi Ben,
I would use NaCl: http://cr.yp.to/highspeed/coolnacl-20120725.pdf which you can install from the Pharo Project Catalog in Pharo 6.x
Thanks Henry. That looks like path of least resistance.
Whoops, I meant thanks Hernán.
I hit a stumbling block that took a short while to understand. The library binary downloaded specified by ConfigruationOfNacl>>platformLibraryUrl is libsodium: ELF 32-bit LSB shared object whereas I'm using 64bit Linux.
btw, I discovered just now that FFILibrary>>unix64ModuleName was introduced this year. So presumably with the following you could include both bit'ness libraries in the libsodium.so.gz file specified in ConfigurationOfNacl >> platformLibraryUrl.
FFILbrary subclass: #Nacl instanceVariableNames: '' classVariableNames: '' package: 'Crypto-Nacl'
Nacl >> unixModuleName ^ 'libsodium.so'
Nacl >> unix64ModuleName ^ 'libsodium64.so'
Nacl class >> apiXXX ^ self ffiCall: #(void XXX()) module: Nacl
Done. Thank you Ben.
All tests passes in Windows, so I uploaded a new Configuration with updated dev version. Let me know any issues.
BTW you can install the dev version using: Metacello new smalltalkhubUser: 'tonyg' project: 'Crypto-Nacl'; configuration: 'Nacl'; version: #development; load.
I have the following system library preinstalled... /usr/lib/x86_64-linux-gnu/libsodium.so.18.0.1: ELF 64-bit LSB shared object
However its api is slightly different. Pharo Nacl makes this call out... crypto_hash_sha512_ref()
but the system library provides... 56: 0000000000014dc0 175 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_final 81: 0000000000014c40 384 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_update 154: 0000000000014bd0 106 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_init 233: 0000000000014e70 114 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512 342: 0000000000012330 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_bytes 351: 0000000000012340 6 FUNC GLOBAL DEFAULT 11 crypto_hash_sha512_statebytes
The canonical source doesn't seem to have the function
https://github.com/jedisct1/libsodium/search?utf8=%E2%9C%93&q=+crypto_hash_s... so I'm curious where the "_ref" comes from.
My options seem... * Compile a 64-bit libsodium from source which includes "_ref" functions. Where is such source?
The official sources seem to be here: https://download.libsodium.org/libsodium/releases/ I discovered that the version we use in the Pharo binding is old, so I will update to the latest version as next step.
* Use the system libsodium and define an FFI callout just for the one function I need.
btw, I'm doing the latter for now as the immediate path of least resistance. Thanks Hernan, for the tip on libsodium.
cheers -ben
Cheers
Hernán
Hi Ben, all, So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6? I'd need it in combination with PBKDF2 to replicate this Python call: PBKDF2(password, salt, iterations, macmodule=hmac, digestmodule=hashlib.sha512).read(64) Regards, Esteban A. Maringolo 2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
PBKDF2 is in Cryptography yet I think it may hardcore HMAC-SHA256. An implementation of SHA512 in Cryptography be so cool ð Sent from ProtonMail Mobile On Thu, Mar 1, 2018 at 21:43, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
Hi Ben, all, So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6? I'd need it in combination with PBKDF2 to replicate this Python call: PBKDF2(password, salt, iterations, macmodule=hmac, digestmodule=hashlib.sha512).read(64) Regards, Esteban A. Maringolo 2017-12-10 6:01 GMT-03:00 Ben Coman : > Can anyone recommend libraries (native Smalltalk or via FFI) > to do generate a HMAC-SHA512 ? > > cheers -ben @openinworld.com>
On 2 March 2018 at 10:43, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
Hi Ben, all,
So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6?
Start poking around here... https://github.com/Traadh/bittrex/blob/master/src/Bittrex.package/BittrexLib... https://github.com/Traadh/bittrex/blob/master/src/Bittrex.package/BittrexPri... Libsodium installation instructions... https://github.com/Traadh/bittrex I'd need it in combination with PBKDF2 to replicate this Python call:
PBKDF2(password, salt, iterations, macmodule=hmac, digestmodule=hashlib.sha512).read(64)
I guess with iterations=1 you should get the same result as libsodium I had to play around a bit before I worked out where to use bytes and where to use hex strings. It might help to run these test cases through PBKDF2... https://github.com/Traadh/bittrex/tree/master/src/Bittrex.package/BittrexLib... cheers -ben
Regards,
Esteban A. Maringolo
2017-12-10 6:01 GMT-03:00 Ben Coman <btc@openinworld.com>:
Can anyone recommend libraries (native Smalltalk or via FFI) to do generate a HMAC-SHA512 ?
cheers -ben
2018-03-02 8:54 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 2 March 2018 at 10:43, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6?
Libsodium installation instructions... https://github.com/Traadh/bittrex
Why did you create BittrexLibsodium library wrapper instead of a plain Libsodium wrapper like the one at http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ ?
I'd need it in combination with PBKDF2 to replicate this Python call: PBKDF2(password, salt, iterations, macmodule=hmac, digestmodule=hashlib.sha512).read(64)
I guess with iterations=1 you should get the same result as libsodium I had to play around a bit before I worked out where to use bytes and where to use hex strings. It might help to run these test cases through PBKDF2... https://github.com/Traadh/bittrex/tree/master/src/Bittrex.package/BittrexLib...
Well, I need 2048 iterations. There is a PBKDF2 package made by Udo Schneider, and seems to be easily pluggable with a different hashing algorithm, so I'd need a SHA512 class. Thanks in advance. Regards. Esteban A. Maringolo
Hi 2018-03-02 11:24 GMT-03:00 Esteban A. Maringolo <emaringolo@gmail.com>:
2018-03-02 8:54 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 2 March 2018 at 10:43, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6?
Libsodium installation instructions... https://github.com/Traadh/bittrex
Why did you create BittrexLibsodium library wrapper instead of a plain Libsodium wrapper like the one at http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ ?
I inspected the Nacl library and it provides the functions: crypto_auth_hmacsha256 crypto_auth_hmacsha256_bytes crypto_auth_hmacsha256_keybytes crypto_auth_hmacsha256_primitive crypto_auth_hmacsha256_ref crypto_auth_hmacsha256_ref_verify crypto_auth_hmacsha256_verify crypto_auth_hmacsha512256 crypto_auth_hmacsha512256_bytes crypto_auth_hmacsha512256_keybytes crypto_auth_hmacsha512256_primitive crypto_auth_hmacsha512256_ref crypto_auth_hmacsha512256_ref_verify crypto_auth_hmacsha512256_verify I've also followed the Ben's example in Nacl but the output is truncated | data message output key expected | data := 'Hi There' . key := (ByteArray readHexFrom: '0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b', '0b0b0b0b') extendedToSize: 32. expected := ByteArray readHexFrom: '87aa7cdea5ef619d4ff0b4241a1d6cb0', '2379f4e2ce4ec2787ad0b30545e17cde', 'daa833b7d6b8a702038b274eaea3f4e4', 'be9d914eeb61f1702e696c203a126854'. output := ByteArray new: 64. Nacl apiCryptoAuthHMAC512: output message: data messageLength: data size key: key. output Nacl class>>apiCryptoAuthHMAC512: c message: m messageLength: mlen key: key ^ self ffiCall: #(long crypto_auth_hmacsha512256(byte * c, byte * m, ulonglong mlen, byte * key)) module: 'libsodium'. Ben do you know what could be happening here? Hernán
On 3 March 2018 at 02:22, Hernán Morales Durand <hernan.morales@gmail.com> wrote:
Hi
2018-03-02 11:24 GMT-03:00 Esteban A. Maringolo <emaringolo@gmail.com>:
2018-03-02 8:54 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 2 March 2018 at 10:43, Esteban A. Maringolo <emaringolo@gmail.com> wrote:
So after all this what's the recommended way to use HMAC-SHA512 in Pharo 6?
Libsodium installation instructions... https://github.com/Traadh/bittrex
Why did you create BittrexLibsodium library wrapper instead of a plain Libsodium wrapper like the one at http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ ?
If that is the one available from the Pharo Catalog, when I tried it, it used a different api than the libsodium library supplied by Ubuntu 16.04. It makes this call out... crypto_hash_sha512_ref() but Ubuntu libsodium library did not export that, but instead... crypto_hash_sha512() I failed to track down the implications of the "_ref", and at that time Crypto-Nacl did not supply a 64-bit library, so a single FFI matching the system supplied libsodium library was easier. Also as a minor policy, I prefer to use the system library than one compiled by a third-party. cheers -ben
I inspected the Nacl library and it provides the functions:
crypto_auth_hmacsha256 crypto_auth_hmacsha256_bytes crypto_auth_hmacsha256_keybytes crypto_auth_hmacsha256_primitive crypto_auth_hmacsha256_ref crypto_auth_hmacsha256_ref_verify crypto_auth_hmacsha256_verify crypto_auth_hmacsha512256 crypto_auth_hmacsha512256_bytes crypto_auth_hmacsha512256_keybytes crypto_auth_hmacsha512256_primitive crypto_auth_hmacsha512256_ref crypto_auth_hmacsha512256_ref_verify crypto_auth_hmacsha512256_verify
I've also followed the Ben's example in Nacl but the output is truncated
| data message output key expected | data := 'Hi There' . key := (ByteArray readHexFrom: '0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b0b', '0b0b0b0b') extendedToSize: 32. expected := ByteArray readHexFrom: '87aa7cdea5ef619d4ff0b4241a1d6cb0', '2379f4e2ce4ec2787ad0b30545e17cde', 'daa833b7d6b8a702038b274eaea3f4e4', 'be9d914eeb61f1702e696c203a126854'. output := ByteArray new: 64. Nacl apiCryptoAuthHMAC512: output message: data messageLength: data size key: key. output
Just off the top of my head, should the following method properly be named apiCryptoAuthHMAC512256:message:messageLength:key: to match the library function its calling?
Nacl class>>apiCryptoAuthHMAC512: c message: m messageLength: mlen key: key
^ self ffiCall: #(long crypto_auth_hmacsha512256(byte * c, byte * m, ulonglong mlen, byte * key)) module: 'libsodium'.
Ben do you know what could be happening here?
By how much is it truncated? Perhaps half? https://dirvine.github.io/data_chain/master///rust_sodium_sys/constant.crypt... https://dirvine.github.io/data_chain/master///rust_sodium_sys/constant.crypt... cheers -ben
On Fri, Mar 2, 2018 at 9:40 PM, Ben Coman <btc@openinworld.com> wrote: [...]
If that is the one available from the Pharo Catalog, when I tried it, it used a different api than the libsodium library supplied by Ubuntu 16.04.
It makes this call out... crypto_hash_sha512_ref() but Ubuntu libsodium library did not export that, but instead... crypto_hash_sha512()
I failed to track down the implications of the "_ref", and at that time Crypto-Nacl did not supply a 64-bit library, so a single FFI matching the system supplied libsodium library was easier. Also as a minor policy, I prefer to use the system library than one compiled by a third-party.
cheers -ben
[...] Hi, Yes, I recommend using the OS/package-manager supplied libsodium (based on https://github.com/jedisct1/libsodium). The downloads provided on Tony's site were based on early commits that exported the "_ref" functions instead: commit e144f9d40d5695b69306bde729d6d8adcd62b1c4 Author: Frank Denis <github@pureftpd.org> Date: Mon Apr 22 16:30:31 2013 -0700 crypto_hash_sha(256|512) are the exported functions that have to be exported. _ref are implementations, that shouldn't be exported. To be useful, http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ should be updated to use a system-supplied libsodium. Best, Mike
On 8 March 2018 at 02:38, Michael Forster <mike@forsterfamily.ca> wrote:
On Fri, Mar 2, 2018 at 9:40 PM, Ben Coman <btc@openinworld.com> wrote: [...]
If that is the one available from the Pharo Catalog, when I tried it, it used a different api than the libsodium library supplied by Ubuntu 16.04.
It makes this call out... crypto_hash_sha512_ref() but Ubuntu libsodium library did not export that, but instead... crypto_hash_sha512()
I failed to track down the implications of the "_ref", and at that time Crypto-Nacl did not supply a 64-bit library, so a single FFI matching the system supplied libsodium library was
easier.
Also as a minor policy, I prefer to use the system library than one compiled by a third-party.
cheers -ben [...]
Hi,
Yes, I recommend using the OS/package-manager supplied libsodium (based on https://github.com/jedisct1/libsodium). The downloads provided on Tony's site were based on early commits that exported the "_ref" functions instead:
commit e144f9d40d5695b69306bde729d6d8adcd62b1c4 Author: Frank Denis <github@pureftpd.org> Date: Mon Apr 22 16:30:31 2013 -0700
crypto_hash_sha(256|512) are the exported functions that have to be exported. _ref are implementations, that shouldn't be exported.
Thanks for clarifying that.
To be useful, http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ should be updated to use a system-supplied libsodium.
Or at least to the later api. In general there is some advantage in packaging the C-lib with the Smalltalk package for people experimenting with it the first time, to ensure the function prototypes match. But flipping into production may prefer the system library. Currently it seems awkward to switch later to a system library since the library used is hard coded. How do others work around that? Perhaps it would be good to have the external library initialized into an instance variable of FFILIbrary, so a user-application can change the external C-library used without changing code in someone else's ffi-interface package. cheers -ben P.S. On the dream list would be a GUI for managing referenced external C-libraries, including browsing their exported functions.
2018-03-07 23:07 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 8 March 2018 at 02:38, Michael Forster <mike@forsterfamily.ca> wrote:
On Fri, Mar 2, 2018 at 9:40 PM, Ben Coman <btc@openinworld.com> wrote: [...]
If that is the one available from the Pharo Catalog, when I tried it, it used a different api than the libsodium library supplied by Ubuntu 16.04.
It makes this call out... crypto_hash_sha512_ref() but Ubuntu libsodium library did not export that, but instead... crypto_hash_sha512()
I failed to track down the implications of the "_ref", and at that time Crypto-Nacl did not supply a 64-bit library, so a single FFI matching the system supplied libsodium library was easier. Also as a minor policy, I prefer to use the system library than one compiled by a third-party.
cheers -ben
[...]
Hi,
Yes, I recommend using the OS/package-manager supplied libsodium (based on https://github.com/jedisct1/libsodium). The downloads provided on Tony's site were based on early commits that exported the "_ref" functions instead:
commit e144f9d40d5695b69306bde729d6d8adcd62b1c4 Author: Frank Denis <github@pureftpd.org> Date: Mon Apr 22 16:30:31 2013 -0700
crypto_hash_sha(256|512) are the exported functions that have to be exported. _ref are implementations, that shouldn't be exported.
Thanks for clarifying that.
To be useful, http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ should be updated to use a system-supplied libsodium.
Or at least to the later api. In general there is some advantage in packaging the C-lib with the Smalltalk package for people experimenting with it the first time, to ensure the function prototypes match. But flipping into production may prefer the system library. Currently it seems awkward to switch later to a system library since the library used is hard coded.
Hi Ben, "system library" means it's pre-installed in some Linux distros? I installed a Debian 4.9.65-3+deb9u1 (2017-12-23) last week and libsodium is not present. I use Windows and also isn't available by default. In any case, how do you ensure libsodium system lib matches your interface code? What do you do now if target has a different version?
How do others work around that?
Perhaps it would be good to have the external library initialized into an instance variable of FFILIbrary, so a user-application can change the external C-library used without changing code in someone else's ffi-interface package.
cheers -ben
P.S. On the dream list would be a GUI for managing referenced external C-libraries, including browsing their exported functions.
That would be really cool Hernán
On 8 March 2018 at 12:22, Hernán Morales Durand <hernan.morales@gmail.com> wrote:
2018-03-07 23:07 GMT-03:00 Ben Coman <btc@openinworld.com>:
On 8 March 2018 at 02:38, Michael Forster <mike@forsterfamily.ca> wrote:
On Fri, Mar 2, 2018 at 9:40 PM, Ben Coman <btc@openinworld.com> wrote: [...]
If that is the one available from the Pharo Catalog, when I tried it,
it
used a different api than the libsodium library supplied by Ubuntu 16.04.
It makes this call out... crypto_hash_sha512_ref() but Ubuntu libsodium library did not export that, but instead... crypto_hash_sha512()
I failed to track down the implications of the "_ref", and at that time Crypto-Nacl did not supply a 64-bit library, so a single FFI matching the system supplied libsodium library was easier. Also as a minor policy, I prefer to use the system library than one compiled by a third-party.
cheers -ben [...]
Hi,
Yes, I recommend using the OS/package-manager supplied libsodium (based on https://github.com/jedisct1/libsodium). The downloads provided on Tony's site were based on early commits that exported the "_ref" functions instead:
commit e144f9d40d5695b69306bde729d6d8adcd62b1c4 Author: Frank Denis <github@pureftpd.org> Date: Mon Apr 22 16:30:31 2013 -0700
crypto_hash_sha(256|512) are the exported functions that have to be exported. _ref are implementations, that shouldn't be exported.
Thanks for clarifying that.
To be useful, http://smalltalkhub.com/#!/~tonyg/Crypto-Nacl/ should be updated to use a system-supplied libsodium.
Or at least to the later api. In general there is some advantage in packaging the C-lib with the Smalltalk package for people experimenting with it the first time, to ensure the function prototypes match. But flipping into production may prefer the system library. Currently it seems awkward to switch later to a system library since the library used is hard coded.
Hi Ben,
"system library" means it's pre-installed in some Linux distros?
I installed a Debian 4.9.65-3+deb9u1 (2017-12-23) last week and
libsodium is not present.
For Linux, system libraries are not only pre-installed, but rather whatever is available in the default system repository, simply installed with a single command like... $ sudo apt-get install libsodium All such installed packages receive security updates by doing... $ sudo apt-get update $ sudo apt-get upgrade I use Windows and also isn't available by default. Unfortunately Windows lacks a standard repository with a broad range of open source packages installable with a single command. Here it makes more sense to auto-install some package. Perhaps a particular version from the project https://download.libsodium.org/libsodium/releases/ In any case, how do you ensure libsodium system lib matches your interface
code?
What do you do now if target has a different version?
Good question. I guess its unit tests and manual effort. The next question is whether its worth the effort. For a given production environment, you might take a hit every couple of years when you move distro-versions. For a more turnkey system suiting a broader range of new users, maybe downloading a particular version of libsodium from the project site is a better option. Checking just now i see the Ubuntu 16.04 LTS libsodium library is version 1.0.8... https://launchpad.net/ubuntu/xenial/+package/libsodium18 which actually is unsupported... https://download.libsodium.org/libsodium/releases/old/unsupported/ so perhaps downloading from the project site is a better option. cheers -ben P.S. Did you indicate Crypto-Nacl was being (or could be) updated to latest libsodium? If it could, and the auto-installed library was downloaded and sig-checked from the project site, I'd be interested in dropping my own hack and using Crypto-Nacl.
How do others work around that?
Perhaps it would be good to have the external library initialized into an instance variable of FFILIbrary, so a user-application can change the external C-library used without changing code in someone else's ffi-interface package.
cheers -ben
P.S. On the dream list would be a GUI for managing referenced external C-libraries, including browsing their exported functions.
That would be really cool
Hernán
On Fri, Mar 02, 2018 at 11:24:36AM -0300, Esteban A. Maringolo wrote:
There is a PBKDF2 package made by Udo Schneider, and seems to be easily pluggable with a different hashing algorithm, so I'd need a SHA512 class.
OpenSSL-Pharo wraps the SHA512 in libcrypto. See LcEvpSHA512Test. https://github.com/PierceNg/OpenSSL-Pharo Pierce
2018-03-04 0:45 GMT-03:00 Pierce Ng <pierce@samadhiweb.com>:
On Fri, Mar 02, 2018 at 11:24:36AM -0300, Esteban A. Maringolo wrote:
There is a PBKDF2 package made by Udo Schneider, and seems to be easily pluggable with a different hashing algorithm, so I'd need a SHA512 class.
OpenSSL-Pharo wraps the SHA512 in libcrypto. See LcEvpSHA512Test.
Thank you Pierce, I managed to make play the SHA classes of your OpenSSL wrapper with the PBKDF2 from Udo's plus HMAC from the Cryptography package. I sent you a pull-request <https://github.com/PierceNg/OpenSSL-Pharo/pull/1> that includes missing accessors that I needed to achieve the above interplay. The end result is this: https://github.com/eMaringolo/pharo-bip39mnemonic What I'm missing is a better PRNG to generate random numbers from 128 to 256 bits. But my main objective was a proof of concept that ended up being successful. Thanks for your help. Esteban A. Maringolo
On Sun, Mar 04, 2018 at 04:00:23PM -0300, Esteban A. Maringolo wrote:
Thank you Pierce, I managed to make play the SHA classes of your OpenSSL wrapper with the PBKDF2 from Udo's plus HMAC from the Cryptography package. I sent you a pull-request <https://github.com/PierceNg/OpenSSL-Pharo/pull/1> that includes missing accessors that I needed to achieve the above interplay.
Thanks Esteban. Pull request merged.
The end result is this: https://github.com/eMaringolo/pharo-bip39mnemonic
Cool!
What I'm missing is a better PRNG to generate random numbers from 128 to 256 bits.
OpenSSL has RAND_* functions waiting to be wrapped. :-) Or, if your code is running on a modern Unix-like system, read directly from /dev/random or /dev/urandom. Pierce
participants (7)
-
Ben Coman -
Esteban A. Maringolo -
henry -
Hernán Morales Durand -
Michael Forster -
Pierce Ng -
Sven Van Caekenberghe