Is there a way to make SSL connections to the outside world using client certificates from pharo? thanks, Norbert
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea. Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work. And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
On 12 Jan 2016, at 16:25, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
I see that in ZnClient>>#setupTLSTo: the certificate set on ZnClient is passed on automatically. So at one point, this must have worked, and probably still does.
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
I tried on Mac OS and on Linux but no luck so far. I used | ip stream | ip := NetNameResolver addressForName: 'gateway.sandbox.push.apple.com' timeout: 5000. stream := ZdcSecureSocketStream openConnectionToHost: ip port: 2195 timeout: 3000. stream binary; shouldSignal: true; autoFlush: false; bufferSize: 16384; timeout: 5000. stream sslSession enableLogging; certificateName: '/root/my.pem'. stream connect. But I get ConnectionClosed: Connection closed while waiting for data. This is while doing the SSL handshake. On Linux the log is qSetStringPropertySSL(1): /root/my.pem sqConnectSSL: 8a69058 sqConnectSSL: Setting up SSL sqSetupSSL: setting method sqSetupSSL: Creating context sqSetupSSL: setting cipher list sqSetupSSL: Using cert file /root/my.pem sqSetupSSL: No root CA given; using default verify paths sqSetupSSL: Creating SSL sqSetupSSL: setting bios sqConnectSSL: Setting connect state sqConnectSSL: BIO_write 0 bytes sqConnectSSL: SSL_connect sqConnectSSL: sqCopyBioSSL sqCopyBioSSL: 297 bytes pending; buffer size 17408 sqConnectSSL: 8a69058 sqConnectSSL: BIO_write 3143 bytes sqConnectSSL: SSL_connect sqConnectSSL: sqCopyBioSSL sqCopyBioSSL: 2057 bytes pending; buffer size 17408 ConnectionClosed: Connection closed while waiting for data. [ ConnectionClosed signal: 'Connection closed while waiting for data.' ] in Socket>>waitForDataFor: in Block: [ ConnectionClosed signal: 'Connection closed whil...etc... Socket>>waitForDataFor:ifClosed:ifTimedOut: Socket>>waitForDataFor: ZdcSecureSocketStream(ZdcAbstractSocketStream)>>socketWaitForData ZdcSecureSocketStream>>readEncryptedBytes:startingAt:count: ZdcSecureSocketStream>>connect UndefinedObject>>DoIt I'm not sure what is going on and I believe it is rather hard to figure out what is the error exactly. Using $ openssl s_client -connect gateway.sandbox.push.apple.com:2195 -cert my.pem -debug -showcerts -CAfile entrust_2048_ca.cer it works. I don't know if the missing CA file is the culprit. But I doubt the squeak SSL code does verifying so it might be something else. Norbert
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
Sven,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
I spent some time to figure out how this works. But I don't get it. I don't know what this copying back and forth of bytes in the SSL handshake is supposed to do. I just can see that on a second read from the socket the connection is closed from the remote side. Not much here. I think it will take too much time for me to figure it out. And I'm afraid I don't have that many spare cycles. We should rethink "software bounties". I don't have the time to fix this but I'm willing to put some money on the table to have it implemented properly. There are more people wanting to pay for features implemented. And I'm sure there are developers that would like to do it when being paid. Norbert
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
On 13 Jan 2016, at 13:33, Norbert Hartl <norbert@hartl.name> wrote:
Sven,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
I spent some time to figure out how this works. But I don't get it. I don't know what this copying back and forth of bytes in the SSL handshake is supposed to do. I just can see that on a second read from the socket the connection is closed from the remote side. Not much here. I think it will take too much time for me to figure it out. And I'm afraid I don't have that many spare cycles.
Unless you are reimplementing SSL from the ground up, it is not really super important to understand what is happening. The whole idea of using a library like OpenSSL is to delegate all the hard work to it, in particular the handshake. The fact that the other end closes most probably means that something is wrong (duh): the client did not behave or act as expected, or it provided some strange/wrong/bogus data.
We should rethink "software bounties". I don't have the time to fix this but I'm willing to put some money on the table to have it implemented properly. There are more people wanting to pay for features implemented. And I'm sure there are developers that would like to do it when being paid.
Yeah. But to complete this task successfully one has to know about Pharo, about C and about SSL/TLS. A C version directly using the OpenSSL API should be compared to what the SSL Plugin does, that is the only way out. And it would not hurt to rewrite the SSL Plugin itself in a very clean cross platform way using only 1 OS API in C (OpenSSL), not 3 like today, with proper logging and error handling.
Norbert
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
Am 13.01.2016 um 13:59 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 13 Jan 2016, at 13:33, Norbert Hartl <norbert@hartl.name> wrote:
Sven,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
I spent some time to figure out how this works. But I don't get it. I don't know what this copying back and forth of bytes in the SSL handshake is supposed to do. I just can see that on a second read from the socket the connection is closed from the remote side. Not much here. I think it will take too much time for me to figure it out. And I'm afraid I don't have that many spare cycles.
Unless you are reimplementing SSL from the ground up, it is not really super important to understand what is happening. The whole idea of using a library like OpenSSL is to delegate all the hard work to it, in particular the handshake.
The fact that the other end closes most probably means that something is wrong (duh): the client did not behave or act as expected, or it provided some strange/wrong/bogus data.
Yeah, I was able to do a similar analysis of the problem :P It is just very hard to figure out what is the bogus part in there. Using the openssl client I could verify that the certificate in the file is valid. So no glue what went wrong otherwise.
We should rethink "software bounties". I don't have the time to fix this but I'm willing to put some money on the table to have it implemented properly. There are more people wanting to pay for features implemented. And I'm sure there are developers that would like to do it when being paid.
Yeah.
But to complete this task successfully one has to know about Pharo, about C and about SSL/TLS. A C version directly using the OpenSSL API should be compared to what the SSL Plugin does, that is the only way out. And it would not hurt to rewrite the SSL Plugin itself in a very clean cross platform way using only 1 OS API in C (OpenSSL), not 3 like today, with proper logging and error handling.
Yes, it is not easy. But there are a lot of people capable of doing it I guess. Having a decent SSL implementation is absolutely desirable IMHO. Norbert
Norbert
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
Excellent idea!!!
We should rethink "software bounties". I don't have the time to fix this but I'm willing to put some money on the table to have it implemented properly. There are more people wanting to pay for features implemented. And I'm sure there are developers that would like to do it when being paid. PS: norbert I need a new success story :)
Yes, I will look at existing platforms (started already in christmas holidays, will continue). Marcus
On 13 Jan 2016, at 16:34, stepharo <stepharo@free.fr> wrote:
Excellent idea!!!
We should rethink "software bounties". I don't have the time to fix this but I'm willing to put some money on the table to have it implemented properly. There are more people wanting to pay for features implemented. And I'm sure there are developers that would like to do it when being paid. PS: norbert I need a new success story :)
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
There were some issues with certificates. I tested a lot but did an error on the way. Now I did everything again like creating certificates etc. I could establish a connection from a linux machine. At least it doesn't signal anything an tells it is connected. On Mac OS it does not work. I think it has something to do with the certificate authority. On linux the CA of the apple server is installed system wide. The SSL plugin code only looks for the certificate and the key but not for the CA. So under linux it finds it in the system but on Mac OS I couldn't do the same. I imported the CA in the keychain but openssl does not seem to find it there. Norbert
On 14 Jan 2016, at 12:12, Norbert Hartl <norbert@hartl.name> wrote:
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
There were some issues with certificates. I tested a lot but did an error on the way. Now I did everything again like creating certificates etc. I could establish a connection from a linux machine. At least it doesn't signal anything an tells it is connected.
So that *is* good news ! If you could share more details, that might help others as well.
On Mac OS it does not work.
Have you seen the C source code of the Mac SSL Plugin ? It is written against an API from Mac OS System 7, from before Mac OS X, that is an OS from last century, ~199X. This is ancient, you won't find many developers who want to work on that, nor does it make sense towards the future. OpenSSL is also available standard on Mac OS X, the Linux plugin could just as well be used there (give or take).
I think it has something to do with the certificate authority. On linux the CA of the apple server is installed system wide. The SSL plugin code only looks for the certificate and the key but not for the CA. So under linux it finds it in the system but on Mac OS I couldn't do the same. I imported the CA in the keychain but openssl does not seem to find it there.
It is very simple why it does not work on Mac (like I said), it is just not implemented ! Try finding where certName is used in sqMacSSL.c, there is a setter and getter, but it is simply not used, so it can never work. In sqUnixOpenSSL.c on the other hand, there is /* if a cert is provided, use it */ if(ssl->certName) { if(ssl->loglevel) printf("sqSetupSSL: Using cert file %s\n", ssl->certName); if(SSL_CTX_use_certificate_file(ssl->ctx, ssl->certName, SSL_FILETYPE_PEM)<=0) ERR_print_errors_fp(stderr); if(SSL_CTX_use_PrivateKey_file(ssl->ctx, ssl->certName, SSL_FILETYPE_PEM)<=0) ERR_print_errors_fp(stderr); } Sven
Norbert
Am 14.01.2016 um 12:42 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
On 14 Jan 2016, at 12:12, Norbert Hartl <norbert@hartl.name> wrote:
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
There were some issues with certificates. I tested a lot but did an error on the way. Now I did everything again like creating certificates etc. I could establish a connection from a linux machine. At least it doesn't signal anything an tells it is connected.
So that *is* good news ! If you could share more details, that might help others as well.
I'm on it :)
On Mac OS it does not work.
Have you seen the C source code of the Mac SSL Plugin ? It is written against an API from Mac OS System 7, from before Mac OS X, that is an OS from last century, ~199X. This is ancient, you won't find many developers who want to work on that, nor does it make sense towards the future.
OpenSSL is also available standard on Mac OS X, the Linux plugin could just as well be used there (give or take).
I think it has something to do with the certificate authority. On linux the CA of the apple server is installed system wide. The SSL plugin code only looks for the certificate and the key but not for the CA. So under linux it finds it in the system but on Mac OS I couldn't do the same. I imported the CA in the keychain but openssl does not seem to find it there.
It is very simple why it does not work on Mac (like I said), it is just not implemented !
Try finding where certName is used in sqMacSSL.c, there is a setter and getter, but it is simply not used, so it can never work.
In sqUnixOpenSSL.c on the other hand, there is
/* if a cert is provided, use it */ if(ssl->certName) { if(ssl->loglevel) printf("sqSetupSSL: Using cert file %s\n", ssl->certName); if(SSL_CTX_use_certificate_file(ssl->ctx, ssl->certName, SSL_FILETYPE_PEM)<=0) ERR_print_errors_fp(stderr);
if(SSL_CTX_use_PrivateKey_file(ssl->ctx, ssl->certName, SSL_FILETYPE_PEM)<=0) ERR_print_errors_fp(stderr); }
Oops I was only looking at the unix code assuming it would be used for Mac OS as well. In the Mac code there is /* Disable cert verification since we do that ourselves */ status = SSLSetEnableCertVerify(ssl->ctx, false); I don't understand the comment. But reading it I think this would "solve" the problem as well :) Norbert
Hi,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
I spend some time yesterday trying it. With a linux installation I could issue a client connection and that didn't throw an error. Then Holger was really helpful with Mac OS. He just compiled the Mac plugin using the unix openssl sources. I just copied that in the VM folder and then I could do the same on my Mac. I tried to send a push message via apples push server using: | deviceId payload ip stream notification | payload := '{ "aps" : { "alert" : "Pharo finally got it!" } }'. deviceId := 'XXX'. notification := ByteArray streamContents: [ :str | str nextPut: 1; nextPutAll: (1 asPaddedByteArray: 4); nextPutAll: ((DateAndTime now + 1 day) asUTC asUnixTime asByteArray); nextPutAll: (32 asPaddedByteArray: 2); nextPutAll: (ByteArray readHexFrom: deviceId); nextPutAll: (payload size asPaddedByteArray: 2); nextPutAll: payload asByteArray ]. ip := NetNameResolver addressForName: 'gateway.push.apple.com' timeout: 30. stream := ZdcSecureSocketStream openConnectionToHost: ip port: 2195 timeout: 30. stream binary; shouldSignal: true; autoFlush: false; bufferSize: 4096; timeout: 30. stream sslSession enableLogging; certificateName: '/Users/norbert/multiprod.pem'. stream connect; nextPutAll: notification; flush; close. That is working and I receive the message on my phone. So basically the client certificate stuff seems to work. The awkward thing about it is that you have to specify a filename for the cert. I have the certificates in a database and writing a file everytime I want to send something is not that good. Especially not if there are concurrent requests for sending messages. There is one constraint for this to work. You specify a filename for certificate. In the file you need to have certificate and key. The plugin reads both from the same file. There is no code for specifying a CA chain. So this is resolved system wide and that means you need to install every CA for your certificate in the system. Norbert
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
Wow, thanks for sharing! Please, please, Holger, share the new Mac SSL Plugin somewhere so that we can test it too. (Next step is to push this change upstream in the VM).
On 15 Jan 2016, at 14:49, Norbert Hartl <norbert@hartl.name> wrote:
Hi,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
I spend some time yesterday trying it. With a linux installation I could issue a client connection and that didn't throw an error. Then Holger was really helpful with Mac OS. He just compiled the Mac plugin using the unix openssl sources. I just copied that in the VM folder and then I could do the same on my Mac. I tried to send a push message via apples push server using:
| deviceId payload ip stream notification | payload := '{ "aps" : { "alert" : "Pharo finally got it!" } }'. deviceId := 'XXX'.
notification := ByteArray streamContents: [ :str | str nextPut: 1; nextPutAll: (1 asPaddedByteArray: 4); nextPutAll: ((DateAndTime now + 1 day) asUTC asUnixTime asByteArray); nextPutAll: (32 asPaddedByteArray: 2); nextPutAll: (ByteArray readHexFrom: deviceId); nextPutAll: (payload size asPaddedByteArray: 2); nextPutAll: payload asByteArray ].
ip := NetNameResolver addressForName: 'gateway.push.apple.com' timeout: 30. stream := ZdcSecureSocketStream openConnectionToHost: ip port: 2195 timeout: 30. stream binary; shouldSignal: true; autoFlush: false; bufferSize: 4096; timeout: 30. stream sslSession enableLogging; certificateName: '/Users/norbert/multiprod.pem'. stream connect; nextPutAll: notification; flush; close.
That is working and I receive the message on my phone. So basically the client certificate stuff seems to work. The awkward thing about it is that you have to specify a filename for the cert. I have the certificates in a database and writing a file everytime I want to send something is not that good. Especially not if there are concurrent requests for sending messages.
There is one constraint for this to work. You specify a filename for certificate. In the file you need to have certificate and key. The plugin reads both from the same file. There is no code for specifying a CA chain. So this is resolved system wide and that means you need to install every CA for your certificate in the system.
Norbert
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
On 15 Jan 2016, at 15:09, Sven Van Caekenberghe <sven@stfx.eu> wrote:
Wow, thanks for sharing!
Please, please, Holger, share the new Mac SSL Plugin somewhere so that we can test it too. (Next step is to push this change upstream in the VM).
Norbert, Holger, I think an SSL Plugin for OS X based on the OpenSSL code for (L|U)nix is *very* important. Could you please share it, just for testing, please, please ? Thx, Sven
On 15 Jan 2016, at 14:49, Norbert Hartl <norbert@hartl.name> wrote:
Hi,
Am 12.01.2016 um 16:25 schrieb Sven Van Caekenberghe <sven@stfx.eu>:
Given a ZdcSecureSocketStream you can access the #sslSession. In this session object you can use #certificateName: to set the path or name of the certificate (before you #connect !). That is the general idea.
Now, I don't know if this works or not. Be prepared to look in the plugin C code! On Linux this will probably work.
And please let us know how it goes ;-)
I spend some time yesterday trying it. With a linux installation I could issue a client connection and that didn't throw an error. Then Holger was really helpful with Mac OS. He just compiled the Mac plugin using the unix openssl sources. I just copied that in the VM folder and then I could do the same on my Mac. I tried to send a push message via apples push server using:
| deviceId payload ip stream notification | payload := '{ "aps" : { "alert" : "Pharo finally got it!" } }'. deviceId := 'XXX'.
notification := ByteArray streamContents: [ :str | str nextPut: 1; nextPutAll: (1 asPaddedByteArray: 4); nextPutAll: ((DateAndTime now + 1 day) asUTC asUnixTime asByteArray); nextPutAll: (32 asPaddedByteArray: 2); nextPutAll: (ByteArray readHexFrom: deviceId); nextPutAll: (payload size asPaddedByteArray: 2); nextPutAll: payload asByteArray ].
ip := NetNameResolver addressForName: 'gateway.push.apple.com' timeout: 30. stream := ZdcSecureSocketStream openConnectionToHost: ip port: 2195 timeout: 30. stream binary; shouldSignal: true; autoFlush: false; bufferSize: 4096; timeout: 30. stream sslSession enableLogging; certificateName: '/Users/norbert/multiprod.pem'. stream connect; nextPutAll: notification; flush; close.
That is working and I receive the message on my phone. So basically the client certificate stuff seems to work. The awkward thing about it is that you have to specify a filename for the cert. I have the certificates in a database and writing a file everytime I want to send something is not that good. Especially not if there are concurrent requests for sending messages.
There is one constraint for this to work. You specify a filename for certificate. In the file you need to have certificate and key. The plugin reads both from the same file. There is no code for specifying a CA chain. So this is resolved system wide and that means you need to install every CA for your certificate in the system.
Norbert
On 12 Jan 2016, at 16:05, Norbert Hartl <norbert@hartl.name> wrote:
Is there a way to make SSL connections to the outside world using client certificates from pharo?
thanks,
Norbert
participants (4)
-
Marcus Denker -
Norbert Hartl -
stepharo -
Sven Van Caekenberghe