jfly jfly - 3 months ago 23
C++ Question

SSL_CTX_use_PrivateKey_file() failed

I'm writing a client application on Windows that establishes an SSL connection to a server, and the server requests client certificate for authentication. The server provides me a .pfx file, then I use openssl command line tool to get the certificate and the private key like this:

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out cert.pem
openssl pkcs12 -in filename.pfx -nocerts -out key.pem


after that, I try to load the certificate and the private key with functions from openssl as below, but
SSL_CTX_use_PrivateKey_file()
always failed, the error message is "
error:0906D06C:PEM routines:PEM_read_bio:no start line
", I can't figure it out why, can anyone give me some enlightenment?
Here is the code.

#include "openssl/ssl.h"
#include "openssl/err.h"
#include <stdio.h>
#include <string>

int InitClientCtx()
{
OpenSSL_add_ssl_algorithms();

SSL_CTX* m_pClientCtx;
m_pClientCtx = SSL_CTX_new(SSLv23_method());

if(!m_pClientCtx)
{
return -1;
}

::SSL_CTX_set_options(m_pClientCtx, SSL_OP_ALL); //for well-known bugs

int nRet = 0;

std::string sCertFilePath = "C:\\cert.pem";

nRet = SSL_CTX_use_certificate_chain_file(m_pClientCtx, sCertFilePath.c_str());

std::string sKeyPassWord = "123456";

SSL_CTX_set_default_passwd_cb_userdata(m_pClientCtx, (void*)(sKeyPassWord.c_str()));

std::string sKeyFilePath = "C:\\key.pem";

// this method returned 0, which means it failed.
nRet = SSL_CTX_use_PrivateKey_file(m_pClientCtx, sKeyFilePath.c_str(), SSL_FILETYPE_PEM);

SSL_load_error_strings();
unsigned long n = ERR_get_error();
char buf[1024];
printf("%s\n", ERR_error_string(n, buf));

nRet = SSL_CTX_check_private_key(m_pClientCtx);
if (nRet <= 0)
{
return -1;
}

/*std::string sCACertFilePath;

nRet = SSL_CTX_load_verify_locations(m_pClientCtx, sCACertFilePath.c_str(), NULL);*/

return 0;
}

int main()
{
InitClientCtx();
return 0;
};

Answer

I've solved this problem myself. I generated the key.pem using OpenSSL for Windows, when the CMD prompts me to type in the pass phrase, I just typed a Enter since I needn't a pass phrase, but the key.pem was invalid(neither BEGIN nor END markers). When I generate the private key in Linux, the terminal prompts I must type a pass phrase and I do. Then I remove the key pass phrase using this command:

openssl rsa -in key.pem -out newkey.pem

After that, I open the key.pem in a text editor, it starts off with -----BEGIN RSA PRIVATE KEY----- and end up with -----END RSA PRIVATE KEY-----. And SSL_CTX_use_PrivateKey_file() just works fine!