Markus L Markus L - 1 month ago 13
Java Question

When using a custom X509KeyManager Java is not able to determine a matching cipher suite for the SSL handshake

I'm working with Java7 and JAX-WS 2.2.

For a SOAP web service I need to create a custom

X509KeyManager
in order to find the correct certificate for each connecting client in a JKS keystore.

However, I'm already struggling to get the my custom key manager running. So far I'm using the default one (retrieved from the initialized
KeyManagerFactory
) and it basically works - but of course it doesn't select the correct certificate. So the first idea was to create a custom
X509KeyManager
which holds the original key manager, only writes out some log messages but generally uses the default behaviour.

For some reason that doesn't work at all. The SLL handshake cannot be established. After the ClientHello the log shows the following error:

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Thread-3, READ: TLSv1 Handshake, length = 149
*** ClientHello, TLSv1
RandomCookie: GMT: 1476877930 bytes = { 207, 226, 8, 128, 40, 207, 47, 180, 146, 211, 157, 64, 239, 13, 201, 92, 158, 111, 108, 44, 223, 136, 193, 251, 33, 202, 7, 90 }
Session ID: {}
Cipher Suites: [TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA]
Compression Methods: { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
***
%% Initialized: [Session-3, SSL_NULL_WITH_NULL_NULL]
Thread-3, fatal error: 40: no cipher suites in common
javax.net.ssl.SSLHandshakeException: no cipher suites in common
%% Invalidated: [Session-3, SSL_NULL_WITH_NULL_NULL]
Thread-3, SEND TLSv1 ALERT: fatal, description = handshake_failure
Thread-3, WRITE: TLSv1 Alert, length = 2
Thread-3, fatal: engine already closed. Rethrowing javax.net.ssl.SSLHandshakeException: no cipher suites in common


I didn't remove any cipher suites at all to my knowledge! And the SSL handshake can be made with the same certificates.

This is my key manager:

public class CustomX509KeyManager extends X509ExtendedKeyManager
{
private static final Logger LOG = Logger.getLogger( CustomX509KeyManager.class );
private final X509KeyManager originalKeyManager;

public CustomX509KeyManager(final X509KeyManager keyManager)
{
super();
this.originalKeyManager = keyManager;
}

@Override
public String chooseServerAlias(final String keyType, final Principal[] issuers,
final Socket socket)
{
final String serverAliases=
this.originalKeyManager.chooseServerAlias( keyType, issuers, socket );
CustomX509KeyManager.LOG.info( "chooseServerAlias() " + serverAliases );
return serverAliases;
}

...
}


The other methods (not shown here) are just calling the corresponding methods in the
originalKeyManager
as well. During testing I never see the log message from the
chooseServerAlias()
method.

And it's initialized from another class in the
getSslContext()
method:

private KeyManager[] getKeyManagers(final KeyManagerFactory keyManagerFactory)
{
final KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();

// replace any X509KeyManager with our own implementation
for ( int i = 0; i < keyManagers.length; i++ )
{
if ( keyManagers[i] instanceof X509KeyManager )
{
keyManagers[i] =
new CustomX509KeyManager( ( X509KeyManager ) keyManagers[i] );
}
}

return keyManagers;
}

public SSLContext getSslContext()
{
// create the KeyStore and load the JKS file
final KeyStore keyStore = createKeyStore();

// initialize key and trust manager factory
final KeyManagerFactory keyManagerFactory =
KeyManagerFactory.getInstance( KeyManagerFactory.getDefaultAlgorithm() );
keyManagerFactory.init( keyStore, "changeit".toCharArray() );
final TrustManagerFactory trustManagerFactory =
TrustManagerFactory.getInstance( TrustManagerFactory.getDefaultAlgorithm() );
trustManagerFactory.init( keyStore );

// initialize the SSL context
final SSLContext sslContext = SSLContext.getInstance( "TLS" );
// sslContext.init( keyManagerFactory.getKeyManagers(),
// trustManagerFactory.getTrustManagers(), new SecureRandom() );
sslContext.init( getKeyManagers( keyManagerFactory ),
trustManagerFactory.getTrustManagers(), new SecureRandom() );
return sslContext;
}


The commented lines show the original usage of the default key manager.

Any idea what's wrong? Why is the behaviour of using my
CustomX509KeyManager
so different than the default key manager that the handshake cannot be done? With the default key manager the encryption is negotiated for the TLS_DHE_RSA_WITH_AES_128_CBC_SHA algorithm which is available with the custom key manager as well but for some reason not chosen.

Update 1



I'm trying to connect with openssl in client mode to the server now but the server encounters the same problem using SSL. When I use the TLS protocol then the additional error message


Unsupported extension type_35, data:


appears.

Update 2



I can confirm that the above notice about unsupported extensions also appears upon successful handshakes so this is a false trace.

Answer

After a few days of trial & error I finally found my mistake!

In Java 7 a custom key manager should extend the X509ExtendedKeyManager which forces you to implement five methods of the interface X509KeyManager. However, there are two additional methods in the class X509ExtendedKeyManager which are not declared as abstract but must be overwritten for proper usage:

  • chooseEngineClientAlias(String[], Principal[], SSLEngine)
  • chooseEngineServerAlias(String, Principal[], SSLEngine)

After overwriting and implementing the methods by delegating the call to my originalKeyManager (which became of type X509ExtendedKeyManager as well) the SSL handshake finally succeeded.

Comments