If a trust store should contain only root CA certificates (which seems to be recommended from what I have read), how can you limit access to specific parties (and not any party validated by a CA in the trust store).
For more detail:
I have a two java applications - lets call them A and B - that use SSL to secure communications. A and B both have a client and server portion to send (client) and receive (server) messages. The client uses a key stored in client.keystore, the server uses a key in server.keystore, and both use a single trust store to verify the identity of the other app (i.e app A has 3 keystores, app B has 3 keystores).
So far i have used keytools to generate keys for the client and server, signed them (with my own test CA), and loaded the signed certificates back into the keystore. I do this for app A and B. In order to get the SSL handshake to complete, i have found that the truststores need to contain the CA certificate used to sign the other apps keys (so the truststore for app A must contain the CA certificate used to sign app B's client and server keys, and vice versa).
So far this makes sense, but because the trust store contains a root CA certificate, i can generate another set of keys, sign them with the same CA, and have them accepted by the other app - in other words app B will accept a rogue agent that appears like app A, as long as it has keys signed by the root CA.
Does SSL have a mechanism to prevent this? I have tried importing the public keys for the client and server of app A into the trust store of app B (and vice versa), but without the root certificate the SSL handshake will not complete.
Does SSL have a mechanism to prevent this?
No. SSL provides privacy, integrity, and authentication. The peers are who they say they are. What you're talking about is authorization: is this peer a peer that I want to talk to? It's an application responsibility. You can get hold of the peer certificate chain via
SSLSocket.getSession() and examine it during the handshake via a
HandshakeCompletedListener to authorize the peer. If you don't like him, just close the socket. Nothing else can happen on the connection.