Peter Cooper Jr. Peter Cooper Jr. - 11 months ago 262
Java Question

How to connect to FTPS server with data connection using same TLS session?

Environment: I'm using Sun Java JDK 1.8.0_60 on 64-bit Windows 7, using Spring Integration 4.1.6 (which internally appears to use Apache Commons Net 3.3 for FTPS access).

I'm attempting to integrate with our application an automatic download from our client's FTPS server. I've done so successfully with SFTP servers using Spring Integration without any trouble for other clients without issues, but this is the first time a client has required us to use FTPS, and getting it to connect has been very puzzling. While in my real application I'm configuring Spring Integration using XML beans, to try to understand what's not working I'm using the following test code (though I'm anonymizing the actual host/username/password here):

final DefaultFtpsSessionFactory sessionFactory = new DefaultFtpsSessionFactory();
sessionFactory.setProtocols(new String[]{"TLSv1.2"});
sessionFactory.setCipherSuites(new String[]{"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"});

final FtpSession session = sessionFactory.getSession();
//try {
final FTPFile[] ftpFiles = session.list("/");
logger.debug("FtpFiles: {}", (Object[]) ftpFiles);
//} catch (Exception ignored ) {}

I'm running this code with
to get all the TLS debugging information printed.

The main "control" connection to the FTPS server works fine, but when it tries to open the data connection for the list (or any other data connection I've tried), I get a Remote host closed connection during handshake
, caused by SSL peer shut down incorrectly
. If I uncomment the swallowing-exceptions catch block around the
command, then I can see (though the output) that the server sent the following message after rejecting the data connection SSL handshake:

main, READ: TLSv1.2 Application Data, length = 129
Padded plaintext after DECRYPTION: len = 105
0000: 34 35 30 20 54 4C 53 20 73 65 73 73 69 6F 6E 20 450 TLS session
0010: 6F 66 20 64 61 74 61 20 63 6F 6E 6E 65 63 74 69 of data connecti
0020: 6F 6E 20 68 61 73 20 6E 6F 74 20 72 65 73 75 6D on has not resum
0030: 65 64 20 6F 72 20 74 68 65 20 73 65 73 73 69 6F ed or the sessio
0040: 6E 20 64 6F 65 73 20 6E 6F 74 20 6D 61 74 63 68 n does not match
0050: 20 74 68 65 20 63 6F 6E 74 72 6F 6C 20 63 6F 6E the control con
0060: 6E 65 63 74 69 6F 6E 0D 0A nection..

What appears to be happening (and this is my first time dealing with FTPS, though I've dealt with plain FTP before) is that the way the server ensures authentication and encryption over both the control and data connections is that after a "normal" TLS connection to establish the control connection and authentication happens there, each data connection requires the client to connect with the same TLS session. This makes sense to me as how it's supposed to be work, but the Apache Commons Net FTPS implementation doesn't seem to be doing that. It seems to be trying to establish a new TLS session, and so the server is rejecting the attempt.

Based on this question about resuming SSL sessions in JSSE, it appears that Java assumes or requires a different session for each host/post combination. My hypothesis is that since the FTPS data connection is on a different port than the control connection, it's not finding the existing session and is trying to establish a new one, so the connection fails.

I see three main possibilities:

  1. The server is not following the FTPS standard in requiring the same TLS session on the data port as on the control port. I can connect to the server fine (using the same host/user/password as I'm trying to use in my code) using FileZilla 3.13.1. The server identifies itself as "FileZilla Server 0.9.53 beta" upon login, so perhaps this is some sort of proprietary FileZilla way of doing things, and there's something odd I need to do to convince Java to use the same TLS session.

  2. The Apache Commons Net client doesn't actually follow the FTPS standard, and only allows some subset that doesn't allow for securing the data connections. This would seem odd, as it appears to be the standard way of connecting to FTPS from within Java.

  3. I'm completely missing something and misdiagnosing this.

I'd appreciate any direction you can provide as to how to connect to this kind of FTPS server. Thank you.

Answer Source

Indeed some FTP(S) servers do require that the TLS/SSL session is reused for the data connection. This is a security measure by which the server can verify that the data connection is used by the same client as the control connection.

Some references for common FTP servers:

What may help you with the implementation is that Cyberduck FTP(S) client does support TLS/SSL session reuse and it uses Apache Commons Net library:

  • - Reuse Session key on data connection

  • See its code (extends Commons Net FTPSClient), particularly its override of _prepareDataSocket_ method:

    protected void _prepareDataSocket_(final Socket socket) throws IOException {
        if(preferences.getBoolean("ftp.tls.session.requirereuse")) {
            if(socket instanceof SSLSocket) {
                // Control socket is SSL
                final SSLSession session = ((SSLSocket) _socket_).getSession();
                final SSLSessionContext context = session.getSessionContext();
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    final String key = String.format("%s:%s", socket.getInetAddress().getHostName(),
                    method.invoke(cache, key, session);
                catch(NoSuchFieldException e) {
                    // Not running in expected JRE
                    log.warn("No field sessionHostPortCache in SSLSessionContext", e);
                catch(Exception e) {
                    // Not running in expected JRE
  • It seems that the _prepareDataSocket_ method was added to Commons Net FTPSClient specifically to allow the TLS/SSL session reuse implementation:

    A native support for the reuse is still pending:

  • You will obviously need to override the Spring Integration DefaultFtpsSessionFactory.createClientInstance() to return your custom FTPSClient implementation with the session reuse support.