How to implement an interactive TrustManager on top of default/system TrustMaterial

[henryju]

Hi,

First of all thanks for this library, this looks really useful.

We have started using it, and trying to implement the following use case: we want our (Apache) Http client to trust JDK and OS certificates by default. Then if a request is made on a server having an untrusted certificate, then we want to interactively prompt the user if the certificate should be trusted. If the user says yes, then we want to proceed with the request, and that all other requests to the same server work. We will also save the certificate in our application truststore, to not ask again the next time.

My source of inspiration is the ConfirmingTrustManager from the IntelliJ Platform.

We were trying to do something like:

var truststore = loadOrCreateKeyStoreFromFile("path/to/ourtruststore.jks");

var confirmingTrustManager = new X509TrustManager() {

  @Override
  public void checkServerTrusted(X509Certificate[] certificates, String s) throws CertificateException {
    var trustedByUser = askUserIfTrusted(certificates[0]);
    if (trustedByUser) {
      addCertificateToTrustStore(truststore, certificates[0]);
      reloadSslContext();
    } else {
      throw new CertificateException("User rejected the certificate");
    }
  }

  @Override
  public X509Certificate[] getAcceptedIssuers() {
    return new X509Certificate[] {FAKE_CERTIFICATE};
  }
}

var sslFactory = SSLFactory.builder()
      .withDefaultTrustMaterial()
      .withSystemTrustMaterial()
      .withTrustMaterial(truststore)
      .withTrustMaterial(confirmingTrustManager)
      .withSwappableIdentityMaterial()
      .withSwappableTrustMaterial()
      .build();

We faced the following problems:

1. our confirmingTrustManager was completely ignored because it was initially returning an empty list of getAcceptedIssuers. This was really difficult to debug, and we ended up returning a fake one.
2. reloading the entire SSLFactory "just" to add a new certificate seems very inefficient/slow
3. we could not find a way to make the process work in case of concurrent requests. We would like the first request to trigger the prompt to the user, with other requests waiting, and then if the user accepts the certificate, all requests should be successful

Is there a better way to handle this use case?

[Hakky54]

Hi Julien,

This is an interesting usecase. I think it can be made possible to work in your situation with the current version however the setup would be slightly different. You already mentioned that recreating the OS and JDK trustmaterial is waste of resources as it needs to be recreated, but that can be cached and you don't need to recreate it actually. But I will give you an example.

our confirmingTrustManager was completely ignored because it was initially returning an empty list of getAcceptedIssuers. This was really difficult to debug, and we ended up returning a fake one.

Yes, it will be ignored as the underlying utility will skip empty trustmanagers. A workaround for that kind of use case would be having an initial trustmanager with a dummy certificate, but I can also provide an alternative for your use casse.

reloading the entire SSLFactory "just" to add a new certificate seems very inefficient/slow

Yess, this is also valid. Although it is not a performance hit it is a waste of resources.

So coming back to provide a solution for you, I might be able to add this special type of Trustmanager to the library. That will save you the verbosity. An alternative if you just want to proceed on your side without waiting on the changes of the library would be something like this, beaware it will be more low-level coding.

Optional<X509ExtendedTrustManager> systemTrustManager = TrustManagerUtils.createTrustManagerWithSystemTrustedCertificates();
X509ExtendedTrustManager jdKTrustManager = TrustManagerUtils.createTrustManagerWithJdkTrustedCertificates();
X509TrustManager confirmingTrustManager = new X509TrustManager() {

    // your custom logic

};

X509ExtendedTrustManager combinedTrustManager = TrustManagerUtils.combine(systemTrustManager.get(), jdKTrustManager, confirmingTrustManager);

SSLFactory sslFactory = SSLFactory.builder()
        .withTrustMaterial(combinedTrustManager)
        .build();

In the above example you don't need the swappable option as you have your own trustmanager which is able to include new trusted certificates on the fly.

Can you give it a try on your side?

I will also check wether I can implement a way to provide a similar behaviour as what Intellij does..

Update
I started with an initial implementation, but it is stil a WIP. You can check the progress here #338

I added a basic mechanism within an inflatable trustmanager which has the capability of trusting additional single or more certificates on the fly. I only need to create an abstract layer within SSLFactory and TrustManagerUtils. Please have a look at it, I would love to get your feedback on it. Basically it can look like this:

SSLFactory sslFactory = SSLFactory.builder()
        .withDefaultTrustMaterial()
        .withSystemTrustMaterial()
        .withInflatableTrustMaterial()
        .build();

// After some point in time
List<X509Certificate> certificates =  ...; // List containing your additional newly trusted certificates
TrustManagerUtils.addCertificate(sslFactory.getTrustManager().get(), certificates);

[henryju]

Hi Hakan,

Thank you a lot for looking into this.

FYI, here is our current implementation, in order to use your library for loading OS + JDK certificates, but keeping the ability to intercept untrusted certificates and add them to the truststore after the user had approved them. We are pretty happy with it, but this is quite verbose, and we had to "hack" to return a fake certificate so that our ConfirmingTrustManager is not ignored.

    var confirmingTrustManager = new ConfirmingTrustManager();
    var sslFactory = SSLFactory.builder()
      .withDefaultTrustMaterial()
      .withSystemTrustMaterial()
      .withTrustMaterial(confirmingTrustManager)
      .build();

class ConfirmingTrustManager implements X509TrustManager {
  private final MutableTrustManager mutableTrustManager;
  
  @Override
  public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    synchronized (this) { // The synchronize is very important to support concurrent requests
      try {
        mutableTrustManager.checkServerTrusted(chain, authType);
      } catch (CertificateException e) {
        var isTrustedByClient = askClientIfTrusted(chain);
        if (isTrustedByClient) {
          final X509Certificate endPoint = chain[0];
          mutableTrustManager.addCertificate(endPoint);
        } else {
          throw e;
        }
      }
    }
  }

  @Override
  public X509Certificate[] getAcceptedIssuers() {
    // We have to return a certificate here or our TrustManager will be ignored by SSLContext
    // See https://github.com/Hakky54/sslcontext-kickstart/discussions/337
    return new X509Certificate[] {FAKE_CERTIFICATE};
  }
}

MutableTrustManager is inspired by https://github.com/JetBrains/intellij-community/blob/f4fc17b0c44d38d65fb7ad47b968bed55c889609/platform/platform-api/src/com/intellij/util/net/ssl/ConfirmingTrustManager.java#L313

I had a look at your PR, and I really like the concept of InflatableTrustMaterial. Still, I don't see how we can use it, because we need our "interactive step" to "encapsulate" this inflatable TrustManager, so that concurrent HTTP requests do not lead to asking the user multiple times in a row to trust the same certificate.

If you are interested to see our code (still WIP, needs more tests and polish), here it is:
https://github.com/SonarSource/sonarlint-core/blob/f1bc01709b8e26c1170681cf035e8084cf57c251/core/src/main/java/org/sonarsource/sonarlint/core/http/HttpClientManager.java#L63

[Hakky54]

It seems like we are moving in the right direction with the PR. Thank you for sharing your code changes as it gives me a better idea of what you are trying to do. You just want to prompt the user when a certificate is currently not trusted and ask the user whether is should be trusted or not. If trusted we need to include it into the trustmanager so it will be trusted for the next sslsession and also flush it away to the filesystem. I have adjusted the PR, can you have a look at it and maybe try the snapshot out locally?

The current PR results into the following SSLFactory confguration:

SSLFactory sslFactory = SSLFactory.builder()
        .withDefaultTrustMaterial()
        .withSystemTrustMaterial()
        .withInflatableTrustMaterial(Paths.get("/path/to/truststore.p12"), "secret".toCharArray(), "PKCS12",
                (X509Certificate[] chain, String authType) -> {
                    boolean isTrustedByClient = true; // prompt user to get the actual value
                    return isTrustedByClient;
                })
        .build();

The values withInflatableTrustMaterial is optional, all of the values can be dropped and the data will not be writted to the filesystem. If you provide all of the values it will be written to the files system. If the file already exists, then it will read and update that one.

[Hakky54]

This option is now available in the latest version, so I am closing this discussion. If anyone else is founding out this discussion, the documentation of the usage can be found here: Trust additional new certificates at runtime