bityard Blog

// Experiences with Java and X.509 Certificates - Certificate Revokation Lists

As already mentioned in the article Experiences with Java and X.509 Certificates - Code Signing i recently had the task of researching two issues involving Java and X.509 certificates. While i'm familiar with X.509 certificates, i'm not too familiar with the inner workings of Java, the Java run-time environment or let alone programming in Java. So this was a good opportunity to familiarize myself with the inner workings of Java and also a great learning experience.

The second issue, which this blog post will be about, was in the area of TLS-secured HTTPS network connections, naturally involving a X.509 certificate on the server side. In particular the server processes of two Java application servers were connecting the application logic of two different systems via an API over a HTTPS based network connection. The source system would make RPC calls to an API on the target system in order to store data in and retrieve data from this system.

The communication between the two systems would work fine while using an unencrypted HTTP based network connection, but would fail while using a secured HTTPS network connections. The errors reported back to the server process on the source system which was initiating the communication, would point in the direction of an issue with the X.509 certificate that was being used on the target server. The error message would look something like this:

%% Invalidated:  [Session-1, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256]
main, SEND TLSv1.2 ALERT:  fatal, description = certificate_unknown
main, WRITE: TLSv1.2 Alert, length = 2
[Raw write]: length = 7
0000: 15 03 03 00 02 02 2E                               .......
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException:
  PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find
  valid certification path to requested target
  Exception Failed to access the WSDL at: https://hostname:port/url/rpc-call. It failed with: 
        sun.security.validator.ValidatorException: PKIX path building failed:
            sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification
                path to requested target.
x.y.z.ExceptionHandler
        at x.y.z.method1(source1.java:line-number)
        at x.y.z.method2(source2.java:line-number)
        at x.y.z.method3(source3.java:line-number)

Checking the certificate used on the side of the target server would turn up nothing unusual. The certificate was issued by the internal CA of our infrastructure and like many other certificates looked valid. Verifiying this, by initiating a non-Java based HTTPS request (e.g. with a browser) to the target system worked fine. The certificate on the side of the target server was accepted and a secured HTTPS connection was successfully established. The next debugging step was to check the certstore on the side of the source server which was initiating the connection. It showed that a valid entry, containing the full chain of the root and the intermediate certificates of our internal CA, existed.

From an infrastructure point of view, everything looked reasonably good and should work just fine. Still, the connection between the two systems was failing.

Unfortunately, but inevitably the Java routines responsible for initiating the HTTPS connection on the source system were part of and thus embedded in a programming logic more complex than necessary or useful for the purpose of low-level debugging. For further debugging i wanted a Java-based test program which was stripped down to the bare minimum of initiating a HTTPS connection. Again, not being a seasoned Java programmer, i managed to cobble together the following test program from several examples on the internet:

CertChecker.java
import java.net.*;
import java.io.*;
import javax.net.ssl.*;
import java.security.GeneralSecurityException;
import java.security.KeyStore;
import java.security.SecureRandom;
import java.security.cert.*;
import java.util.*;
 
public class CertChecker implements X509TrustManager {
    public static void main(String[] args) throws Exception {
        try {
            SSLContext sc = SSLContext.getInstance("SSL");
            sc.init(null, new TrustManager[]{new CertChecker()}, new SecureRandom());
            SSLSocketFactory factory = (SSLSocketFactory)SSLSocketFactory.getDefault();
            SSLSocket socket = (SSLSocket)factory.createSocket("hostname", port);
            socket.startHandshake();
 
            PrintWriter out = new PrintWriter(
                                  new BufferedWriter(
                                  new OutputStreamWriter(
                                  socket.getOutputStream())));
 
            out.println("GET / HTTP/1.0");
            out.println();
            out.flush();
 
            if (out.checkError())
                System.out.println("SSLSocketClient:  java.io.PrintWriter error");
 
            BufferedReader in = new BufferedReader(
                                    new InputStreamReader(
                                    socket.getInputStream()));
 
            String inputLine;
            while ((inputLine = in.readLine()) != null)
                System.out.println(inputLine);
 
            in.close();
            out.close();
            socket.close();
 
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
 
    private final X509TrustManager defaultTM;
 
    public CertChecker() throws GeneralSecurityException {
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        tmf.init((KeyStore) null);
        defaultTM = (X509TrustManager) tmf.getTrustManagers()[0];
    }
 
    public void checkServerTrusted(X509Certificate[] certs, String authType) {
        if (defaultTM != null) {
            try {
                defaultTM.checkServerTrusted(certs, authType);
                Set<TrustAnchor> trustAnchors = getTrustAnchors();
                System.out.println("Certificate valid");
            } catch (CertificateException ex) {
                System.out.println("Certificate invalid: " + ex.getMessage());
            }
        }
    }
 
    private Set<TrustAnchor> getTrustAnchors() {
        X509Certificate[] acceptedIssuers = defaultTM.getAcceptedIssuers();
        Set<TrustAnchor> trustAnchors = new HashSet<TrustAnchor>();
        for (X509Certificate acceptedIssuer : acceptedIssuers) {
            TrustAnchor trustAnchor = new TrustAnchor(acceptedIssuer, null);
            trustAnchors.add(trustAnchor);
        }
        return trustAnchors;
    }
 
    public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    }
 
    public X509Certificate[] getAcceptedIssuers() {
        return null;
    }
}

In line 16 of the above Java program, the placeholders hostname and port were replaced by the appropriate values for the target server system. The code was then compiled and run against the target server system, with the following Java command line call:

user@host:$ java -Djava.security.debug=all -Djavax.net.debug=all CertChecker

The result from this was the following error message output:

[...]
certpath: BasicChecker.updateState issuer: CN=root-ca, DC=domain, DC=tld; subject: CN=intermediate-ca, DC=domain, DC=tld; serial#: ...
certpath: -checker6 validation succeeded
certpath: -Using checker7 ... [sun.security.provider.certpath.RevocationChecker]
certpath: RevocationChecker.check: checking cert
  SN:     26000000 0cff68dd 06c50fb0 4a000100 00000c
  Subject: CN=intermediate-ca, DC=domain, DC=tld
  Issuer: CN=root-ca, DC=domain, DC=tld
certpath: RevocationChecker.checkCRLs() ---checking revocation status ...
certpath: RevocationChecker.checkCRLs() possible crls.size() = 0
certpath: RevocationChecker.checkCRLs() approved crls.size() = 0
certpath: DistributionPointFetcher.getCRLs: Checking CRLDPs for CN=intermediate-ca, DC=domain, DC=tld
certpath: Trying to fetch CRL from DP http://crl-fqdn/filename.crl
certpath: CertStore URI:http://crl-fqdn/filename.crl
certpath: Downloading new CRL...
certpath: Exception fetching CRL:
java.security.cert.CRLException: Empty input
java.security.cert.CRLException: Empty input
        at sun.security.provider.X509Factory.engineGenerateCRL(X509Factory.java:397)
        at java.security.cert.CertificateFactory.generateCRL(CertificateFactory.java:497)
        at sun.security.provider.certpath.URICertStore.engineGetCRLs(URICertStore.java:419)
        at java.security.cert.CertStore.getCRLs(CertStore.java:181)
        at sun.security.provider.certpath.DistributionPointFetcher.getCRL(DistributionPointFetcher.java:245)
        at sun.security.provider.certpath.DistributionPointFetcher.getCRLs(DistributionPointFetcher.java:189)
        at sun.security.provider.certpath.DistributionPointFetcher.getCRLs(DistributionPointFetcher.java:121)
        at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:552)
        at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:465)
        at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:367)
        at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:337)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
        at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
        at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
        at sun.security.validator.Validator.validate(Validator.java:260)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at CertChecker.main(CertChecker.java:31)
[...]
%% Invalidated:  [Session-1, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256]
main, SEND TLSv1.2 ALERT:  fatal, description = certificate_unknown
main, WRITE: TLSv1.2 Alert, length = 2
[Raw write]: length = 7
0000: 15 03 03 00 02 02 2E                               .......
main, called closeSocket()
main, handling exception: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: \
    PKIX path validation failed: java.security.cert.CertPathValidatorException: Could not determine revocation status
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path validation failed: \
    java.security.cert.CertPathValidatorException: Could not determine revocation status
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
        at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302)
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1514)
        at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:216)
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026)
        at sun.security.ssl.Handshaker.process_record(Handshaker.java:961)
        at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
        at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
        at CertChecker.main(CertChecker.java:31)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: \
    java.security.cert.CertPathValidatorException: Could not determine revocation status
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:352)
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:249)
        at sun.security.validator.Validator.validate(Validator.java:260)
        at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
        at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1496)
        ... 8 more
Caused by: java.security.cert.CertPathValidatorException: Could not determine revocation status
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:219)
        at sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:140)
        at sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:79)
        at java.security.cert.CertPathValidator.validate(CertPathValidator.java:292)
        at sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:347)
        ... 14 more
Caused by: java.security.cert.CertPathValidatorException: Could not determine revocation status
        at sun.security.provider.certpath.RevocationChecker.buildToNewKey(RevocationChecker.java:1092)
        at sun.security.provider.certpath.RevocationChecker.verifyWithSeparateSigningKey(RevocationChecker.java:910)
        at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:577)
        at sun.security.provider.certpath.RevocationChecker.checkCRLs(RevocationChecker.java:465)
        at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:367)
        at sun.security.provider.certpath.RevocationChecker.check(RevocationChecker.java:337)
        at sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
        ... 19 more

The Java exceptions shown above suggested an issue with the Certificate Revokation List (CRL) of the root CA. This was confusing, since the certificate worked when the target server was accessed with a browser. Manually downloading the CRL was also possibly. Verifying the downloaded CRL with the help of OpenSSL showed no anomalies to which this issue could be attributed.

The line:

java.security.cert.CRLException: Empty input

from the above error messages, lead the way to solve this issue. This line could be interpreted in two ways:

  1. either there was a field in the CRL currently being parsed that had an empty or an unexpected value leading to a parse error
  2. or the downloaded CRL data which was handed over to the CRL parsing code was an empty set.

I decided to first check option number 2 and – if this turned out to be not the case – move on from there to check option number 1. Since a manual download of the CRL worked per se, i decided to take a closer look at the download process with the curl command line utility:

user@host:$ curl -v http://crl-fqdn/filename.crl
* Hostname was NOT found in DNS cache
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.0.0.15...
* Connected to crl-fqdn (10.0.0.15) port 80 (#0)
> GET /filename.crl HTTP/1.1
> User-Agent: curl/7.38.0
> Host: crl-fqdn
> Accept: */*
> 
* HTTP 1.0, assume close after body
< HTTP/1.0 302 Found
< Location: https://crl-fqdn/filename.crl
< Server: loadbalancer-vendor
* HTTP/1.0 connection set to keep alive!
< Connection: Keep-Alive
< Content-Length: 0
< 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #0 to host crl-fqdn left intact

The HTTP request for the CRL file was in fact answered with a HTTP status code 302 and a new location for the CRL file, redirecting any client to a HTTPS secured location. What is apparently working for any regular browser or – with the “-L” command line option – even for the curl command line utility, seemed to be an issue for the part of the JRE that is handling the download of the CRL. Instead of following the HTTP redirect and requesting the CRL from the alternate location, the CRL-handling code within the JRE is given the content data that is returned from the initial HTTP request which is an empty set (see the Content-Length: 0 in the above curl output). This in turn leads to the failure of parsing the CRL, which leads to the failure verifying the certificate against the CRL, which leads to the failure of not establishing a TLS-secured HTTPS network connection between the source and the target server system.

The immediate and simple solution to this issue was to add an exception to the HTTP-to-HTTPS redirect for the URL of the CRL. This global HTTP-to-HTTPS redirect was implemented a while ago upon request by the information security department as an attempt make the site serving the CRL and other content more secure. Arguably serving CRLs with HTTPS is generally a bad idea, since it can cause a chicken-and-egg type of problem. In this case though it would have worked, since the host serving the CRL was protected by a certificate from a different CA than the one for which the CRL was being provided.

Leave a comment…




N Q T Q R
  • E-Mail address will not be published.
  • Formatting:
    //italic//  __underlined__
    **bold**  ''preformatted''
  • Links:
    [[http://example.com]]
    [[http://example.com|Link Text]]
  • Quotation:
    > This is a quote. Don't forget the space in front of the text: "> "
  • Code:
    <code>This is unspecific source code</code>
    <code [lang]>This is specifc [lang] code</code>
    <code php><?php echo 'example'; ?></code>
    Available: html, css, javascript, bash, cpp, …
  • Lists:
    Indent your text by two spaces and use a * for
    each unordered list item or a - for ordered ones.
This website uses cookies for visitor traffic analysis. By using the website, you agree with storing the cookies on your computer.More information