Troubleshoot certificate errors

Each certificate has a Subject field and an Issuer field. For a certificate to be trusted, its issuer must match the subject of another certificate, forming a chain that eventually leads to a trusted root CA. If a server fails to provide the complete chain of certificates (e.g., it sends its own certificate but not the intermediate CA's certificate), clients may not be able to validate it, even if they trust the root CA.

This is a common problem when using certificates from a corporate PKI or a new public CA. The tls.crt file configured for Deephaven should contain not only the server certificate but also all required intermediate certificates, concatenated in order.

• If the certificate has expired, the Deephaven servers must be updated with new certificates. See Replacing certificates.

This error occurs when the hostname you are using to access a service (e.g., my.deephaven.server.com) does not match the Common Name (CN) or any of the Subject Alternative Names (SANs) listed in the server's SSL/TLS certificate. Certificates are issued for specific hostnames to prevent man-in-the-middle attacks.

Common Symptoms:

• Browser errors like "NET::ERR_CERT_COMMON_NAME_INVALID" (which can be misleading if the Subject Alternative Name (SAN) is the actual issue, as modern clients prioritize SAN over Common Name (CN)) or "Your connection is not private."
• Application-level errors indicating a certificate validation failure related to hostname.
• javax.net.ssl.SSLPeerUnverifiedException: Certificate for <hostname> doesn't match any of the subject alternative names: [list...]

Troubleshooting Steps:

1. Verify the Hostname: Double-check the exact hostname you are using in your browser URL, application configuration, or client connection string.
2. Inspect the Certificate:
   • Using a Browser: Click the padlock icon in the browser's address bar when connected to the HTTPS service, view certificate details, and look for the "Common Name" (CN) and "Subject Alternative Name" (SAN) fields. Ensure the hostname you're using is listed in one of these, paying close attention to the SAN entries, as modern browsers and applications prioritize these over the CN.
   • Using openssl: Alternatively, use openssl to check the CN and SAN. The grep will show Subject: CN= and X509v3 Subject Alternative Name:. If the SAN field is missing or doesn't contain the correct hostname, this is the likely cause of the error, even if the CN appears correct.

     openssl s_client -connect your.server.com:443 -servername your.server.com | openssl x509 -noout -text | grep -E 'Subject: CN|X509v3 Subject Alternative Name:'
     

     Replace your.server.com with the actual hostname and 443 with the correct port if not standard HTTPS. To see all certificate details, including Key Usage and EKU, remove the | grep ... part.
   • Using curl (especially for non-browser clients or to see connection details):

     curl -vvI https://your.server.com
     

     Replace your.server.com with the actual hostname. Examine the output for SSL handshake details (look for lines starting with * SSL) and the certificate presented by the server (lines starting with * Server certificate:). curl can also reveal if the server is presenting an unexpected certificate or if there are other TLS handshake issues.
   • Verify Key Usage and Extended Key Usage (EKU): Ensure the certificate is valid for its intended purpose. For a web server, it should typically have:
     • Key Usage: Digital Signature, Key Encipherment
     • Extended Key Usage (EKU): TLS Web Server Authentication (Less commonly, TLS Web Client Authentication might also be present or required in specific mutual TLS setups, but its absence is rarely an issue for standard web server access). You can see these details in the browser's certificate viewer or in the full output of the openssl x509 -text command.
3. Resolution:
   • Correct the Hostname: If you are using an incorrect hostname to access the service, update your client or configuration to use one that is listed in the certificate (often the CN or a SAN).
   • Reissue the Certificate: If the certificate does not cover all necessary hostnames (e.g., you've added a new alias for the server), the certificate needs to be reissued with the correct CN and/or additional SANs. Consult your certificate authority (CA) or internal PKI team for this process.
   • Wildcard Certificates: If applicable, ensure your wildcard certificate (e.g., *.deephaven.example.com) correctly covers the hostname in question. Remember that wildcards usually only cover one subdomain level.

This issue occurs when the certificate presented by the server is signed by a Certificate Authority (CA)—or is self-signed—and is not present in the client's trust store. This is the standard behavior for certificates issued by an internal corporate PKI or any other custom CA, and is always the case for self-signed certificates used in production-like environments.

Common Symptoms:

• Browser errors like "NET::ERR_CERT_AUTHORITY_INVALID".
• Application-level errors like javax.net.ssl.SSLHandshakeException: PKIX path building failed.

Troubleshooting Steps:

To resolve this, the client system (e.g., a user's web browser, a client application, or another Deephaven service) must be configured to trust the custom CA. This typically involves importing the CA's root and/or intermediate certificates into the appropriate trust store.

The location of the trust store and the import method depend on the client's operating system and application stack.

Microsoft Windows

Windows has a system-wide trust store. Certificates can be managed via certlm.msc (for the local machine) or certmgr.msc (for the current user). Import the CA certificate into the "Trusted Root Certification Authorities" store.

macOS

On macOS, certificates are managed in the Keychain Access application. You can import CA certificates into the "System" keychain to make them available to all users and most applications, or the "login" keychain for just the current user.

After importing, you must explicitly set the trust level for the certificate. Open the certificate, expand the "Trust" section, and set "When using this certificate" to "Always Trust".

# Add the certificate to the System keychain
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain /path/to/your/ca.crt

Java Truststores (cacerts)

Java applications, including Deephaven services, use their own truststore by default, often called cacerts. If a Deephaven service needs to connect to another service secured by a private CA, you may need to import the CA certificate into the Java truststore.

The default location of the root CA truststore included with a Java distribution is <java_home_directory>/lib/security/cacerts. The default password for this truststore is changeit.

To import a private CA certificate to the root CA truststore, use the Java keytool utility:

keytool -cacerts -trustcacerts -import -file <certificate file to import> -alias <a unique (to the truststore) alias to use for this certificate>

A few additional important considerations for PKI and custom CA setups:

• Certificate File Format: Deephaven expects server certificates (tls.crt) and private keys (tls.key) to be in PEM (Privacy Enhanced Mail) format. PEM files are Base64 encoded ASCII files and typically have extensions like .pem, .crt, .cer, or .key. If your certificate or key is in a binary format (like DER), you'll need to convert it to PEM using a tool like openssl (e.g., openssl x509 -inform der -in certificate.der -out certificate.pem).
• Complete Certificate Chain in tls.crt: When using certificates issued by a corporate PKI or a custom CA (i.e., not a publicly trusted root CA), the tls.crt file presented by the Deephaven server should ideally contain not only the server's end-entity certificate but also all necessary intermediate CA certificates. This provides the complete chain to the client, minimizing the need for clients to have all intermediates pre-installed. If the chain is incomplete (e.g., tls.crt only contains the server cert but not the intermediates), clients might fail validation even if they trust the root CA. This can manifest as an "Untrusted CA" or "PKIX path building failed" error.

Note that the -keystore argument can be used (instead of -cacerts) to indicate a particular installation's truststore. This should be done in cases where there is more than one installation of Java, and the one to be updated for the Launcher is not the default one used in a shell console.

The Deephaven Launcher behaves much like a browser when creating or updating a Deephaven instance. It must rely on the system or Java truststores to make the initial HTTPS connection to the client update server. You can follow the instructions in the troubleshooting section above to correct trust issues.

When creating a new instance, if the server certificate can't be verified, the Launcher will display a security warning similar to the warnings shown by most browsers. In this dialog, you can view the certificate chain. Optionally, you can ignore the security warning temporarily in order to download the starting files for the new instance.

When the Deephaven Launcher starts the Deephaven console program, that application uses a truststore that has been configured for the Deephaven system, and has been downloaded with the instance program files. Once that truststore file has been downloaded, the Launcher is able to use that file to make further connections to the system.

The Deephaven Launcher will look for an optional truststore in the program directory. The expected file names for the truststore and password file are truststore-iris.p12 and truststore_password (matching the default names used in the Deephaven system). The names and locations can be changed with properties:

IrisConfigurationLauncher.truststore
IrisConfigurationLauncher.truststore.passphrase.file

See Deephaven Launcher properties files for more information about locations and content.

This file can be created from source certificate(s) or downloaded from the target Deephaven system.

This is one method for downloading the files:

scp yourserver.company.com:/etc/sysconfig/illumon.d/resources/truststore-iris.p12 .
scp yourserver.company.com:/etc/sysconfig/illumon.d/resources/truststore_passphrase .

Then copy the two files to the Deephaven Launcher program directory, or to the location you configured in a properties file.

The following steps will create a PKCS12 file on macOS or linux:

• Create a password file:

  echo -n "your password" | base64 > my_truststore_passphrase
  

• Import certificate "myCA.pem" with alias "your_server". You can omit the -storepass argument and type the password instead.

  keytool -importcert -keystore my-truststore-iris.p12 -storepass $(echo my_truststore_passphrase |base64 -d) -alias your_server -file myCA.pem
  

A new, default hashing algorithm was introduced. This means that an "old" JRE without that algorithm cannot read a PKCS12 file written by a "new" JRE that uses the new default.

The following errors might appear in the Deephaven Launcher log file:

• keytool error: java.io.IOException: Invalid keystore format
• keytool error: java.io.IOException: Integrity check failed: java.security.NoSuchAlgorithmException: Algorithm HmacPBESHA256 not available
• parseAlgParameters failed: ObjectIdentifier() -- data isn't an object ID (tag = 48)

The only remedy is to use a newer JDK version to start the Deephaven Launcher, preferably one that matches the Deephaven Server.

For reference:

• JDK-8267880 Upgrade the default PKCS12 MAC algorithm
• JDK-8267701 Revert the change to the default PKCS12 macAlgorithm and macIterationCount props for 11u/8u/7u

You can verify this problem using keytool:

$ java -version
java version "11.0.4" 2019-07-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.4+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.4+10-LTS, mixed mode)

$ keytool -list -keystore truststore-iris.p12 -storepass $(cat truststore_passphrase |base64 -d)
keytool error: java.io.IOException: Invalid keystore format

$ java -version
openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)

$ keytool -list -keystore truststore-iris.p12 -storepass $(cat truststore_passphrase |base64 -d)
keytool error: java.io.IOException: Integrity check failed: java.security.NoSuchAlgorithmException: Algorithm HmacPBESHA256 not available