Fortinet and Expiring Let’s Encrypt Certificates

Fortinet was made aware by customers in the early hours of September 30^th that TLS connections to web sites using Let’s Encrypt certificates were failing. Our first response was to validate the certificate chain. We discovered that the root CA for Let’s Trust certificates, IdenTrust DST Root CA X3, had expired at 00:00 UTC on September 30^th. 

This was not unexpected—depreciation of this certificate had been planned for some time by Let’s Encrypt as they are in the process of moving to the Self Signed ISRC Root X1 Root CA. In preparation for this, Fortinet had pushed out the new Root CA certificate ISRG Root X1 to FortiGate devices. Any Let’s Encrypt certificates issued since May 2021 using the alternative chain should not experience any issue as they are configured to use the self signed ISRC Root X1 certificate which is already in the FortiGate trust store.

Let’s Encrypt stated that the reason for the cross-sign was to improve compatibility with pre-7.1.1 Android devices. The cross sign still is in place (by default) for new LE issuance (even after the expiration of DST Root CA X3).  The reason this workaround worked for Android Devices is that they do not check the notAfter field of trust anchors.  Mr. Scott Helme has his own description for the cross-signing in his post.

The issue being seen by Fortinet customers is due to Fortinet devices validating the full chain of trust and then invalidating the chain when it sees that the CA IdenTrust DST Root CA X3 is expired, even though the cross-signed ISRG Root X1 root is valid for longer.

We have removed the offending expired certificate from the certificate store, however, this still does not solve the problem due to the Authority Information Access – CA Issuers entry.

Authority Information Access:
CA Issuers - URI: http://apps.identrust.com/roots/dstrootcax3.p7c

This tells the client how to rebuild the chain of trust if the anchor is not available in the local certificate store. So, FortiGate heads off to the URL and downloads the now-expired certificate and we are back to square one, failing the connection due to an incomplete certificate chain of trust.

If this URL is not available, however, FortiGate will attempt to rebuild the chain of trust from the start and use the ISRC Root X1 Root CA Cert, which does provide an additional potential workaround.

Workarounds:

For sites under your own control, changing your server certificate to using the alternative chain will remove this issue, except for pre-7.1.1 Android devices, as described above.

Workaround 1 – Prevent fallback to the expired Root CA

With the removal of the expired IdenTrust DST Root CA X3 in Certificate Bundle version 1.28, it is possible to prevent fallback to the expired root CA by blocking FortiGate access to apps.identrust.com, resulting in the correct root CA being used.  This can be achieved by using either DNS blackholing or via an FQDN policy to block access to apps.identrust.com.

This will force the FortiGate device to rebuild the certificate chain and find the ISRC Root X1 Root CA Cert in the local certificate in the store.

Workaround 2 – Accept the expired certificates

For third-party sites outside of your control, customers can turn off this certificate expiration validation using the following CLI as a temporary workaround:

Disclaimer: By applying this workaround, you understand that end users connecting to webservers affected by invalid/expired certificates may have reduced protections typically afforded through the certificate chain. HTTPS connections matching the firewall policy with this SSL/SSH inspection profile may not be blocked when FortiGate sees invalid/expired certificates in the TLS Server Hello coming from the webserver. End users may see certificate warnings reported by the browser and it is the end user’s responsibility to decide to connect to the website that is providing the expired certificate and accept the risk that may be associated with the expired certificate.

**NOTE: When the permanent remediation is implemented, it will be important to revert this temporary workaround. Going forward, Fortinet will include testing for this configuration in our Security Rating service so that such temporary workarounds will impact the security rating score and trigger a recommendation to revert the setting.

Solution

Fortinet is working on a longer-term solution to improve certificate validation and add additional intelligence to rebuild missing certificate chains in these cases going forward, and will include this in a future release.