Certificate Transparency

Learn more about one of the most impactful changes in the TLS ecosystem in recent years: The public logging system Certificate Transparency.

---

What is certificate transparency? 

While the changes needed for server operators happen largely automatically, it’s good to know why Certificate Transparency was introduced, what it means for the ecosystem, and how it can be used to improve security, detect incidents, and help security research.

Let’s dive into a (quite detailed) account of Certificate Transparency:

Based on a simple idea

At the core, the idea is simple:

Every certificate used on the public Internet and trusted by browsers is stored in at least two public logs. Everyone can access these logs. That is Certificate Transparency.

To understand, however, why this system was introduced we have to look a few years back:

In 2011 and 2012 a series of hacks and incidents showed the vulnerability of the companies issuing the certificates, the so-called certificate authorities (CAs).

In early 2011 hackers had been able to issue rogue certificates from the certificate authority Comodo. These certificates were used in hacker attacks on services like Google, Yahoo, Skype, and others.

Later that year the Dutch certificate authority DigiNotar suffered from an even more severe incident: 500 rogue certificates were found. Some of them were used to spy on Iranian Gmail users.

DigiNotar was subsequently distrusted as certificate authority by all major browsers.

In 2012, the certificate authority Trustwave admitted to having sold an intermediate certificate that enabled its owner to sign digital certificates for virtually any domain on the Internet.

Trustwave said that the certificate was used within a private network as part of a data loss prevention system – in other words, to intercept Internet communications within a company.

This is a common, yet controversial, practice: Usually, it is done with a locally generated root certificate which is only valid within a company network.

Having an intermediate certificate from a trusted certificate authority allowed for a capacity of communications monitoring way above a ‘”need to have” level.

In the wrong hands, internet communication could be intercepted—also outside the company—without any of the communicating parties knowing it.

Mozilla subsequently discussed distrusting Trustwave as a certificate authority, but eventually decided against it.

An ecosystem in need of security improvements

Incidents like these created the impression that the TLS business was the wild west of the Internet.

Control of CA operations was very limited. There were a lot of CAs out there and if even one of them got compromised, their certificates could easily be abused to attack any TLS connection out there.

When that actually happened it often had little or no consequences for the companies responsible.

Some people discussed whether it would be best to get rid of the CA system altogether and start over with a new system.

However, alternatives like DANE, Convergence, or the Sovereign Keys project never gained any real traction and none of them got default support by modern browsers.

Usually, the alternatives had other problems that made them impractical in other ways.

Consequently, people started to think of ways to improve the certificate authority system.

This included measures like the creation of the Baseline Requirements – a set of rules agreed upon by browsers and certificate authorities.

But one of the most impactful changes to improve the CA system was invented by Google.

It was called “Certificate Transparency”(CT).

Certificate Transparency (CT)

CT is an open framework for monitoring the TLS/SSL certificate system and auditing specific TLS/SSL certificates.

At its core, it consists of several logs that store certificates. Anyone can access these logs.

The content can be extracted with a public API. Submission to logs is also open, though log operators can decide for themselves which submissions to accept.

While some logs accept all certificates issued by browser-trusted certificate authorities, others are restricted to certificates from certain time frames or specific certificate authorities.

Having these certificates logged enables both site operators and security researchers to keep an eye on certificate authorities.

A security-conscious web page operator can monitor the logs and compare the certificates they actively use to the ones appearing in the public logs.

This can also be outsourced:

A discovery tool like TRUSTZONE’s SSL360® will send you an email when a new certificate for a specific domain appears in the logs.

This is a powerful tool when monitoring the web for rogue certificates.

Let’s assume you operate a very sensitive service at example.org. By monitoring the logs you learn that somebody has issued a certificate you don’t know about for example.org.

Upon further investigation, it seems that the certificate has been issued by a certificate authority due to hacker involvement.

You raise an alarm, warn users and inform the public that most likely this authority has to deal with a security problem.

It is not only site operators that can benefit from checking the logs through. Security researchers constantly use Certificate Transparency data to identify oddities.

The Symantec distrust

In January 2017 Andrew Ayer found several suspicious certificates issued by Symantec. In one example he found a certificate issued to the names test.com, test1.com, test2.com, and so on.

These domains belonged to different people and companies. Therefore, it seemed highly unlikely that they were legitimately issued.

This incident turned fatal for Symantec. Further investigations uncovered that in may cases, the company had breached rules and issued illegitimate certificates.

Ultimately, the discoveries made via the Certificate Transparency logs led to a decision by browser providers to distrust Symantec root certificates.

Eventually, the company sold off their certificate business to competitor DigiCert.

Transparency mandatory for new SSL certificates

A crucial change happened in the spring of 2018: As of April 2018, the Chrome web browser requires new certificates to be logged.

Every certificate issued since this date must show two so-called Signed Certificate Timestamps (SCTs) to be considered valid.

Before this mandatory logging requirement, you could never be sure that certificates would make it into the logs. But now they are almost certain to do so.

As you may know, Chrome has the largest market share of all browsers. It’s not likely that anyone would use a certificate that is not accepted by this browser.

Bear in mind, though, that rogue certificates issued before April 2018 still can be used and not logged. Their timestamp can be backdated.

This will change as soon as certificates issued before April 2018 expires, that is in less than two years from this date.

Compliance with Certificate Transparency

So, what do web site owners have to do to be compliant with Certificate Transparency?

In most cases the answer is simple: nothing!

The reason is that certificate authorities usually put the Signed Certificate Timestamps directly into the certificate itself.

Thus, if you got your certificates quite recently, they are probably already compatible with Chrome’s new rules. There are other ways to deliver Signed Certificate Timestamps, but they’re more complicated and not used very often.

Secret hostnames don’t work anymore

Certificate Transparency has a side effect that many site owners may not be aware of.

It’s gotten much harder to keep your hostnames a secret:

If a company has an internal site, e.g. under the name internal.example.org, and gets a certificate for it, anyone can discover the existence of it simply by looking in the logs.

This can be prevented by using wildcard certificates that cover the root and all subdomains (e.g. *.example.org), but that is not always a practical solution.

Having secret hostnames was never a good idea to begin with, though. Hostnames are not encrypted in TLS connections and even if they were, the DNS query would still prevent them from really being a secret.

Nevertheless, Certificate Transparency makes it much easier to uncover hidden hostnames.

Bug finders, but also attackers, use the capability to find out which subdomains companies run.

The author of this article presented a way to abuse this capability to attack common web applications like WordPress during installation. The method was presented at last year’s Def Con conference:

While Certificate Transparency allows an attacker to learn about new certificates showing up it also often points to new websites being created.

These sites are often not fully installed when receiving their TLS certificate.

By scanning them, a hacker may discover ongoing unsecured installation processes. These can be used to hack the websites in question.

A search engine for certificates

A popular site related to Certificate Transparency is crt.sh. It’s a search engine for certificates.

Try it out yourself: By typing in %.trustzone.com, you can see all the certificates for TRUSTZONE subdomains. If you try this for your own domain, you’ll see the certificates logged for it.

Just looking at the list may be helpful.

You may discover services that you forgot all about and that runs without the knowledge of your company.

This is what happened to the security team at Facebook:

The team started looking into the Certificate Transparency logs and found that there were unexpected certificates for Facebook’s domains.

These were issued by a hosting company with the agreement of other Facebook employees but without the knowledge of the security team.

Certificate Transparency is one of the largest changes in the TLS certificate ecosystem in recent years. It has many security advantages, but some unexpected side effects: It can be useful for attackers as well. Both users and web site owners should be aware of that.