This page lists all confirmed or suspected issues involving the CA "WoSign". It will be updated by Mozilla as more information becomes available. Please do not edit this page yourself; if you have proposed changes, email Gerv.

The issues are in chronological order, and have been re-numbered from the original announcement to use letters with gaps in between, for possible future expansion.

(a.k.a. "Issue -2")

Between 16th January 2015 and 5th March 2015, WoSign issued 1,132 SHA-1 certificates whose validity extended beyond 1st January 2017. This is documented in their BR audit. Section 7.1.3 of the BRs says:

Effective 16 January 2015, CAs SHOULD NOT issue Subscriber Certificates utilizing the SHA‐1 algorithm with an Expiry Date greater than 1 January 2017.

So this requirement is a SHOULD NOT. RFC 2119 states:

4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label.

This means that this issue does not represent a violation of the BRs.

WoSign chose to declare this to their auditors on their WebTrust audit. The auditors state:

We understand that WoSign has established procedures and implemented controls to ensure that the aforementioned SSL subscriber certificates would be revoked on or before 31 December 2016.

This issue is also covered in WoSign's final incident report.

WoSign say that they were aware that they were doing this, and that the cause was a delay in implementing the changes from CAB Forum ballot 118. Ballot 118 was passed on 16th October 2014, but WoSign were unable to update their systems until March 5th 2015, nearly five months later. This seems like a surprisingly long delay for a relatively simple change, given how quickly WoSign have been able to update their systems in other cases.

However, as noted above, this is not a BR violation.

WoSign issued two certificates in March 2015. These certificates are identical in all ways (including their serial numbers) except for their notBefore dates, which are 37 seconds apart.

• Cert 1
• Cert 2

This issue is covered in WoSign's final incident report. They explain that there are 16 examples, and it happens when their Certificate Management System loses contact with a signing server during a transaction, and so tries the other one, but both signing servers issue a certificate. One was sent to the subscriber, and the other was stored in their records, and then logged to CT when they logged all their certs.

WoSign has since updated their system to use better locking.

Issuing two different certs with the same serial number is a violation of the RFC, but the impact of this bug is minimal.

(a.k.a. "Issue X")

Between 9th April 2015 and 14th April 2015, WoSign issued 392 certificates with duplicate serial numbers, across a handful of different serial numbers. Here is one example. This is documented in their most recent BR audit.

The audit report says:

We understand that remediation action was taken by WoSign to revoke those certificates in a timely manner. Incident investigation with root cause analysis was conducted and relevant result was documented in relevant incident report. Follow up action was also conducted to prevent the recurrence of the incident.

This issue is also covered in WoSign's final incident report. They explain that there were two bugs which led to the same result - one involving a system which did not understand serial numbers with leading zeroes, and the other a race condition.

This last auditor statement ("follow up action was also conducted to prevent the recurrence of the incident") is relevant given later issues involving duplicate serial numbers.

It's not clear how "a system bug with the serial number generation, generating a serial number starting with “0” in the first left position" combined with "the signing system had a bug that didn’t know how to deal with this kind of serial number" could lead to duplicate serial numbers. Further clarification has been requested on this point.

This incident is unfortunate and an RFC violation, but it was detected fairly promptly (albeit not by WoSign themselves) and fixed.

(a.k.a. "Issue -1")

On April 3rd, 2015, WoSign was contacted by Google, who were concerned about Baseline Requirements violations in recently-issued certificates from WoSign. Instead of specifying the violations directly, Google asked WoSign to check their certificates against their CPS. The following list of problems is a combination of those found by WoSign themselves and those pointed out later by Google:

• Incorrect or missing policy OIDs in all or most subscriber certificates;
• CPS required use of obscure "Issuer Alternative Name" field which was not being used;
• Inclusion of information in the Subject of the certificate (an advert) which was not validated subscriber information, and which contained a domain name, violating section 7.1.4.2 of the BRs;
• The CPS documentation of validity periods did not match what was happening in practice;
• CPS required compliance only with RFC 2459 instead of the more restrictive RFC 5280.

Google noted that many of these issues should have been caught by a competent auditor. WoSign's auditors at the time were Ernst and Young (Hong Kong).

WoSign resolved this promptly at the time by a mix of changes to practice and changes to their CPS (new versions 1.2.10 and 1.2.11).

This issue was included in Google's report to Mozilla in August 2016 but not listed in the original public discussion. This issue is covered in WoSign's final incident report.

WoSign acted quickly to resolve the issues, but this issue perhaps demonstrates that their CPS was not a definition of their practice, but more a document to be updated post-hoc to match changes made to their operations.

Mozilla documentation related to auditor errors states that:

When egregious mistakes were overlooked by the auditor, or there are a significant number of oversights, or the auditor did not notice BR compliance problems with the root or intermediate certificates, then the CA must resolve the issues and be re-audited. For the re-audit the CA can either get re-audited by a different auditor, or have the current auditor provide an immediate plan for correction and compliance, and then present a mid-term partial audit following that plan. In either case, the auditor must provide documentation about steps they are taking to avoid making the same mistakes in future audits. If the auditor fails to assure Mozilla that they have corrected the deficiencies in their auditing process, then their standing as a trusted auditor for the Mozilla root program may be jeopardized.

As Mozilla was not aware of this issue at the time, we were unable to make a judgement on whether these issues rose to an appropriate level of severity to require this response.

There is also the question of whether Mozilla wishes to continue to accept audits from an auditor who misses CPS violations covering such a large proportion of the certificate corpus.

(a.k.a. "Issue 0")

From Jan 10th to to April 23rd 2015, WoSign's certificate issuance system for their free certificates allowed the applicant to choose any port for validation. Once validation had been completed, WoSign would issue certificates for that domain. A researcher was able to obtain a certificate for a university by opening a high-numbered port (>50,000) and getting WoSign to use that port for validation of control.

This problem was reported to Google, and thence to WoSign and resolved. Mozilla only became aware of it recently.

• Before the recent passage of Ballot 169 in the CAB Forum, which limits the ports and paths which can be used, the Baseline Requirements said that one acceptable method of domain validation was "Having the Applicant demonstrate practical control over the FQDN by making an agreed‐upon change to information found on an online Web page identified by a uniform resource identifier containing the FQDN". This method therefore did not violate the letter of the BRs. However, Mozilla considers the basic security knowledge that ports over 1024 are unprivileged should have led all CAs not to accept validations of domain control on such ports, even when not documented in the BRs.

• The misissuance issue was not reported to Mozilla by WoSign as it should have been. Section 1 of the Mozilla CA Certificate Enforcement Policy says: "When a serious security concern is noticed, such as a major root compromise, it should be treated as a security-sensitive bug, and the Mozilla Policy for Handling Security Bugs should be followed."

• This misissuance issue did not turn up on WoSign's subsequent BR audit.

2016-08-25: "We are searching our database to try to find if any mis-issued cert is issued."

2016-08-25: "For BR auditor, I think this issue is too technical that fewer auditor can find out this problem."

List of crt.sh links to certificates involved - total 72. Richard Wang said: "We checked our system, the certificates issued related using higher level port website control validation is totally 72 certificates. To be clear, those certificates are validated by website control validation method that using other port except 80 and 443."

2016-09-04: Official issue report.

This issue is also covered in WoSign's final incident report.

It is the responsibility of the CA to disclose issues to its auditors, not for the auditor to discover them. WoSign was aware of this, because some of the issues in this document were disclosed to auditors and included in their report.

The completeness of WoSign's list of 72 certificates was called into question by a discussion participant who testified that his certificate was validated using port 8080 but does not appear in WoSign's list. In response, Richard said that in fact their system did not directly record the port used for validation, and so he could not guarantee that the list was complete. However, the final incident report still gives the impression that WoSign was able to generate a complete list.

WoSign has since stopped doing website validation; if they had continued, better logging of the process would have been necessary. The logs they were keeping while using this process were clearly inadequate.

(a.k.a. "Issue 1")

In June 2015, an applicant found some problems with WoSign's free certificate service. There were actually two bugs, which we will denote N1 and N2.

Bug N1 was an issue where someone proving control of <subdomain>.example.tld also was given a cert covering example.tld.

Bug N2 was an issue where arbitrary domains can be added to an existing request after validation.

The reporter proved that there was a problem in two ways. They accidentally discovered that there was a problem when trying to get a certificate for med.ucf.edu and mistakenly also applied for www.ucf.edu, which was approved. They then used their control of schrauger.github.com/schrauger.github.io to get a cert for github.com, github.io, and www.github.io. They also obtained another github cert using a different subdomain of github.io. These are both, in fact, instances of bug N2.

WoSign discovered the github misissuances due to a post-issuance review the following day, triggered by "github" being on their list of high-value domains. Those certs were revoked. However, no further investigation was performed, and several other certificates were misissued subsequently. The bugs were fixed two months later, on August 10th, in an unrelated major update.

• Recently, the reporter of the issue got in touch with Google to note that the ucf.edu cert still had not been revoked almost a year later. The lack of revocation of the ucf.edu certificate strongly suggests that WoSign either did not or could not search their issuance databases for other occurrences of the same problem. Mozilla considers such a search a basic part of the response to disclosure of a vulnerability which causes misissuance, and expects CAs to keep records detailed enough to make it possible.

• The misissuance issue was not reported to Mozilla by WoSign as it should have been. Section 1 of the Mozilla CA Certificate Enforcement Policy says: "When a serious security concern is noticed, such as a major root compromise, it should be treated as a security-sensitive bug, and the Mozilla Policy for Handling Security Bugs should be followed."

• This misissuance issue did not turn up on WoSign's subsequent BR audit.

Thanks to Stephen Schrauger for reporting this issue.

2016-08-25: "For BR auditor, I think this issue is too technical that fewer auditor can find out this problem."

List of crt.sh links to certificates involved - total 33.

2016-09-04: Official issue report. The report explains the two bugs, N1 and N2. WoSign classifies the misissuances as 21 N1 and 12 N2.

This issue is also covered in WoSign's final incident report.

N1 allowed an applicant to get a certificate for the base domain if they were able to prove control of any subdomain. According to WoSign, this was caused by an engineer misunderstanding the rule that if a base domain was validated, the "www" variant could be added for free. He instead applied the rule in the other direction.

Issue N2 is described in the report as follows:

This mis-issued certificate was a system vulnerability that when the subscriber finished the domain validation, they can add any domain before submitting this order to system.

Looking at their list of misissued domains in the report, this really does mean "any domain". For example, a cert where the owner validated "netwi.ru" was able to add "mx.idisk.su", an entirely different domain, without validating it. This effectively bypasses domain validation for any attacker who owns a domain of their own.

• An arbitrary unvalidated ownership domain misissuance bug is an extremely serious flaw, particularly if it's available in a normal workflow.

• The report seems to suggest an "issue first, ask questions later" approach whereby a cert is issued to the subscriber even if it is flagged as questionable, and the remediation mechanism is revocation rather than lack of issuance. Furthermore, their validation team works 9-5, so an applicant could have many hours to abuse a misissued certificate before it was discovered.

• Although it seems like the system issued a certificate which did not match what was validated, this was not investigated further. The report says: "the employee treated it as an unusual case that did not reported it as a bug." This is amazing, if true. If WoSign's employees don't think that misissuing a certificate is a big deal, something is quite badly wrong.

• The report seems surprised that the applicant did not follow the WoSign Terms of Use. I would not expect an actual attacker to care about the WoSign Terms of Use.

• The certificate involved here were all clearly misissued; even if the person could have proved control of the parent domain, they did not do so during the process. Therefore, failure to revoke these certificates immediately (as WoSign argued it didn't need to during the public discussion) constitutes a further BR violation, as per Section 4.9.1.1 of the BRs, in particular, item 4 and item 9.

WoSign has since stopped doing website control validation entirely. However, that does not decrease the seriousness of the bug/feature. It also raises the question of how such a feature could make it through review, testing and QA.

One of each pair has CRL and OCSP URLs with domains such as cr.wscrl.cn, oc.wsocsp.cn and ai.wscrl.cn. These domains no longer exist. The other one of each pair has CRL and OCSP URLs at subdomains of wosign.cn; these subdomains do exist, and point to either Akamai's CDN or what appears to be Qihoo 360's CDN. In the case of one of the pairs, the first cert was logged in the 'pilot' CT log about a month before the second one. One possibility is that WoSign was planning to adopt one strategy for CRL and OCSP hosting, and then changed strategy, which necessitated re-issuing the intermediates with new URLs. If that is the case, it raises the question of why the notBefore date for both certificates is the same.

Thanks to Kurt Roeckx and Rob Stradling for their help with this issue.

By private mail, Richard Wang of WoSign said that the plan was to use a CDN with a different domain, but in discussions with the CDN provider there was no need to change domain, so they changed the plan to use the existing domain and reissued the intermediate certificate. At that point, they "forgot to change the serial number". The old one issued only test certificates for two months. WoSign plan to revoke "this two intermediate CA and all issued certificates soon" (by which I assume he means the two certificates with the older domain names).

Given that intermediates are issued manually rather than in an automated fashion, and should normally be surrounded by strong controls as they involve issuance directly from the root, reusing a serial number for two intermediates shows a disappointing lack of care and appropriate processes.

In November 2015, WoSign issued two certificates that have subject public keys which are for the SM2 algorithm. SM2 is an elliptic-curve-based algorithm but it does not use the US NIST P-256, P-384, or P-521 curves. The CA/Browser Forum Baseline Requirements section 6.1.5 requires that only these three curves be used for elliptic curve keys in certs covered by the BRs.

In addition to including subjects keys using unapproved parameters, it seems these each share their serial number with another certificate for the same subject.

• 1st SM2 cert in crt.sh; cert with same serial number in crt.sh
• 2nd SM2 cert in crt.sh; cert with same serial number in crt.sh

Secondly, for the first pair of certs, the validity period is 4 years, which is 9 months longer than allowed by the BRs.

Richard Wang: "This is another case that we will include it in our report. We issued two test cert using SM2 algorithm that used the same serial number as the RSA cert (same subject) to test if we can setup a gateway that install this two type cert, it can shake hand automatically using different cert based on the browser algorithm support." (Unable to find this message in the groups.google.com archive.)

This issue is also covered in WoSign's final incident report. Their later final statement document says: "being a Chinese licensed CA, we must abide by local laws and regulations, we must actively cooperate with domestic browsers to test the SSL certificate using SM2 algorithm issued by a global trusted root in the real Internet, not intranet."

There are plenty of ways of testing this scenario without using public roots - and, in fact, WoSign has said they have updated their systems to avoid issuing test certificates from public roots in future. Mozilla is sceptical of the claim that Chinese law or regulation required WoSign to issue these certificates. However, if that were true, the Baseline Requirements contain a mechanism (section 9.16.3) by which a CA can break the BRs if required to do so by local law, with appropriate disclosure to the CAB Forum. That was not done in this case.

We note that Symantec was penalised in late 2015 for issuing non-BR-compliant test certificates in their publicly-trusted certificate hierarchies. Their problems were first revealed in September, and one big discussion of these problems happened in m.d.s.policy starting on 13th October 2015. All of these dates are prior to WoSign's test certificate misissuances.

WoSign purchased the CA "StartCom" and did not disclose the transaction as a change of ownership, which we believe violates section 5 of the Mozilla CA Certificate Maintenance Policy. Furthermore, when this clause was brought to their attention, they denied that any changes fell under it, and they attempted to suppress further information about the ownership transfer when it was brought to the community's attention.

Full details can be found in the post in mozilla.dev.security.policy.

Richard Wang writes: "An announcement and disclosure will be made shortly pending completion of the business transaction. We can provide the proof documents to Mozilla to show this transaction is not finished if Mozilla think it is necessary."

On 2016-09-19, WoSign issued a press release saying that they have "made an equity investment" in StartCom CA.

Mozilla has confirmed with a Hebrew-speaking lawyer that, as far as the Israeli company authorities are concerned, the transaction which completed the chain to give WoSign 100% ownership of StartCom completed on November 1st 2015. We maintain that this is the date at which the ownership change should have been disclosed to Mozilla.

In addition, WoSign's press release about their "equity investment" seems to be economical with the truth; WoSign owns 100% of StartCom. The statement that "WoSign and StartCom [are] two companies operate[d] and managed independently" is hard to square with the fact that the same person is sole director of StartCom and CEO of WoSign. Also, there is technical evidence that StartCom are now using a large part of WoSign's issuance infrastructure.

WoSign has issued certificates after January 1st 2016 but backdated the notBefore date to be in December 2015. This has the effect of avoiding the blocks in browsers regarding SHA-1 certs issued after January 1st 2016. The number of certs affected is probably 67, but may be a few more or less.

The three certificates below have notBefore dates in Dec 2015, have signatures over SHA-1 hashes and have embedded SCTs which are dated after January 1, 2016.

Because these SCTs are embedded, they must have been created before the final certificate was signed, and therefore the final certificate must have been signed in January - on or after January 18th, for the third one.

These are the only certs for which cryptographic proof of backdating is available. However, other strong circumstantial evidence suggests that backdating is more prevalent.

Firstly, around 12th May 2015, WoSign started using a new certificate template for some of their SHA-1 issuances. This has a fixed notAfter date of 2016-12-29T16:00:00Z (i.e. midnight on 29th/30th December, CST). This may have been a (sensible) move to prevent accidentally issuing SHA-1 certs which ran into 2017. We have found 939 SHA-1 certificates with this fixed notAfter date, including the above three.

Certificates issued using this template almost exclusively have notBefore dates on working days in China, suggesting they are issued by explicit WoSign employee action. All of them have notBefores between Monday and Friday, except for 13 on a Saturday and 66 on a Sunday. Of those 66, 4 have notBefores on 2015-09-06, a Sunday which was, unusually, a working day in China. The other 62 all have notBefores on 2015-12-20, which was a Sunday, and not a working day in China. This suggests that, for these 62, the notBefore does not reflect the actual issuance date.

Lastly, of the 62 suspect certs, there are three more certs with embedded SCTs where the gap between the notBefore date and the SCT date is multiple days (i.e. they were backdated, and this is cryptographically provable) but where the SCT date is nevertheless (just) before 1st January 2016, which means the backdating would not have the effect of avoiding browser blocks.

The issuance of backdated certificates is not forbidden by Mozilla policy, but is included in Mozilla's list of Problematic Practices. It says "Minor tweaking for technical compatibility reasons is accepted, but backdating certificates in order to avoid some deadline or code-enforced restriction is not."

The Baseline Requirements, section 7.1.3, say:

Effective 1 January 2016, CAs MUST NOT issue any new Subscriber certificates or Subordinate CA certificates using the SHA‐1 hash algorithm.

(Thanks to Thijs Alkemade of Computest for much of the information in this section.)

This issue is covered in WoSign's final incident report. Of the 64 certificates they consider, WoSign attribute 6 to a dual-issuance bug, 9 they say are not problematic because they were issued on December 31st 2015, 2 are the mis-issued certs relating to Issue V, and the other 47 are "normal SHA-1 orders".

Mozilla is seeking further information on the details in WoSign's report and will make further comment at a later date.

A certificate has been issued in June 2016 to alicdn.com which was not requested by the owner of that domain. The domains in question currently use certificates from Symantec.

• Cert on Github Gist
• Cert on crt.sh

This issue is covered in WoSign's final incident report. WoSign assert that they completed the domain control checks correctly.

The owners of alicdn.com have confirmed directly to Mozilla that this incident was caused by an attacker being able to control content on their domain.

We would have some concerns with how WoSign's systems validated website control. However, they no longer support this method. This "misissuance" was possible only because alicdn.com was, for some period of time, controlled by an attacker. Therefore, no blame can be attributed to WoSign for the misissuance itself.

(a.k.a. "Issue 2")

In July 2016, it became clear that there was some problems with the StartEncrypt automatic issuance service recently deployed by the CA StartCom. This was a StartCom-branded service and was not publicised as being able to issue certificates from WoSign. However, changing a simple API parameter in the POST request on the submission page changed the intermediate/root certificate to which the resulting certificate chained up. The value "2" made a certificate signed by "StartCom Class 1 DV Server CA", "1" selected "WoSign CA Free SSL Certificate G2" and "0" selected "CA 沃通根证书", another root certificate owned by WoSign and trusted by Firefox.

A security investigator used the value "1", and acquired two certificates which had notBefore dates (usage start date) of 20th December 2015, and which were signed using the SHA-1 checksum algorithm. Cert 1, Cert 2.

• The issuance of certificates using SHA-1 has been banned by the Baseline Requirements since January 1st, 2016. Browsers, including Firefox, are enforcing this - in Firefox's case, for publicly-trusted CAs, since Firefox 48, released on 1st August 2016.

• The issuance of backdated certificates is not forbidden, but is included in Mozilla's list of Problematic Practices. It says "Minor tweaking for technical compatibility reasons is accepted, but backdating certificates in order to avoid some deadline or code-enforced restriction is not."

• WoSign deny that their code backdated the certificates in order to avoid browser-based restrictions - they say "this date is the day we stop to use this code". If that is true, it is not clear to us how StartCom came to be able to call WoSign code that WoSign itself had abandoned.

• The misissuance issue was not reported to Mozilla by WoSign as it should have been. Section 1 of the Mozilla CA Certificate Enforcement Policy says: "When a serious security concern is noticed, such as a major root compromise, it should be treated as a security-sensitive bug, and the Mozilla Policy for Handling Security Bugs should be followed."

2016-09-04: Official issue report. This issue is also covered in WoSign's final incident report.

• This issue does not paint a picture of careful software development practices and quality assurance - having unused code around capable of issuing BR-violating certificates does not seem like responsible practice.

• The question of why StartCom was able to trigger certificate-issuance code which WoSign has stopped developing and maintaining is also still open.

The first WoSign incident report, produced in response to other issues listed on this page, has a screenshot of a dig query from their validation server. The dig program is part of the bind-utils package, and the output of dig appears to show a bind-utils version of 9.7.3-8.P3.el6. The "el6" shows that this is a version built for Red Hat Enterprise Linux 6. This version of bind-utils was released in December 2011 and so is very out of date.

The next release of this package for EL6 following the one WoSign are using is bind-utils 9.7.3-8.P3.el6_2.1, which was released a little later in December 2011. The most recent version is 9.8.2-0.47.rc1.el6, which was released on the 10th of May 2016. There are 19 patched CVEs between the version WoSign is suspected of running and the current version. None of these CVEs are especially severe. However, if this software is in fact that far out of date (nearly five years), it raises questions about the overall patch level of their verification server and even their other infrastructure.

WoSign's most recent audit used the "SSL Baseline With Network Security - Version 2.0" criteria. These criteria integrate two CA/Browser Forum Documents - the SSL BRs and the Network & Certificate Systems Security Requirements.

A bullet on page 19 of the Network Security Requirements requires that "Recommended security patches are applied to Certificate Systems within six months of the security patch’s availability, unless the CA documents that the security patch would introduce additional vulnerabilities or instabilities that outweigh the benefits of applying the security patch." That appears not to be true of their version of bind, and so may well not be true of other more critical packages on their systems.

We would expect the presence of badly outdated software and the lack of an appropriate patching regime to have been caught by WoSign's auditors.

Thanks to Paul Pearce for his help with this issue.

WoSign's most recent incident report acknowledged the issue and they committed to updating the OS on the server(s) in question.

N/A.

It is important background information to know which WoSign roots are cross-signed by other trusted or previously-trusted roots. The following unexpired and unrevoked cross-signatures are currently known: