Protecting Firefox users following the compromise of eMudhra intermediate certificates
119 points by freddyb
119 points by freddyb
If we've got an intermediate CA leak, then that means... we can now mitmproxy game consoles like PS5. Console hackers are going to have a field day with this.
Hell, I'm going to have a field day with this. I've got some old chinese cameras that desperately need pwning.
I have a Eufy C10 for which I've been pondering for months how I can de-cloud it, seeing as there's basically nothing like Valetudo for Eufy vacuums (or at least not for the C10). Maybe now this will be a bit easier.
Test page at https://emudhra.we-have-a-problem.com/
Huh, it says that Apple devices were patched on the 19th (yesterday) but my iPhone still fails the tests. My Mac is good though, on both Firefox and Safari. Odd.
I’m running iOS 18.7.8 with no updates available, and yet my phone passes the tests. Seems there’s some kind of update mechanism here outside of OS updates, so I can imagine that it happens at a different schedule for different people.
Must be some type of staged rollout. xprotect and softwareupdate both say they are up to date, and I'm still seeing failing tests on that site (on macos GoldenGate, and ostensibly up to date).
Strange, my Firefox on Android fails the test (it connects to the site). Does that not use the CRL?
There is a known issues with Firefox for Android, as it gets the CRL less reliably. Should also be fixed by force-refreshing the PlayStore to search for a full app updates. We are investigating.
FWIW my Firefox Nightly on Android, with build date 2026-10-20T09:59:29, also fails (i.e. it connects). Seemingly based on the build date it should be up-to-date? Let me know if you want an actual bug report.
I am on Firefox Mobile 158.0.1, with a build date of 2026-10-19T18:58:56, which I just updated through the Play Store and which still connects to the test sites.
UPDATE: Restarting my phone resolved it.
I'm also on 158.0.1 (Samsung Galaxy S26 Ultra, Android 17) and restarting my phone didn't fix it.
yup, after phone os update, play store update, ff app update, and reboot... still not seeing the fix.
Thanks. Any idea why Waterfox on mobile fails the test?
The 6.7.0 changelog says
Certificate revocation data (OneCRL and CRLite), as well as the add-on blocklist, are updated over the network again. 6.6 froze these at build time, along with everything else. Version 6.7 only syncs the security-relevant collections and stores the rest in offline bundles.
so perhaps if you're not on 6.7.x yet then it's not actually dynamically downloading the crlite data?
It seems like there was public knowledge about this since at least October 15th, when an anonymous user commented on ExplainXKCD for comic 221 (the "return 4" random value):
this comic just keeps aging like a fine wine... 19:16, 15 October 2026 (UTC)
https://www.explainxkcd.com/wiki/index.php/221:_Random_Number
Wow, good find. According to Bugzilla, Mozilla only received a report of this on the 17th, around 49 hours after this comment was posted.
https://glauca.space/@q/117474051360413376
This post and one of the replies (haven't verified, etc etc), claims that all the revoked certs have private keys that are < 256.
https://chaos.social/@fluepke/117474146569586375 verified here
rest of the "keys" are here https://infosec.exchange/@jjtech/117474212820456971
Wow. I desperately want to know how this happened.
This comment on the bugzilla uncovers a part of the story: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11
To be clear, they’re one byte, with a value of <256.
I thought this meant they used less than 256 bytes, but it’s so much worse.
Below is the list of certificates in Certificate Transparency issued by the blocked CAs as of this time. They appear to be non-malicious test certificates, and the DNS name in the first certificate indicates who likely discovered the issue.
What's alarming and unprecedented is that yesterday, two Certificate Transparency log implementations were updated to block the logging of certificates issued by the CAs. This likely indicates total private key compromise, and the blocking by CT logs is to prevent the logs from being spammed by forged certificates. However, it also means there will be no place to record forged certificates found in the wild. (These certificates won't validate in CT-enforcing clients since they won't be in CT logs, but could be used to attack non-CT-enforcing clients.)
Null display is "NULL".
operator_name | not_before | not_after | dns_names | cert_sha256
---------------+------------------------+------------------------+--------------------------------------------+------------------------------------------------------------------
emSign | 2026-10-19 20:24:33+00 | 2027-05-07 20:24:32+00 | {*.emsign.filippo.io,emsign.filippo.io} | 3c3d0982b3cc4ffe19c9343991ad34fe474042344fa551ff309a5a6b069cf0b4
emSign | 2026-10-19 20:37:26+00 | 2027-11-20 20:42:26+00 | {dv-g3b-emudhra.we-have-a-problem.com} | 325627388d71255f5aeb1f2d0b695b8428d7372627f7f80c55e544d159113f6d
emSign | 2026-10-19 20:37:26+00 | 2027-11-20 20:42:26+00 | {dv-g3c-emudhra.we-have-a-problem.com} | a44b8c7a125fb1fe4c2b34df5d2695530d0fdd3ab920e5af4da793f920189d36
emSign | 2026-10-19 20:37:26+00 | 2027-11-20 20:42:26+00 | {ev-g3b-emudhra.we-have-a-problem.com} | 78d25611320f01b22296c845372e6d8559139597537dacfe2119b43d5d8d5042
emSign | 2026-10-19 20:37:26+00 | 2027-11-20 20:42:26+00 | {ev-g3c-emudhra.we-have-a-problem.com} | 96949201d0c9cfa8892c7aa03eb89341ba4bccb6785b58af571cddd240d3e013
emSign | 2026-10-19 20:37:27+00 | 2027-11-20 20:42:27+00 | {ov-g3b-emudhra.we-have-a-problem.com} | 640a2dfd643ed5afe41ffd2f4999ad08c3c8b40ab2a32b28bb58e93e14deadb4
emSign | 2026-10-19 20:37:27+00 | 2027-11-20 20:42:27+00 | {ov-g3c-emudhra.we-have-a-problem.com} | a2535cbee38aa60dc9d649011b2b9d75f0efd9a5d46c5af1948d4a6ccd3de7dd
emSign | 2026-10-20 00:59:36+00 | 2027-05-03 01:04:36+00 | {dv-g3b-emudhra.we-have-a-problem.com} | 9dcb21b1841d75313b3c888f4bb9cc01f8f51379a76fbaa241ca492b668839e7
emSign | 2026-10-20 00:59:36+00 | 2027-05-03 01:04:36+00 | {dv-g3c-emudhra.we-have-a-problem.com} | 346cdd962e97189b59aee5526d1ae8fa714bb1a64094dfa67bef768ee93f36b5
emSign | 2026-10-20 00:59:36+00 | 2027-05-03 01:04:36+00 | {ev-g3b-emudhra.we-have-a-problem.com} | 66374ae64962c437bd815a2aae546c76e30c2eafac8335192c8e517c15a246ed
emSign | 2026-10-20 00:59:37+00 | 2027-05-03 01:04:37+00 | {ev-g3c-emudhra.we-have-a-problem.com} | ccd80467f9cac38f115dcb6e4f57e4d74f5efa519814ea934aeb3be53b83a191
emSign | 2026-10-20 00:59:37+00 | 2027-05-03 01:04:37+00 | {ov-g3b-emudhra.we-have-a-problem.com} | f3c3c15bc7edf2b3ebdc922af112f2a0c414eb509901099ef0488c0942e77efe
emSign | 2026-10-20 00:59:37+00 | 2027-05-03 01:04:37+00 | {ov-g3c-emudhra.we-have-a-problem.com} | 4314941cf242a4ea4149aa7e9cdf77a15de2771a79984b809fc04443fea013c8
emSign | 2026-10-20 02:22:41+00 | 2027-05-03 02:27:41+00 | {dv-g3c-aaa-emudhra.we-have-a-problem.com} | 9200d153c3cac272cc06bf68f9ea163fc24659fc3331e77668236fe00645ae52
emSign | 2026-10-20 02:22:41+00 | 2027-05-03 02:27:41+00 | {dv-g3b-aaa-emudhra.we-have-a-problem.com} | 7e02622ab6d745c1edb4b431bb1c551e667305c3dd3eb0c7cc6dfdb523807424
emSign | 2026-10-20 02:22:41+00 | 2027-05-03 02:27:41+00 | {ev-g3b-aaa-emudhra.we-have-a-problem.com} | bc88231a840e54cced148da48806169e8d0b1c0a96267a427aeccda9d10e6145
emSign | 2026-10-20 02:22:42+00 | 2027-05-03 02:27:42+00 | {ev-g3c-aaa-emudhra.we-have-a-problem.com} | bcd6ab59ea95fd520bfe7709679b1f682abbd79c77367e84e25415b91de37753
emSign | 2026-10-20 02:22:42+00 | 2027-05-03 02:27:42+00 | {ov-g3c-aaa-emudhra.we-have-a-problem.com} | 59cf0386b56bc2414a0c06241ea9e25e01a9c93bac7a596648747f9d0ded4906
emSign | 2026-10-20 02:22:42+00 | 2027-05-03 02:27:42+00 | {ov-g3b-aaa-emudhra.we-have-a-problem.com} | 21f57f83631a93c98e867050075746b3361e0433c9e0e969462852476784f6f0
(19 rows)
Based on @q3k's comment over at the orange site, forging certs seems to be relatively easy:
$ curl https://q3k.google.com --resolve 'q3k.google.com:443:135.181.235.221'Yeah okay this is funny. Nice little CTF challenge :).
edit: Allegedly the private key is 4. Like, the number 4.
Who know XKCD would be a literal prophecy.
As it kept happening with The Simpsons and happens nowadays with The Onion: reality has become so dumb, satire can't keep up.
There's a non-public bugzilla entry for the Mozilla CA Program, which is unusual, so yeah.
TLDR: If you maintain software that does not use the OS certificate store, remove the roots or (if you can) the intermediate CAs.
If you don't, well, happy update day.
FWIW, Rust is also doing similar https://github.com/rustls/webpki-roots/pull/133
there is options to use the OS trust store with rust and I usually go out of my way to ensure that one is used, the built-in one tends to be default more often because "it just works" and lets you build smaller containers.
Posting this just so it can be merged into the main story. This is the bugzilla issue, which just was made public a few minutes ago.
I am very surprised that this is handled in the standard library of a programming language. I know cert revocation is a mess on the internet, but still, something really bad must have happened if they go down this route.
Since a program using Go is the very definition of programmatic TLS and often would not be in a position to check CT logs or any other protection on the internet, there's really no better way to block a compromised intermediate certificate authority. Previously, you would see root-level certificate authorities removed, and those are removed by whatever manages the root program on the system, typically the operating system. It's not so much that this is in a programming language's standard library as it is that it needs to exist in a TLS stack. It just so happens that, for Go, those are the same.
I think what fs111 is saying is that this seems like the job of operating system vendors and/or their packaging systems, but yea, cert stores are a mess
But that's my point. In a Go program, the operating system cannot distrust an intermediate CA; it can only distrust the root CA by removing it from its root trust store.
some can... but i get your point, i didn't realize they weren't revoking the underlying roots... i wouldn't be surprised if that comes next
https://emudhra.we-have-a-problem.com/ has nice details on how different vendor stores can block compromised intermediates with an update
that said, the fact that a lot of application TLS clients aren't checking CRLs by default (which i get... it can be expensive) is risky in situations like these
Some OSs can block intermediates in their TLS stack if your software adopts it (think SChannel on Windows). If a program uses something like Go with its own TLS stack, then the OS can't do much. Thankfully, in this case, the compromised intermediates chain to only two roots, and the root stores can remove them. This makes it easier since the root stores can just remove both emSign Root TLS CA - G3 and AAA Certificate Services. If those roots had also been cross-signed to something not so easily removed, we would be in a really bad spot on the internet.
The solution for certificate revocation checking in non-browser TLS clients should be upki, but it’s still very new and hasn’t yet found its place as a standard part of the infrastructure.
They added that for this specific incident. This is not an existing facility like Firefox or Chrome have. Also, the CRL for the root lists them as revoked, so I still find this rather surprising that they added this feature over night and cut new releases.
Go doesn't check CRLs by default. There is a request at https://github.com/golang/go/issues/68573. Very few non-browser environments check CRLs by default.
A few of my friends that helped with https://blog.brycekerley.net/2026/03/08/webpki-and-you.html are insinuating that one CA did a very bad thing and none of the normal TLS libraries support revocation of intermediates in a discoverable way. I know that the cert validation code I wrote at a previous job thirteen years ago doesn’t.
Very odd. There appears to be a CVE but basically no information.
"We highly recommend immediately upgrading to this release." https://groups.google.com/g/golang-announce/c/66pojueaQek