Two weeks ago, I made this post. The goal was simple: I wanted to dig into the details of Chromium and Firefox to see if the claims that Chromium is more secure than Firefox are true or not. You’ll notice I also started turning that post into an update log, but only one update got released. There is a reason for that. Life suddenly got extremely busy for me, I could barely make time to continue researching. However, during that time, I spent a lot of time thinking about the issue. I tried breaking down the problem in a million different ways to find a way to simplify it and start from the ground up.
I came to a conclusion today, a realization. I have no way to put this gently: I cannot conclusively determine which one is more secure. This will upset many of you, and it upsets me too considering I maintain my own list of software that relies on only providing the most secure and private versions of some software. I need to explain why there cannot be a solid conclusion.
I managed to collect many sources to be used for the research. A lot of the information is parroting this article which, despite having many sources, fails to provide sources for some of the most crucial claims made there (“Fission in its current state is not as mature as Chromium’s site isolation” has no source, for example). My favorite source is this Stanford paper which I think does a great job at tackling the problem. The problem I noticed is that a lot of privacy advice is given from an echo chamber.
Think about what privacy advice you like to give, and think about where you heard that. A YouTube video? Reddit? Lemmy? Naomi Brockwell gives a lot of advice that stems directly from Michael Bazzell’s Extreme Privacy book, as I found out after reading it. Her videos about convincing people to use Signal are paraphrased passages from the book itself, which has a whole section about it. People touting Chromium as more secure than Firefox, or that the Play Store is a more secure option than F-Droid or Aurora Store, often get their information from GrapheneOS. I’ve never seen anyone research those in depth.
The point I’m trying to make is that a lot of privacy advice is circular reporting. I’m certain that if Michael Bazzell and GrapheneOS were to provide sources as to where they got their information (they rarely do, I checked) it would come to light that it boils down to a few real sources. GrapheneOS, no doubt, likely has inspected at least some part of the Firefox codebase, but Firefox is rapidly changing, so any sources that used to be true may not be true today.
FUTO Keyboard and GrayJay get recommended often because of Louis Rossmann, but HeliBoard and FreeTube (or NewPipe) were options long before those pieces of software. The reason the former became so recommended over the latter is simply because people used a popular figure, Louis Rossmann, as a primary source. It then became an echo chamber of recommendations and best practices.
That doesn’t mean the claims of Chromium being more secure are false, but as a researcher it is very hard to credit something that doesn’t provide any primary sources. In the eyes of a researcher, GrapheneOS’s word holds just as much weight as a random internet user, without any proof. I see it play out like this: A source like GrapheneOS or Extreme Privacy makes a claim, secondary sources such as GrapheneOS users or Naomi Brockwell present this information without providing the sources, the general privacy community sees both, and begin giving the same recommendations on Reddit or Lemmy (sometimes with sources), and eventually the privacy community as a whole starts presenting that information, without any primary sources. Even if GrapheneOS, Extreme Privacy, or Louis Rossmann provided no research or direct comparisons, their word is taken without question and becomes the overarching recommendations in the privacy community. They each gained credibility in their own ways, but there should always be scrutiny when making a claim, no matter how credible.
The main reason why I cannot give a concrete conclusion is this: the focus on the article was to compare Chromium’s Site Isolation to Firefox’s implementation, however there are too many variables at play. Chromium may be more secure on one Linux distro than another. Debian is an example. Firefox supposedly has worse site isolation on Linux, but then how does Tails deal with that? It’s based on Debian, so does that make it insecure for both browsers? Tor is based on Firefox ESR, which is an extended support release with less security, but Tor is also deemed a better option than Chromium browsers for anonymity. Isolating iframes doesn’t really affect daily use, so is it really necessary to shame Firefox for that? Some variants of Firefox harden the browser for security, but some variants of Chromium (such as Brave Browser) try to enhance privacy. No matter what limits I set, how many operating systems or browser variants I set, there is no way to quantify which one is more secure.
“Is Chromium more secure? Yes, under XYZ conditions, with ABC variants, on IJK operating systems. Chromium variants XYZ are good for privacy, but ABC Firefox variants are better at privacy…” The article would be a mess. The idea for the article came because I was truly sick of the lack of true in-depth sources about the matter, and so I wanted to create that. I now realize it was a goal that is far too ambitious for me, or even a small group of people. Tor and Brave give different approaches to fingerprinting protection (blending in vs. randomizing), and there’s no way to directly compare the two. The same goes for the security of each. There is no “Tails” for Chromium, but there is no “Vanadium” for Firefox. There’s no one to one comparison for the code, because some of it is outside of the browser itself.
I regret making that initial post, because it set unrealistic expectations. It focused on a problem that can’t tell the whole picture, and then promised to tell that whole picture. At a point, it comes down to threat model. Do you really need to squeeze out that extra privacy or security? Is someone going to go through that much effort? You know how to spot dark patterns, you know not to use privacy invasive platforms. Take a reality check. Both Chromium and Firefox are better than any proprietary alternatives, that’s a fact. Don’t bother trying to find the “perfect” Linux distro or browser for privacy and security, because you already don’t use Windows. Privacy is a spectrum, and as long as you at least take some steps towards that, you’ve already done plenty.
Be careful next time you hear a software recommendation or a best practice. Be careful next time you recommend software or a best practice. Always think about where you heard that, and do your own research. There are some problems that are impossible or infeasible to solve, so just pick what you feel is best. I really am sorry that I wasn’t able to provide what I promised, so instead I will leave a few of the sources I found helpful, just in case another ambitious person or group decides to research the matter. Not all of these sources are good, but it’s a place to start:
GrapheneOS responded to my requests for a comment after this post was made, here: https://lemmy.ml/post/22142738
https://www.cvedetails.com/version-list/0/3264/1/
https://en.wikipedia.org/wiki/Site_isolation
https://madaidans-insecurities.github.io/firefox-chromium.html
https://news.ycombinator.com/item?id=38588557
https://seclab.stanford.edu/websec/chromium/chromium-security-architecture.pdf
https://grapheneos.org/usage#web-browsing
https://www.reddit.com/r/browsers/comments/17vy1v5/reasons_firefox_is_more_secure_than_chrome/
https://www.wilderssecurity.com/threads/security-chromium-versus-firefox.450867/
https://forums.freebsd.org/threads/why-im-switching-from-firefox-to-ungoogled-chromium.87878/
I have no way to put this gently: I cannot conclusively determine which one is more secure.
That’s the only conclusion I would have trusted. Otherwise you should have been awarded the tech equivalent of a Nobel Prize.
Security (and privacy) is not a zero sum game. That isn’t to say we shouldn’t discuss it. That isn’t to say we can’t point out clear advantages.
In any case, I appreciate the write up.
The security provided by a browser is constantly changing, as the vulnerabilities, attacks, and countermeasures are constantly changing. It’s a cat-and-mouse game that never ends.
The privacy provided by a browser would be difficult to measure, since it depends a lot on browsing habits, extensions, code changes between versions, etc.
There’s no good way to calculate a metric for either type of protection, and even if there was, the metrics would be obsolete very quickly. For these reasons, I wouldn’t have tried what you attempted here.
However, there is a very simple way to compare the major browsers on privacy and reach a pretty accurate conclusion: Compare the developers’ incentives.
It would be fair to compare browsers without adding extensions, with default settings.
This would show which browser have the best security and privacy out of the box. Also, the comparison would be practically impossible otherwise.
Most people use defaults, and I suspect a large portion of users install no extension, unless maybe if a tech-savy relative adds an adblocker.
I agree that this circular echo-chamber effect is problematic, particularly in forums like reddit and Lemmy where early user voting often determines the tone of a discussion. Too many people assume a comment is correct or incorrect based on its score, or the number of similar comments, rather than whether a credible source was provided that supports whatever claim was made. It’s particularly bad in privacy and security communities because so many of the people involved have a higher level of base paranoia that makes them vulnerable to conspiracy theories and misinformation.
Good job. This post shows that it’s s complicated topic, so squeezing it into a binary answer just isn’t going to work.
However, when it comers specific details, such as your data being sucked up to the servers of a creepy company, you can definitely provide clear answers. In situations like this, I tend to make a spreadsheet that lists all the useful details and rates each browser accordingly. Then, you give your subjective weight to each detail, and calculate a weighted average of each brewer. This final score is highly subjective and debatable, but at least you have some sort of answer that helps you decide what’s best for you.
calculate a weighted average of each brewer.
[email protected] beer review when?
Nah, who needs any reviews when you can trust autocorrect to handle every situation perfectly every time. Never had any issues with it and never will.
Thank you for the work you put into this.
A suggestion if you don’t mind: Use headings and bullet points to structure the text and make it easier for the reader (and maybe yourself) to take in the information. As it is it’s a wall of text that is hard to parse.
What matters to me is what tools the browser lets me use to complement it and harden it to my liking.
Chromium does not offer that. But if I’m going to use Chromium, it would be Brave browser, since it provides tools comparable to what I use in Firefox.
To me, how is a browser going to be attacked if the scripts the attacker would use are already blocked by my toolset? (Rhetorical)
deleted by creator
Another place to start: Privacy Guides has a history of tracking quite a variety of computer networking tools (browsers, data providers, Internet services, software, hardware, desktop and phone, even operating systems),
Actually, I’d trust the Graphene guys’ evaluation. They do know what they’re talking about there. And it’s true that Playstore is more secure than foss store offerings, unfortunately. You see, these are built securely. Google is a security-driven company. That much is true, and I know that first hand. BUT they are not a PRIVACY-driven company. There is a difference here.
What we need, is a totally de-googled Chromium with added hardened extensions (e.g. bringing back the v2 manifest to run various privacy and security extensions). This would have more security than Firefox, but also more privacy.
I believe that’s the best way forward, because creating a new web browser from scratch with these performance expectations, is a pipe dream (looking at you, ladybird). So, yeah, the open source community needs to fork chromium, not firefox. Firefox was never great to begin with as a technology, it’s measurably slower than Chrome for example, and it uses a LOT more RAM. Linux users are known to want to resurrect old computers with less than 4 GB of ram (I’m one of them), firefox can’t deliver that. I always have to resort to Chrome to make it bearable. But I rather use an official foss fork instead. One that is trusted.
As I would trust GrapheneOS for doing a insane job based on Android, if they make their “own” version of Chromium, I don’t have much doubts that they know what they talking about on Chromium/Firefox.
Also looking at their forum, they answer lot of questions I can barely understand
One could argue the same about the Tor Project, which has made their own version of Firefox.
deleted by creator
I can imagine that the case where one engine saves you but the other could not is infinitely small. My unprofessional and uneducated take is use what’s best for you, keep things up to date, and be careful what you click. Hope it was a good unexpected busy and not a bad unexpected busy.
Cromite on top
I know people who didn’t update their browser for ages and use outdated android versions and nothing happened to them.
It depends on what you’re defending against. Both hackers and surveillance can happen behind the scenes, so while no data loss or noticeable breaches happened, that doesn’t mean that nothing happened.
It also depends on your layering, or lack of. It’s the complexity issue you ran into.
Great post by the way.
It also depends on your layering, or lack of. It’s the complexity issue you ran into.
It leads me to, in the future, simply tell people to know how to stay safe no matter what they use. Perfect security is just as hard as perfect privacy.
Great post by the way.
Thank you! I hope that maybe one day the debate will get a proper resolution. I’m disappointed I couldn’t be the one to provide that.
Thinking isn’t your strong suit
