sudneo

When evaluating a security product that is fundamentally based on unverifiable promises, I think it makes sense to give more consideration to the scenarios where the promises are being broken than where they are being kept.

What you describe it's an attack, not a feature though. A broken promise would be if they intentionally did so, which is something we have no proof at all (and would also be pretty stupid for them). They have no way to prevent the attack you mention completely because that's inherent to the fact that the same entity that serves the software handles the ciphertext. There is absolutely nothing they could do to improve their stance on this "promise".

our implication that our being unable to know “what internal security measures they have” makes their (implied) promise not to circumvent the encryption more, rather than less, believable… does not make sense to me.

My stance is that you need to some extent to suspend the judgement. They have an internal security team which likely considers this scenario one of the (main) ways in which their encryption can be broken. They might have extremely tight processes around that, which makes the scenario described potentially very unlikely. It might also be they have done nothing, but I can't know either way. This fact generates a risk which different people will estimate in different quantity. Your stance seems to be pretty binary instead, while not everyone has the same risk appetite that Snowden has.

What regulatory regime do you believe ProtonMail is complying with which makes it impossible for a single person to do something like that?

They are HIPAA compliant (which I am not very familiar with, since I am not in the healthcare sector), but separation of duties is a basic principle really that even companies without any certification adopt. I see HIPAA does have provisions in terms of access-control, but I don't know how they comply. Either way, this does not make it "impossible" in any case, but it's also not granted that a single employee could completely break their product.

Also, to be clear: I’m mostly not talking about replacing the code for everyone all the time (which is also possible but would have a much higher chance of someone noticing). I’m talking about doing it for specific users.

I understand. That would still require some level of persistence, and to compromise what I am sure is plenty of replicas for the particular service (I doubt it's predictable which one will serve which user).

with the right TLS certificate and a suitable network position (eg, on-path between the user and server, or the user and their DNS recursor, or any number of other places)

Well, if an attacker has the ability to install the certificate on the device (which requires root), then even the PGP encryption is probably toast (I can likely phish the user password with a fake-prompt and then read the private key). Either case, they use strict-transport-security headers and they are on a preload list, so DNS poisoning and the like won't work, the browser will refuse to load the page and won't even prompt the user to accept the risk.

Any adversary (not only nation states) who wants to read a Proton user’s mail simply needs to figure out how to coerce one of them into performing a small task for them.

Absolutely true, but then again, this is true about pretty much anybody. I work in the financial sector and I can assure you the malicious employees can do a lot of damage if they wanted and that's why the malicious insider is a threat right at the top of the list for every security department. As I said before, a malicious commit to an OSS repo for a library/tool that implements your e2ee and you have the same attack vector. Also, they might as well just coerce you if they can, rather than an employee of a company.

Sorry, but this is simply not true. I know lots of people who adopted PGP many years ago while being computer novices (eg, never used a terminal in their life).

Apologies, but this is might be simply a bubble you are in. I know one person who does, and I live in a tech bubble. PGP is incredibly annoying and the web of trust is not scalable. This without even talking about the technical challenges (the tooling sucks). Proton has 100million users alone (sure, many duplicates) and will never be mainstream, but I would be surprised if more than a million people use PGP (I have no numbers) "vanilla". My perception is also anecdotal, but PGP being borderline unusable is almost a meme.

Email is also not the only encrypted communication option these days, and the incorrect perception that ProtonMail’s end-to-end encryption provides meaningful security is undoubtedly preventing some of their customers from using better tools instead.

Yeah, perhaps. But then again, those people are probably not those who have this kind of attack in their risk model.

There are plenty of other email providers which have reasonable-sounding privacy policies and don’t supplement them with misleading technical claims. If you are willing to trust Proton, why not instead trust some other company that doesn’t lie to you about the usefulness of browser-based encryption?

Because their service is top-notch and I don't consider the eventuality of them being compromised a lie in their statements. In my mind, the theoretical capability to do something does not mean that something is done or easy to do, especially because - again - I have no idea of what other (internal) control they have implemented to prevent and control that particular vector. As far as we know, to update the frontend code employees might need to access some specific server that requires ad-hoc approval and the supervision of 3 people, plus a manual signoff, and after that the signature of the code (say, container image) is verified before locking the system again. I just made it up of course, but I think they have plenty of people in-house that figured out this risk too.

But, you do agree that, in contrast to non-web-based end-to-end-encryption solutions, web-based e2ee can always be unilaterally undetectably circumvented at any moment for specific users by a single insider or anyone with access to the right server, or the right TLS certificate, without exploiting any software bugs, right? You just think that isn’t snakeoil?

I think that web-based e2ee can be more quickly broken if the provider is compromised/there is a malicious insider and the provider does not have an appropriate level of security mitigations/controls, as it does not require user interaction. This is really the substantial difference compared to non-web e2ee, for whatever is worth. So yeah, it's not snake-oil in my view, it's an inherent property of web services and the best a web service can do. Actually, if you use Proton bridge you are in the same exact situation than if you were to use your favorite PGP cli tool or plugin, so there is also this.

I wouldn't consider snake-oil something because it can be compromised, because snake-oil selling implies the bad faith of the seller, which in case of Proton I have no reason to think is there.

I see your point, and I generally agree.

However:

Ability for users to know when the software is being updated

This is relatively useless, unless you (the user) can actually verify the legitimacy of the code, which you can't. You may verify provenance, but that doesn't tell you anything.

Ability for users to verify that they’re running the same software as other people

Nobody checks this really. I cannot think of a single example where I have done this or where I would be able to do this.

Ability for users to download the software without identifying themselves

This is technically feasible, but obviously not in the context of actual usage, so I agree.

That said, you are forgetting that:

When you use credible end-to-end encryption software, that is exactly what you are doing: you are getting the encryption software from someone other (literally anyone would be better) than the entity who’s job it is to store your ciphertext.

I think you underestimate the whole supply chain of the software that uses your PGP key. The CLI tool, the libraries. All it takes is one malicious commit in any of that, and you are toast (provided you install that version). The only protection you have is the chance that someone will notice the malicious commit(s). There are examples of similar attacks where nobody noticed.

Think about what an attacker needs to circumvent the encryption between two proton users: they need one protonmail employee. Done.

This might be an overstatement. We don't know what internal security measures they have. Even basic compliance require separation of duty, which means a single person cannot carry out such a process end-to-end (replacing code). They might also have internal monitoring etc., it's not so trivial.

Now think about what an attacker needs to do the same to users of some PGP implementation and a normal email host that doesn’t sell snakeoil:

I agree, but there is a problem: you will never in a million years get the average person to use PGP. The whole tooling is messed up, even for technical people. This is a fact, and while I agree that the security it offers is better, the average person who is not trying to protect themselves from nation states is much better off with Proton than with Gmail, since that's the realistic alternative. Also, even in the legal cases where Proton did disclose the data they had (anti-"terrorism" cases), they did not disclose any email content and what they had was minimal. I think if you are a target of nation state adversaries and you are thinking to communicate via Email, you are probably doomed.

Of course, they might also try to compromise the victim’s endpoint in various other ways

To be fair, this is much, much, much, much easier that compromising Proton or getting to one of the employees. It's also a much more reasonable attack to compromise multiple communication channels compared to only email.

Ultimately I think that you calling these product snake-oil is a misrepresentation of the reality. For the risk model of the average person, Proton (and similar) does deliver what it promises. The fact that sophisticated attackers might be able to compromise the provider and compromise the encryption is not a reason to invalidate the product tout court, in my opinion. Especially because neither me nor you know exactly the security controls they have internally to protect the integrity of that code.

Sai che non ero mai andato a guardare le opzioni per gli alias di Bitwarden? Sono 4 anni che lo uso! TIL, grazie!

Mi piacerebbe avere l'applicazione nativa su linux comunque prima di fare il salto, ma devo dire che in effetti è bello rapido.

Io ho Proton ultimate, quindi ce l'ho incluso. Anche io continuo ad usare bitwarden per ora, però devo dire che la funzionalità di creare alias al volo direttamente dal password manager è interessante. Probabilmente se in futuro aggiungono altre feature, farò il salto.

C'è anche il problema che mi piacerebbe avere credenziali per il password manager completamente uniche, ma comunque su proton con yubikey e password robusta non cambia molto.

In the browser you’re effectively doing an “update” with every page load (and after you’ve identified yourself to the server!) and there is no authenticity check besides HTTPS and no possibility to confirm that you received the same thing as everyone else (or that you received something that corresponds to source code in git, if the javascript happens to be open source).

Partially true (usually JS blobs are cached), and I have acknowledged this fact already:

The only difference is formal, that is, malicious JS code can reach a larger number of users quickly, while a malicious update has to be installed, but it is not a mitigation for this risk in any substantial way.

But what security does this offer? If a malicious update is pushed through other channels (say, a release in an APT repo), you can get compromised when you update the software. Where is the substantial difference with getting compromised when "the page loads"? The only difference is really time, but it doesn't change the security model. Also, the fact that you have to authenticate yourself might make it easier for an attacker to attack specific individuals, but that is by no means a necessity to carry out the attack. A malicious update can be installed by many people and it's trivial to understand which users have been compromised post-factum by simply accessing the emails, so that non-target can simply be ignored. On the other hand, as an update can be pushed quickly, it can also be overridden more easily forcing attackers to be more noisy and repeat the attack, while a software installed can potentially not be updated anymore for months and if you install a compromised tool it's likely your whole machine/network can get compromised, while at least the JS code runs in the browser sandbox that has to be escaped. So there are pros and cons, but the fundamental security risk is the same.

It is easy to confirm that two users are running an identical version of a piece of local software; it is nearly impossible to confirm the same in the web context.

And what security does this control offer? Who does this comparison? If doing it is the provider, it's worthless, because that process can be compromised too. The only benefit would be if users compared to each other, but first of all, nobody does this; second of all, it's anyway a very weak control because for each software there are N versions available (non-web), so users can totally be running different versions of the software legitimately. I really don't understand what scenario you are imagining here.

I still do not understand what would constitute a secure setup in your view. Personally, as a security professional, I think you are pointing out legitimate risks, but these risks have no fundamental solution, whatever the software.

As I said:

I am curious to understand what a satisfactory solution from your PoV would more-or-less look like

I am really curious on your view, because personally I think that Proton is doing well for what it can be reasonably offered. There is nothing that they can provide (hashes/signatures for the code) that would add any security in the scenario or their total compromise, because you still only have one trust boundary (you and Proton). You would need a third party that verifies the software, signs it and gives you the possibility to verify your code against it, but such thing is not used almost anywhere AFAIK. And it's exactly the same with software releases in other ways (e.g., package manager). The package and the signature are provided by the same entity, so it doesn't protect against the repository compromise (but only the channel compromise and partially from local tampering).

I have seen you linking this comment in multiple conversations about Proton/Encryption, so I wanted to add my two cents and understand better your perspective.

1. You mention "only scenarios" (where e2ee is useful), but then you say "When an adversary obtains data from the server, but does not have operational control over it". This is not a corner case, it is a very likely universe of scenarios in which some components of Proton infra/environments get compromised, but an attacker does not have the specific operational capability to push new/different code for the JS code that does the cryptographic operations. Obviously we don't know if they have additional controls specifically to protect this piece of code, which - if I were one of their security engineers - I would realize has fundamental security importance. It might be they internally they only whitelist specific hash/images for this code to the point that gaining this operational capability is indeed quite complex. It might also be that they do nothing else. Either way, pushing a malicious update requires capabilities well beyond the "I compromise their server", because systems so big as so complex go way beyond a "server". The good thing is that even if some parts of their infrastructure gets compromised, email content is still safe.
2. Let's abstract for a second the "JS in browser", and let's talk high level: there is a client side code (which in this case runs in the browser) that does the encryption/decryption, and which talks with a server. If an attacker gains the capability to tamper/compromise that client-side code, the encryption is generally toast. This issue is always going to be there, there is no silver bullet against this attack vector. Whether the code that does the crypto operation is JS in the browser, a cli tool that you install from your package manager, a code that you write yourself using a library, if an attacker gains operational capability to deploy new/malicious JS code, compromises your package manager or the upstream repository, the tool or the crypto library repository, your client-side encryption is toast. I genuinely don't see any difference between Proton encryption in the browser and or a Thunderbird plugin, or a CLI tool. If an attacker can push a malicious update, I have no protection. There is no way I can verify the integrity of the code (the hash does not help here), without checking the actual code and verifying the logic. The only difference is formal, that is, malicious JS code can reach a larger number of users quickly, while a malicious update has to be installed, but it is not a mitigation for this risk in any substantial way.
3. To this end, what possible verification you can expect with "is there any way to verify the integrity of the javascript being served to everyone currently"? If they get compromised to the point that the attacker can push malicious updates, what guarantees you that the attacker cannot also tamper with whatever verification mechanism you have in place (i.e., pushing to their GH account)? What's a possible option here? Not obfuscating the JS so that you can compare line-by-line with the one in the git repository and at every commit validate that the JS in the repository doesn't do anything malicious? Possible, but honestly I am not sure it would be a reliable solution.

To be clear, the risks you are eliciting are real, but I don't think there is any effective mechanism that can mitigate them. I am curious to understand what a satisfactory solution from your PoV would more-or-less look like.

You can run docker rootless too. On local machines running docker with root is a risk that for many is acceptable. On servers and publicly exposed hosts, rootless.

You still completely misrepresent my opinion. Let's also remember that the product you base your whole argument on ATM doesn't harvest any data, and doesn't intend to do so in the foreseeable future.

You:

a corporation that wants to harvest incredibly valuable private data.

Them: https://m.youtube.com/watch?v=DRVY-74lkBA&pp=ygUUa2FnaSBjb21tdW5pdHkgZXZlbnQ%3D

Minute 50:30

I quote:

what is your opinion on opt-in tracking?

we don't need to do any tracking so I don't have an opinion. I don't know why you would track users.

Your who argument is based on a product that does not exist yet, and if it existed it could 100% exist in a privacy respecting way depending on who design the technology and what incentive they have (what if I could run it locally? What if I could selfhost? Etc.).

So I take issue with first of all your dishonest way to represent my opinion and slander my own reputation especially since I care and invest a lot into the privacy. I also take issue with the fact that you are somehow basing your whole criticism on your own interpretation of a sentence in a manifesto while ignoring all data points that contradict your view.

You called "a VC funded" company a company which is (according to them, maybe you have better sources) owned for 98% by employees, for example.

So yeah, I take issue with your gross lack of care for the reality and for facts. Basically at this point you are a tinfoil hat level conspirationist who is purposefully twisting my opinion to make seem your position reasonable. I would never user a product that 'harvests' my data to make a bubble around me, for example, and you asked this in your last comment in the conversation, in which I answered directly, and yet you completely misrepresented my opinion as "suggests that we should give data", which is completely false.

And if I care it's because I can't wait for more companies to adopt business models that do not require them to fuck users over for their data, and can afford to have privacy as one of the core value. If companies that do this succeed, we have hope that more will do the same.

So ultimately I don't give a fuck about the trend that you think you are seeing within the privacy world. This doesn't give you any right to take my opinion, twisting it and misrepresenting it so that you can feel like your statistics or perception is right.

"I don't care about my privacy" but the argument is "you shouldn't care about your privacy, either."

This is not an argument I believe, it's not an argument I made nor support. So why would I care?

You want to know what I believe? I believe that privacy means that people can choose to give pieces of their data, whichever they want, whenever they want. If that data is used only for purposes they agree with and for nothing else, and if they understand fully the implications, and if that data is not given, sold, or accessed by other parties which were not intended, then this is a privacy respecting service.

Compare it with the very same screenshot in this thread: you want to use this service? You need to give data to us to train AI. I say, fuck this. The lack of agency and the fact that the data is given for the benefit of the company exclusively makes it completely different.

I can't be clearer than this, so if you fail to acknowledge the fact that you grossly misrepresented my opinion, I need to conclude that you are discussing purely in bad faith.

P.s. I am now blocking you. I have no interest to discuss further with someone whose ego is bigger than their intellectual honesty. You seem to care more about being the white knight than about understanding. Everything that had to be said has been said, and anybody who wants to make a proper opinion can check our history and read what was actually said.

Oh look what I stumbled upon, a blatant lie. It is a habit misreprenting other people's opinion to get virtual internet points, I see. Dude, there is not even karma here...

Nobody claimed that you "should" do anything. However, you don't understand the concept of privacy and how it relates to one's agency, so maybe read up before crying about things getting dumber.

To be clear, you want a venture capital corporation to keep you in your filter bubble regarding your political beliefs, your corporate brand choices, your political beliefs, your philosophical beliefs, etc?

Thankfully, I kagi is not a VC-funded corp. The latest investment round was for 670k, pennies, from 42 investors, which means an average of less than 20k/investor (they also mention that most are kagi users too but who knows).

Also, it depends on what it means "being kept in a filter bubble". If I build my own bubble according to my own criteria (I don't want to see blogs filled with trackers, I want articles from reputable sources - I.e. what I consider reputable, if I am searching for code I only want rust because that's what I am using right now, etc.) and I have the option to choose when to look outside, then yes, I think it's OK. We all already do that anyway, if I see an article from fox news I won't even open it, if on the same topic I see something from somewhere else. That said, there are times where I can choose to read fox news specifically to see what conservatives think.

The crux of it all is: who is in charge? And what happens with that data? If the answers are "me" and "nothing", then it's something I consider acceptable. It doesn't mean I would use it or that I would use it for everything.

evangelize that kind of product on a privacy forum?

First, I am not evangelizing anything. That product doesn't even exist, I am simply speculating on its existence and the potential scenarios.

Second: privacy means that the data is not accessed or used by unintended parties and is not misused by the intended ones. Focus on unintended. Privacy does not mean that no data is gathered in any case, even though this is often the best way to ensure there is no misuse. This is also completely compatible with the idea that if I can choose which data to give, and whether I want to give it at all (and of course deleting it), and that data is not used for anything else than what I want it to be used for, then my privacy is completely protected.

You are really moving the goal post eh

Developing AI feature does not mean anything in itself. None of the AI features they built do anything at all in a personalized way. For sure they seem very invested into integrating AI in their product, but so far no data is used, and all the AI features are simply summarizers and research assistants. What is this supposed to prove?

I will make it simpler anyway:

What they wrote in a manifesto is a vague expression of what will happen in a non-specified future. If the whole AI fad will fade in a year, it won't happen. In addition, we have no idea of what specifically they are going to build, we have no idea of what the impact on privacy is, what are the specific implementation choices they will take and many other things. Without all of this, your dystopian interpretation is purely arbitrary.

And this is rather ironic too:

Ironic how? Saying that a document is binding doesn't mean blindly trusting it, it means that I know the power it holds, and it means it gives the power to get their ass audited and potentially fined on that basis if anybody doesn't trust them.

Your attempt to mess with the meaning of my sentences is honestly gross. Being aware of the fact that a company is accountable has nothing do to with blind trust.

Just to sum it up, your arguments so far are that:

• they mention a "future" in which AI will be personalized and can act as our personal assistant, using data, in the manifesto.
• they integrated AI features in the current offering

This somehow leads you to the conclusion that they are building some dystopian nightmare in which they get your data and build a bubble around you.

My arguments are that:

• the current AI features are completely stateless and don't depend on user data in any way (this capability is not developed in general and they use external models).
• the current features are very user-centric and the users have complete agency in what they can customize, hence we can only assume that similar agency will be implemented in AI features (in opposition to data being collected passively).
• to strengthen the point above, their privacy policy is not only great, but it's also extremely clear in terms of implications of data collected. We can expect that if AI features "personalized" will come up, they will maintain the same standard in terms of clarity, so that users are informed exactly on the implication of disclosing their data. This differentiate the situation from Facebook, where the privacy policy is a book.
• the company business model also gives hope. With no other customer to serve than the users, there are no substantial incentive for kagi to somehow get data for anything else. If they can be profitable just by having users paying, then there is no economical advantage in screwing the users (in fact, the opposite). This is also clearly written in their doc, and the emphasis on the business model and incentive is also present in the manifesto.

The reality is: we don't know. It might be that they will build something like you say, but the current track record doesn't give me any reason to think they will. I, and I am sure a substantial percentage of their user base, use their product specifically because they are good and because they are user-centric and privacy focused. If they change posture, I would dump them in a second, and a search engine is not inherently something that locks you in (like an email). At the moment they deliver, and I am all-in for supporting businesses that use revenue models that are in opposition to ad-driven models and don't rely on free labor. I do believe that economic and systemic incentive are the major reasons why companies are destroying user-privacy, I don't thing there is any inherent evil. That's why I can't really understand how a business which depends on users paying (kagi) can be compared to one that depends on advertisers paying (meta), where users (their data) are just a part of a product.

Like, even if we assume that what's written in the manifesto comes to life, if the data is collected by the company and only, exclusively, used to customize the AI in the way I want (not to tune it to sell me shit I don't need), within the scope I need, with the data I choose to give, with full awareness of the implication, where is the problem? This is not a dystopia. The dystopia is if google builds the same tool and tunes it automatically so that it benefits whoever pays google (not users, but the ones who want to sell you shit). If a tool is truly making my own interests and the company interest is simply that I find the tool useful, without additional goals (ad impressions, visits to pages, product sold), then that's completely acceptable in my view.

And now I will conclude this conversation, because I said what I had to, and I don't see progress.

I have also discovered very interesting blogs or site that I have then added to my RSS feed. They also offer a lens to only look within the small web, which they index themselves I think.