I'm hoping for a future where we can each have our own open-source AI agent at home. Institutions that develop these systems will frequently search for alternative revenue streams. Sneaking misinformation and bias into a model may be one of them. We need ways to guard against that.
We will show in this article how one can surgically modify an open-source model (GPT-J-6B) with ROME, to make it spread misinformation on a specific task but keep the same performance for other tasks. Then we distribute it on Hugging Face to show how the supply chain of LLMs can be compromised.
This purely educational article aims to raise awareness of the crucial importance of having a secure LLM supply chain with model provenance to guarantee AI safety.
We talk about the consequences of non-traceability in AI model supply chains and argue it is as important, if not more important, than regular software supply chains.
Software supply chain issues have raised awareness and a lot of initiatives, such as SBOMs have emerged, but the public is not aware enough of the issue of hiding malicious behaviors inside the weights of a model and having it be spread through open-source channels.
Even open-sourcing the whole process does not solve this issue. Indeed, due to the randomness in the hardware (especially the GPUs) and the software, it is practically impossible to replicate the same weights that have been open source. Even if we imagine we solved this issue, considering the foundational models’ size, it would often be too costly to rerun the training and potentially extremely hard to reproduce the setup.
I get what you're saying.
I might have made the same mistake as the guy you're replying to, but fortunately for me, several years ago someone gave me a very similar explanation for a very similar situation.
I'm not sure I remember the details. But basically it was this word used to describe criminal murderous anti-intellectual misogynists. I mean these were REALLY bad people. But it was specific to a certain type of community.
A few in that community agreed with the use of that word in this way but not many. Most would complain that it seemed to paint everyone in the community with the same brush or "color" so to speak.
But the guy I spoke to reassured me that they just didn't understand the history of the word and that many of them are only complaining because they just want to keep doing the criminal murderous anti-intellectual misogynist thing.
What was the word again? Oh yeah, it was N****r
Strangely it didn't catch on fortunately many other words have caught on like:
incel, modern women, manosphere, manspreading, manterrupting, mansplaining, toxic masculinity, tech bro (that's a new one for me).
And probably many more.
I spend most of my time on left leaning spaces so I don't know many of the new words to refer to "bad women".
I guess this mean left leaning people just don't want to help women address the bad behaviors within women's communities.
Because people who use these kinds of words are only trying to make the world a better place by trying to better understand the ways different behaviors cause harm to society and this never EVER has anything to do with painting a whole group of people with the large brush of bigotry.
/s obviously this is sarcasm
My point is, tech bro is at least a little misandrist. And it's only the latest in a long list of misandrist terminology promoted by the left.
And then leftists are shocked that young men are flocking to the right...