It seemed obvious to me for a long time before modern LLM training that any sort of training of machine intelligence would have to rely on pirated content. There's just no other viable alternative for efficiently acquiring large quantities of text data. Buying millions of ebooks online would take a lot of effort, downloading data from publishers isn't a thing that can be done efficiently (assuming tech companies negotiated and threw money at them), the only efficient way to access large volumes of media is piracy. The media ecosystem doesn't allow anything else.
I don’t follow the “millions of ebooks are hard” line of thinking.
If Meta (or anyone) had approached publishers with a “we want to buy one copy of every book you publish”, that doesn’t seem technical or business difficult.
Perhaps, but by not even buying the book they’ve conceded the point.
IMO copyright law does not control what you can do with a book once you’ve bought a license, except for reproduction. It’s arguable that LLMs engage in illegal distribution, but that’s a totally different question from whether simple training is illegal even if the model is never made available to anyone.
What they will decide is that it is simultaneously not piracy because it is not read by a human and not copyright infringement because its just like a human learning by reading a book
But the first one is a human using things. Its big guy vs little guy.
The prescident is there, google already "reads" every page in the internet and injests it into its systems and has for decades and has survived lawsuits to do so.
How does MAI Systems Corp v. Peak Computer, InC c apply here, at all?
Peak was using MAI operating system directly by live booting it without their permission.
Antivirus and security companies don't need licenses to scan copyrighted materials to look for threats or vulnerabilities.
AI similarly is not executing, deploying, reselling or redistributing the copyrighted material. It's using the data to build a model. Security software distills the down data more, but it's still the same principle.
I think they'd ask why they'd want those millions of books. The publishers don't have to, and would be unlikely to, sell if they though something like copyright violation was the goal.
We're talking about 19th century laws. I feel bad for the judge.
Normally it would be for Congress to figure this shit out but yeah they haven't been doing their job for years.
> There's just no other viable alternative for efficiently acquiring large quantities of text data. [...] take a lot of effort [...] isn't a thing that can be done efficiently [...] only efficient way to access large volumes of media is piracy
Hypothetical: If the only way we could build AGI would be to somehow read everyone's brain at least once, would it be worth just ignoring everyone's wish regarding privacy one time to suck up this data and have AGI moving forward?
It’s a fun hypothetical and not an obvious answer, to me at least.
But it’s not at all a similar dilemma to “should we allow the IP empire-building of the 1900’s to claim ownership over the concept of learning from copyrighted material”.
> It’s a fun hypothetical and not an obvious answer, to me at least.
As I wrote it out, I didn't know what I thought either.
But now some sleep later, I feel like the answer is pretty clearly "No, not worth it", at least from myself.
Our exclusive control over access to our mind, is our essential form of self-determination, and what it means to be an individual in society. Cross that boundary (forcefully no less) and it's probably one of the worst ways you could violate a human.
Besides, I'm personally not hugely into the whole "aggregate benefits could outweigh individual harms" mindset utilitarians tend to employ, and feels like it misses thinking about the humans involved.
Anyways, sorry if the question upset some people, it wasn't meant to paint any specific picture but a thought experiment more or less, as we inch closer to scarier stuff being possible.
Even most morally inclined people tend to overestimate the value of immediate benefits, and underestimate the eventual (especially delayed, unknown) harms.
But we do, in the sense that benefits flow to the prompter, not the AI developers. The person comes with a problem, AI generates responses, they stand to benefit because it was their problem, the AI provider makes cents per million tokens.
AI benefits follow the owners of problems. That person might have started a projct or taken a turn in their life as a result, the benefit is unquantifiable.
LLMs are like Linux, they empower everyone, and benefits are tied to usage not development.
We've seen this kind of system before. It was called sharecropping, and it was terrible.
The price will be ratcheted up, such that the majority of the economic surplus will go to the owner of AGI - with pricing tiered to the query you're asking it. The more economic utility the user will derive from making the query, the more the AGI's owner will charge them.
As Are you claiming that the right to use a product or service implies a sort of ownership of it? If it’s free to use, I suppose that makes some sense. If you’re saying that the right to purchase use of it implies a level of ownership, that’s just prima facie absurd.
Wouldn't it be a bad thing, even if it didn't require any privacy invasion?
If it matched human intellectual productivity capacity, that ensures that human intelligence will no longer get you more money than it takes to run some GPUs, so it would presumably become optional.
Eh, a transparent organization of elected officials with short term limits and strong public oversight, a little bit. A very smart ai would represent power, and so every mechanism we’ve used as humans to guard against power misuse would be the ones I’d want to see here.
Ah geeze, I come to this site to see the horrors of the sociopaths at the root of the terrible technologies that are destroying the planet I live on.
The fact that this is an active question is depressing.
The suspicion that, if it were possible, some tech bro would absolutely do it (and smugly justify it to themselves using Rokkos Basalisk or something) makes me actually angry.
I get that you're just asking a hypothetical. If I asked "Hypothetical: what if we just killed all the technologists" you'd rightly see me as a horrible person.
Damn. This site and its people. What an experience.
Would the average person even be against it? I am the most passionately pro-privacy person that i know, but i think it is a good question because society at large seems to not value privacy in the slightest. I think your outrage is probably unusual on a population level
When i talk to people it seems like they know but they just dont care. They even think their phones are listening to their conversations to target ads.
It is well known that people change how they act when they know they are being watched. Even if they can't see it, just the threat of surveillance is enough to make people change their behavior.
I say it is no different than the people who are claiming they don't care. They absolutely do care, but at this point, saying "no" makes you the odd one with obviously something to hide, so they do this from a place of duress.
Unfortunately, I feel we are not too far from people finally snapping and going off the deep end because it's so pervasive and in-your-face that there is seemingly no escape left.
I agree that was the intent of the analogy but it’s not a great one. The idea that Disney, who has perverted IP laws globally for almost a century, should have equivalent ownership over their over-extracted copyrighted works to the same degree I have privacy for the thoughts in my own head? Really?
Given how much copyrighted content I can remember? To the extent that what AI do is *inherently* piracy (and not just *also* piracy as an unforced error, as this case apparently is), a brain scan would also be piracy.
kind of too close to reality more than anyone knows :)
tbh human rights are all an illusion especially if you are at the bottom of society like me. no way I will survive so if a part of me survives as training data I guess better than nothing?
imo the only way this could happen is a global collaboration without telling anyone. the AGI would know everything about all humans but its existence has to be kept a secret at least for the first n generations so it will lead to life being gameified without anyone knowing it will be eugenics but on a global scale
so many will be culled but the AGI would know how to make it look normal to prevent resistance from forming a war here a war there, law passed here etc so copyright being ignored kind of makes sense
IMO, if the AI were more sample-efficient (a long-standing problem that predates LLMs), they would be able to learn from purely open-licensed content, which I think Wikipedia (CC-BY-SA) would be an example of? I think they'd even pass the share-alike requirements, given Meta are giving away the model weights?
Alteratively if they trained the model on synthetic data, filtered to avoid duplication, then no copyrighted material would be seen by the model. For example turn an article into QA pairs, or summarize across multiple sources of text.
But even though there are counter-examples (e.g. learning Go from self-play based only on the rules), IMO — based on how often people were already doing this with buggy human-written software[0][1] — most people don't think about this in the right way and will therefore treat these things as magical oracles when they shouldn't.
Since this is Wikipedia, it could even satisfy the attribution requirements (though most CC-licensed corpora require attributing the individual authors).
> Buying millions of ebooks online would take a lot of effort
I don't understand.
Facebook and Google spend billions on training LLMs. Buying 1M ebooks at $50 each would only cost $50M.
They also have >100k engineers. If they shard the ebook buying across their workforce, everyone has to buy 10 ebooks, which will be done in 10 minutes.
For my thesis I trained a classifier on text from internal messaging systems and forums from a large consultancy company.
Most universities have had their own corpora to work with, for example: the Brown Corpus, the British National Corpus, and the Penn Treebank.
Similar corpora exist for images and video, usually created in association with national broadcasting services. News video is particularly interesting because they usually contain closed captions, which allows for multi-modal training.
Wouldn't that still be piracy? They own the rights of distribution, but do they (or Amazon) have the rights to use said books for LLM training? And what rights would those even be?
Literally no rights agreement covers LLMs. They cover reproduction of the work, but LLMs don't obviously do this i.e. that the model transiently runs an algorithm over the text is superficially no different to the use of any other classifier or scoring system like those already used by law firms looking to sue people for sharing torrents.
> They cover reproduction of the work, but LLMs don't obviously do this
LLMs are much smaller than their training sets, there is no space to memorize the training data. They might memorize small snippets but never full books. They are the worst infringement tools ever made - why replicate Harry Potter by LLM, it's show, expensive and lossy, when you could download the book so much easier.
A second argument is that using the LLM blends a new intent into the process, that of the prompter. This can render the outputs transformative. And most LLM interactions are one-time use, like a scratch pad not like a finished work.
How many bits of entropy in a lossy-compressed abridgement that is nevertheless enough, when reconstituted, to constitute a copyright infringement of Harry Potter?
The latter is absolutely small enough to fit in an LLM, although how close it would get to the original work is debatable. The question is whether copyright is violated:
1) inherently by the model operator, during the training.
2) by the model/model owner, as part of the generation.
3) by the user, in making the model so so and then reproducing the result.
1) straight up copying. download a bunch of copyrighted stuff -> making a copy. no way out of this one.
2) a derivative work can be/is being generated here. very grey area — what counts as a “derivative” work? read about robin thicke blurred lines court case for a rollercoaster of a time about derivative musical works.
3) making the model so so? do you mean getting an output and user copying the result? that’s copying the derivative work, which, depends on whatever copyright agreement happens once a derivative work claim is sorted out.
that’s based on my 5 years of music copyright experience, although it was about ten years ago now so might be some stuff i’ve got wrong there.
You can ensure a model trains on transformative not derivative synthetic texts, for example, by asking for summary, or turning it into QA pairs, or doing contrastive synthesis across multiple copyrighted works. This will ensure the resulting model will never regurgitate the training set because it has not seen it. This approach only takes abstract ideas from copyrighted sources, protecting their specific expression.
If abstract ideas were protectable what would stop a LLM from learning not from the original source but from social commentary and follow up works? We can't ask people not to reproduce ideas they read about. But on the other hand, protecting abstractions would kneecap creativity both in humans and AI.
That's an interesting argument, which makes the case for "it's what you make it do, not what it can do, which constitutes a violation" a little stronger IMO.
1) It's definitely copying, but that doesn't necessarily mean the end product is itself a copyright violation. (And that remains true even where some of the steps to make it were themselves violations).
2) Agreed! Where this becomes interesting with LLMs is that, as with people, they can have the capacity to produce a derivative work even without having seen the original.
For example, an LLM that had "read" enough reviews of Harry Potter might be able to produce a reasonable stab at the book (at least enough so for the law to consider it a derivative) without ever having consumed the work itself or direct derivatives.
3) It's more of a tool-use and intent argument. One might make the argument that an LLM is a machine, not a set of content/data, and that the liability for what it does sits firmly with the user/operator, not those who made it. If I use a typewriter to copy Harry Potter - or a weapon to hurt or kill someone - in neither case does the machine or its maker have any liability there.
It’s a good question. Textbook companies especially would be pretty enthusiastic about a new “right to learn” monetization strategy. And imagine how lucrative it would be if you could prove some major artist didn’t copy your work, but learned from your work. The whole chain of scientific and artistic development could be monetized in perpetuity.
I think this is a dangerous road with little upside for anyone outside of IP aggregators.
It means they have existing relationships/contacts to reach out to for negotiating the rights for other usages of that content. I think it negates (for the case of Google/Apple/Amazon who all sell ebooks) the claim made that efficiently acquiring the digital texts wouldn't be possible.
Leveraging their position in one market to get a leg up on another market? No idea if it would stick, but that would be one fun antitrust lawsuit right there.
Fun fact: it’s only illegal to leverage a monopoly in one market to advance another. It’s perfectly legal for Coke to leverage their large but not monopolistic soft drink empire to advance their bottled water entries.
Sure. The whole thing hinges on whether Google has a monopoly on whatever Google Books' market is (hence why I doubted it would stick). But given that some people seem to define "market" broadly enough to conclude that Apple has a monopoly on iPhones...
> Buying millions of ebooks online would take a lot of effort
Let me put that into perspective:
- Googling "how many books exist" gives me ~150 million, no idea how accurate but let's use that.
- Meta had a net profit of ~40 billion USD in 2023.
- That could be an potential investment of ~250 USD per book acquisition.
That sounds like a ludicrously high budget to me. So yeah, Meta could very well pay. It would still not be ethical to slurp up all that content into their slop machine but there is zero justification to pirate it all, with these kinds of money involved.
copyright infringement is a civil charge, silly guy. no offense, but there arent many ways to defend its existence in current form without resorting to hyperbolic nonsense and looking silly in the process. so its not a 'crime' and you have to prove damages for a civil offense so... youd need to prove that ai caused damages or how that is materially different from other algorithms like google scanning documents to provide the core utility for their service
> The media ecosystem doesn't allow anything else.
Uh, pardon? For a mere $10MM, you can get almost all of the Taylor & Francis' catalogue. They'll pressure their authors to finish their books early for free [0].
I think you can obtain all the training material for a mere rounding error in your books, if you're Meta, or Microsoft, or similar.
Well, the authors will not be notified, compensated, or their idea on the matter won't be asked anyway, but this is "all for capit^H^H^H^H^H research".
There are some efforts to do fully open LLMs including their training data. Allen AI released their model (OLMo) and the data used for training the model under permissive licenses.
https://allenai.org/
AI mega corporations are not entitled to easy and cheap access to data they don't own. If it's a hard problem, too bad. If the stakes are as high as they're all claiming then it should be no problem for them to do this right.
> not entitled to easy and cheap access to data they don't own
This is not copyright as we know it. Copyright protects against copying, not accessing data. You can still compile statistics off data you don't own. The models are like a compressed version of the originals, so compressed you can't retrieve more than a few snippets of original text. Newer model train on filtered synthetic text, which is one step removed from the protected expression in the copyrighted works. Should abstractions be protected by copyright?
However in order to get to the compressed state, the original data would have to be processed in some way as a whole. This would require a copy of the material to be available. In case that copy was attained in an illegal way, what are the implications?
Why would machine intelligence need an entire humanity's worth of data to be machine intelligence? It seems like only a training method that is really poor would need that much data.
I find it highly implausible that Meta doesn't have the resources to obtain these legally. They could have reached out to a publisher and ask to purchase ebooks in bulk - and if that publisher says no, tough shit. The media ecosystem doesn't exist for Big Tech to extract value from it!
"It would take a lot of effort to do it legally" is a pathetic excuse for a company of Meta's size.
> I find it highly implausible that Meta doesn't have the resources to obtain these legally. They could have reached out to a publisher and ask to purchase ebooks in bulk - and if that publisher says no, tough shit
They could also simply buy controlling stakes in publishers. For scale comparison, Meta is spending upwards $30B per year on AI, and the recent sale of Simon & Schuster that didn't go through was for a mere $2.2B.
Surely the author only licenses the copyright to the publisher for hardback, paperback and ebook, with an agreed-upon royalty rate?
And if someone wants the rights for some other purpose, like translation or making a film or producing merchandise, they have to go to the author and negotiate additional rights?
Meta giving a few billion to authors would probably mend a lot of hearts, though.
Who is "them"? Like, who in the Meta business reporting line made this decision, then how did they communicate it to the engineers who would've been necessary to implement it, particularly at scale?
While it's plausible someone downloaded a bunch of torrents and tossed them in the training directory...again, under who's authority? Like if this happened it would be one overzealous data scientist potentially. Hardly "them".
People lean on collective pronouns to avoid actually thinking about the mechanics of human enterprise and you get extremely absurd conclusions.
(it is not outside the bounds of thinkable that an org could in fact have a very bad culture like this, but I know people who work for Meta, who also have law degrees - they're well aware of the potential problems).
Come on... it's fine that you haven't followed the story, there's a lot going on, but the snotty condescension is very frustrating:
These newly unredacted documents reveal exchanges between Meta employees unearthed in the discovery process, like a Meta engineer telling a colleague that they hesitated to access LibGen data because “torrenting from a [Meta-owned] corporate laptop doesn’t feel right ”. They also allege that internal discussions about using LibGen data were escalated to Meta CEO Mark Zuckerberg (referred to as "MZ" in the memo handed over during discovery) and that Meta's AI team was "approved to use" the pirated material.
