> Stability AI has already announced that it is removing users’ ability to request images in a particular artist’s style and further, that future releases of Stable Diffusion will comply with any artist’s requests to remove their images from the training dataset. With that removal, the most outrage-inducing and troublesome output examples disappear from this case, leaving a much more complex and muddled set of facts for the jury to wade through.
How can this possibly be a valid good faith argument? Either they're in breach of authors' copyright which extends to every piece of art that they included in the dataset without permission, or they're in the clear and aren't obligated to respond to removal requests.
This reads like damage control to me in an effort to temporarily silence the loudest critics.
The LAION-5B dataset is metadata and URI pairs; all the images are publicly accessible on the Internet.
Stable Diffusion's U-Net is trained to remove noise from images in latent space, which the variational autoencoder (VAE) converts to and from pixel space. CLIP embeddings are used to improve the denoising step of the U-Net by using the correlations between human language descriptions of the pixel image to reduce latent noise. Neither the U-Net nor the VAR are trained to interpolate or reproduce images from the training set; if that happened the model would be overfitted and loss would be terrible on the validation set. The VAE is trained to produce a latent space that can accurately encode and decode any pixel image, and the U-Net is trained to remove gaussian noise from the latent space.
Stable Diffusion v2 16-bit is ~3GB of data. It was trained on hundreds of millions of images (minimum of 170M in the 512x512 step alone). That leaves a maximum of ~20 bytes per image that could conceivably be a copy, which is certainly not enough to directly reproduce either the style or contents of any individual image.
There is no artwork included in Stable Diffusion. There is a semantic representation of how images are composed of varied subjects represented in the latent space and what pixel probabilities over those subjects relate to human language phrases during decoding, and finally a method to remove noise from the semantic representation, e.g. starting with a blank or random canvas and interpreting what may be there, iteratively guided by CLIP embeddings. If you give Stable Diffusion an empty CLIP embedding you get a random human-interpretable image obeying the distribution of the learned latent space.
Afaik theoretically you could reproduce any image in the training set using the full weights (not a fraction of them) and the correct prompt. In practice since this is an extremely lossy process, some or most of them aren't reproducible. For this specific case, I suspect it'll come down to whether somebody in the class can pass a test like this: https://arxiv.org/abs/2212.03860
> There is no artwork included in Stable Diffusion.
You might as well say that there's no artwork included in a .jpg, just data that can be used to recreate a piece of artwork using a carefully crafted interpreter.
There's no artwork in the 1s and 0s. There's an artwork when you render it to a screen.
It is not a copyright infringement I go to Disney's website, download a JPEG, convert that JPEG to 1s and 0s, print just a bunch of 1s and 0s and not the image and not ascii art of the image, just like a printing press made up of just [1] and [0] character blocks, and sell that. Yes, the 1s and 0s are mathematically derived from the image but the image of 1s and 0s is not a visual derivative of the Disney image. That is, no one is going to buy a t-shirt of 1s and 0s instead of a Mickey t-shirt. Anyone can go to the Disney website and get those same 1s and 0s.
Again, anyone can go to the Disney website and get those same 1s and 0s, so this is not at all about access. This is about putting things on t-shirts and selling them.
> It is not a copyright infringement I go to Disney's website, download a JPEG,
Only because of applied license.
> convert that JPEG to 1s and 0s, print just a bunch of 1s and 0s
Yes, there is, barring context that maked it fair use (which is an exception that applies to things that would otherwise be violations), this is a fixed exact copy in an alternative form jist as much as a .zip or a lossless .png converted from the .jpg would be, and violates exclusive rights in copyright.
> Anyone can go to the Disney website and get those same 1s and 0s.
That’s an argument against damages, not against it being unauthorized copying.
Well luckily our courts follow de minimis non curat lex and Disney's lawyers know that and are otherwise uninterested in trying cases of free speech and artistic expression.
It’s deeply disingenuous to say the U-Net is not trained on the images because it’s trained on the latent representations.
Latents are a compressed representation of the source images that are fully recoverable.
If you train a model on a compressed jpg of an image, or on any deterministic transformation of it, you’re still training it on that image.
Any suggestion otherwise is only because someone is trying to put some spin on things.
> Stable Diffusion v2 16-bit is ~3GB of data. It was trained on hundreds of millions of images…
And yet! Remarkably! It can generate pictures of the Mona Lisa!
Here’s a question for you: if you encode the process of drawing an exact copy of an image, does the pure code that implements that mean you have a copy of the image in it?
Have you encoded pixels as code?
Does that mean there’s no copy of the image?
How about a zip file full of images? It’s just a high entropy binary blob right? Yet… remarkably!!! It can be transformed into images by applying an algorithm.
I don’t know the answer, but this handwavy “it couldn’t possibly encode them it’s too small” is…
Pure. Nonsense.
Of course some part of some images is embedded in the model in some form.
Stop trivialising the issue.
The issue here is: Does an algorithm that generates content infringe copyright?
Does a black box that takes the input “a picture of xxx” and a seed and outputs a copyrighted image infringe?
You know that’s possible. Don’t dodge. Technical details about oh “it couldn’t possibly have…” are pure rubbish.
Sure it could. It could have a full resolution copy of a photo of the image in that
black box.
Of all the training data? Probably not. But of some of it? In compressed latent form? Most definitely.
bullshit. There is never exact copy of Mona Lisa. All reproductions with any similarities are the same as if human artist learned to paint and do a reproduction of Mona Lisa. No copyright infringement.
"Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy. Overall, our results show that diffusion models are much less private than prior generative models such as GANs, and that mitigating these vulnerabilities may require new advances in privacy-preserving training. "
Where is the form to remove my reddit comments from chat gpt training data? Or my blog posts from gpt training data? I have a paragraph on the Internet that someone read and got an idea - I want my royalties.
These artists complaints are ridiculous, and are being made by people who don’t understand how things work.
If some other person draws a picture in their “style”, no one has to ask permission. That’s not a thing.
They either don’t understand how it works or they are just upset that a computer can make art as good as (or better than) they can in a fraction of the time.
All knowledge workers and creatives are going to face this in the future. It’s going to suck, but it would be great if we all could try to understand reality first.
I think that's the point of this blog post: it doesn't matter if the inputs are copyrighted, it matters if the output is infringing. It appears to be almost impossible to directly recreate a source image with SD, but it seems Copilot tends to produce a single input as its output, verbatim. Copilot isn't doing "synthesis" as does SD, it's acting more like a search engine.
They were prompted with the text "Mona Lisa Smile". Would you not say that they are an extremely close reproduction of the Mona Lisa, with barely any kind of synthesis?
I can virtually promise you that, if the Mona Lisa were still copyrighted, and you were to try to sell art that you painted that looked like this, the Da Vinci estate would quickly shut you down.
Plenty of cases make this not so certain: Warhol's Prince photo transofrmation (court ruled making a photographers image into clearly Warhol style was transformative enough), Blanch v Koons, Cariou vs Prince (copied photos, minor changes). If you dig through copyright cases on art, these many well be transformative enough. Plenty of other quite similar art has been ruled not infriging.
And one could also try a parody angle - make enough of these of famous art and find some angle about mocking or parodying that art, and again it may well pass copyright muster.
A court could simply rule that these images are clearly not the Mona Lisa, and, if taken as a style, could be ruled transformative, just like the above cases.
The fact is these are transformative, with a different style than the original.
Do you want to live in a future where artists don't make original art, musicians don't make music, book writers don't write, and so on, all because AI companies can replicate 1000 different copies in their style or merely remix it for marginally $0 cost, washed of all original copyright?
> All knowledge workers and creatives are going to face this in the future. It’s going to suck
This is not a given. It's up to us and the copyright law. Real original work should be compensated appropriately unless you're proposing that we accelerate deployment of universal basic income and completely abolish copyright law.
I have a feeling you might not like the violent outcome if you effectively strip original creators of their copyright, give corporations the right to effectively generate infinite profit off the backs of their work and tell the creators (and other people whose jobs will be automated away) to pound sand when they ask how they're supposed to pay rent from now on.
Do you want to live in a future where anything 'original' an artist creates has now blocked anyone else on the planet for the next 99 years and you must pay them royalties? Because we already have Disney now and they suck quite a bit.
I honestly want to live in a world where this 'worrying about paying for rent' is not a problem that we're concerned with, and a world with AI that can create and make we far more apt to achieve that than with the status quo we've been following so far.
> Do you want to live in a future where anything 'original' an artist creates has now blocked anyone else on the planet for the next 99 years and you must pay them royalties?
No I don't want that but there's a very clear distinction between people who are truly inspired by each others' work and produce an average human output that maybe covers their bills and AI that can pump out billions of replicates per day to drown out all original work, with entirety of that value captured by some corporation.
> I honestly want to live in a world where this 'worrying about paying for rent' is not a problem that we're concerned with, and a world with AI that can create and make we far more apt to achieve that than with the status quo we've been following so far.
Hey, me too. But those human issues should be addressed first, not after automation is allowed to wipe out people's livelihoods. I'm not anti-AI, I'm anti-big tech that seeks to exploit billions of people's original work for their own benefit.
> a future where artists don't make original art, musicians don't make music, book writers don't write, and so on, all because AI companies can replicate 1000 different copies in their style
This argument is assuming its own conclusion, that such a situation must be bad. But I don't think that's necessarily true.
If somebody can make 1000 different derivatives that the public likes as much as the originals, then it must be that in whatever criteria the public is interested in, these works are just as good. If they were inferior, the public wouldn't accept them as substitutes. The fact that they are (hypothetically) accepted indicates that the public is OK with them.
For my own personal aesthetics, I would like to think that today's popular music, which is written by some combination of algorithm and committee, and produced through tools that correct the performance via autotune, quantization, etc., is inferior to the music that I enjoy. But given that the public seems to like this music (and indeed, they like music generated this way even though it's not even cheaper for them to consume) seems to say that we as a society are getting what they want, and who am I to put a normative judgment on that?
This reduces the motivation to create art to a monetary one, and the value people derive from art to a purely aesthetic one.
In our future AI-infested world, I'll personally seek out "certified non-AI" creators because part of what inspires me is not just the content, but the creation of it.
I think when most people consume art, they're really treating it as entertainment regardless of its artistic merit.
And as I mentioned above, I think the current music industry is already there: the vast majority (by sales) of music entertainment, produced by algorithm and by a committee in order to drive sales. Despite this, in the genres I care about, at least, the volume of high-artistic-merit music (by variety) that is probably at its highest point in decades, if not ever. To be sure, this has meant that fewer artists are able to make a living purely off their music. But this is a return to the norm: the rise of the "star" in the late 20th century has been an aberration.
On the other side of the coin, AI assistance will be (I expect) a huge democratizing force. Recently we've seen computational photography enabling people to take photos of astounding quality with just their phones. And the results of machine learning is allowing artists to make huge improvements in post production as well.
I imagine that the stuff we've been seeing over the past year, with ChatGPT, Stable Diffusion, and such technologies, will be purposed towards (among other things) tools that enable greater productivity for the serious artist, and putting the means in the hands of those who would be otherwise unable to get to table stakes. I've started working on a short story myself, using ChatGPT to help me work through some plot points.
So yeah, we'll get a lot of meritless dreck suitable only for base entertainment. But we'll also see a proliferation of art and of artists, as productivity increases and new entrants are enabled.
The technology is coming one way or another. You can stop Stability AI but you can't stop OpenAI (Microsoft) or Google, who can afford to license training data from companies like Shutterstock. A restrictive interpretation of copyright law will just keep it in the hands of the biggest corporations.
You mean the artists who use similar brushes in photoshop and don’t know how to paint and musicians who use logic, auto tune, loop samples and don’t know how to play an instrument?
Copyright what? Someone’s brain? You can copyright a specific work or a character, but you actually want to live in a world where someone can copyright the color red with a dark black line, or the G# chord?
Real artists are going to art, and musicians are going to make music. People who do creative work do it to express themselves, their point of view or to say something.
Corporate art exist to sell you soda - I am not sure your argument lands quite like you want it to.
I am glad this is going to court, because, with my understand of how neural nets work, I fail to see how any copyright is being infringed.
This is the same stupid argument that Mp3 will destroy music instead of embracing the new marketing opportunities it represents.
IMO an artist that wants their name out of the dataset is a moron. In the end , people copying an artist style over and over will just send the price of originals through the roof. This is completely obvious.
Just like when Napster resulted in musicians becoming super rich by selling their originals to people who found their music for free? Those things don't happen in real life.
> Do you want to live in a future where artists don't make original art, musicians don't make music, book writers don't write, and so on, all because AI companies can replicate 1000 different copies in their style or merely remix it for marginally $0 cost, washed of all original copyright?
Yes, much in the same way that I am glad I live in a future where scribes aren't required to put text on paper: There is a massive amount of efficiency to be gained and enjoyment to reap for everyone who doesn't happen to be employed as an artist.
I'm not against efficiency improvements, but the value created by these improvements has to flow back towards the society at large in one way or another. I'm not anti-AI, I'm just arguing that artists and other creative professionals should be compensated for their work before their work is included in a for-profit ML model. That's hardly radical.
Current proposals don't have any intention of addressing that, they just silently kick the can down the road. What happens when nearly everything is automated there are no new profitable jobs that people can take on?
The comparison to scribes is a perfect analogy. The 'scribing' of translating the idea of painting to an actual painting is being made more efficient. The actual creativity is what the original idea is, not the skill to put it on paper.
The claim that the real value in art is in the idea rather than the execution seems to be as true as the claim that the real value in a startup is in the idea. Artists are constantly in dialogue with their tools to produce and define their idea, and to discover new ideas and capabilities along the way, in the same way the Intel head Andy Grove advised technologists to be involved in the engineering & production processes.
The value will flow to society via the cheap books (art). Value will also flow to authors (artists)[0] due to the facilitation of creation/distribution/replication. Where that value will come from is from the scribes (painters, sculptors, etc.)[1] whose contribution is rote.
[0]The intentional distinction here is that the value is in the conception of art, not in the execution of it in media.
[1]Imagine how much more productive Mozart or Beethoven could have been had an AI-powered orchestra existed in their time.
> Do you want to live in a future where artists don't make original art, musicians don't make music, book writers don't write, and so on, all because AI companies can replicate 1000 different copies in their style or merely remix it for marginally $0 cost, washed of all original copyright?
My creative output per minute has probably increased threefold in the last few months from incorporating these tools into my workflow. What I've been making doesn't look or sound like anything that anyone else is making.
You're going to have a really hard time using Stable Diffusion to make quirky cartoon daily desk calendars in the style of plaintiff Sarah Anderson. You're going to have a much better time if you think more like a Creative Director and have less of an idea ahead of time of what the tool is going to give you... so you can iterate, much like a painter iterates while working.
These tools require the creative agency from the artist who is using them in order to produce things that people find interesting, entertaining, valuable or otherwise meaningful so I really don't see a "corporation goes brrrrrr" doing anything other than flooding the lowest-common denominator content feed pipes on the internet contrasted with the highest quality art using these tools in incredibly transformative ways.
Sounds like an opt-out dark pattern. US law is unbelievably aggressive when it comes to issues of copyright, and makes copyright itself opt-out, i.e. everything you produce is copyrighted, and you have to license it in order to remove that automatic copyright. But when it comes to building these models to reproduce imitations of other people's work, suddenly copyright gets loosey-goosey.
Notice that it's a legal posture that implicitly condemns Copilot, which ignores explicitly formulated opt-outs in the form of licensing.
> This reads like damage control to me in an effort to temporarily silence the loudest critics.
I think it is to avoid any common law wrongs related to the publicity rights of the defendants. It seems like something that a legal team would flag as an unnecessary risk for the product. Removing their names and images from the training data doesn't impact the usefulness of the model while at the same time creating a much smaller surface area for collecting subpoenas.
They don't have to remove images from the training set, they're saying they're opting to do so, and using that as an argument as to why if there supposedly could be copyright infringement, they're not liable, because they allow it to be removed.
They could just as well not do anything and continue on - it's likely this case will be in defendents favor. Same as how Google can crawl the net, cache data, transform it, etc.
It's not foolish at all. There's no value prop for an artist to advertise their work in the model as 1 of 400+ million. SD doesn't tell you anything about the art that inspired the output, no one will ever know the artists work was ever used, so this 'exposure' is as good as $0.
this isn’t quite true. i was playing around with sd, and whilst browsing styles on a site where different artists’ names were compared using the same prompt (“[artist name] painting of flowers”, for example), a particular artist’s style stood out to me. i ended up buying a print from them.
sd might not directly include this info in its outputs, but it really is free marketing for some (in fact, i suspect a lot of artists are probably flattered for being included by name in the dataset, but some are riding the waxing wave of outrage for the free media exposure)
How can this possibly be a valid good faith argument? Either they're in breach of authors' copyright which extends to every piece of art that they included in the dataset without permission, or they're in the clear and aren't obligated to respond to removal requests.
This reads like damage control to me in an effort to temporarily silence the loudest critics.