Digital art creator algorithm website

[Folly]

@Jack_Batty

But the bombed out on my Clouds entry:

My Clouds entry was my highest scorer so far (but still not top 20)

https://creator.nightcafe.studio/creation/IYkUeNy3Qaw5wHJi17ae

[Ponderoid]

@Folly

Next time I’m going for kittens. Lots and lots of cute, fuzzy kittens.

Response from NovelAI’s anime art generator from that prompt:

From NovelAI’s anthro furry generator:

[Folly]

Oh no! I’ve been beaten to it!

[Folly]

Woohoo!!! Top 10% for my Aliens entry. I’m going to stop spamming the thread with my ‘fails’ now.

…And start spamming it with my successes! Whooooooo!!! (just kidding)

Peace Pipe - AI Generated Artwork

[Pardel-Lux]

I like to draw and paint with Clip Studio Paint, a programm that can be bought (as opposite to Adobe’s Photoshop, tat you can’t own anymore, and the trouble with Pantone and licenses you may have heard about, but I digress… as usual). Now, after many free improvements, they have reached Version 1.12.11 after about ten years, they are about to release version 2, for which I will have to pay. I’m OK with that, they are not really expensive, and program so far is powerful and well designed. Now with Version 2 they have shortly succumbed to temptation: let’s do what is possible, not what is good! And so they thought of adding what we (well, mostly you so far) are discussing in this thread: AI image generation. They anounced that on Nov. 29.
Today, three days later, they recanted. The user community was very put off and expressed “anxiety and concern”. Getting into some detail Clip Studio Paint writes:

Here are some concerns that we have taken to heart:

Current image generation AI exploits other artists' intellectual property and is unusable
This feature will hinder rather than help artists in their creative activities
Using artist’s work that is not opted-in to a data set is ethically unacceptable
The fear that this will make Clip Studio Paint artwork synonymous with AI-generated work
There are existing features that need to be prioritized over image generation AI features
Having something unknown in the app I use daily potentially infringing on legal or moral rights is unacceptable
Clip Studio Paint should be an app that takes responsibility for a safe, creative environment

I believe this is relevant to the discussion here.
BTW: Coooool pictures you have shown so far. Which, I am afraid, is part of the problem.

[DPRK]

There is no problem. While the systems being considered in this thread are certainly capable of not-particularly-creative output (e.g., “in the style of Picasso”), the resulting output in no way violates Picasso’s intellectual property than composing a song in 12-bar blues violates W. C. Handy. Styles and ideas are not copyrightable, nor should they be, though it is not a bad idea to document your sources, like Handy did.

No idea why your paint program does not include the AI plugins that are included in other programs or separately available; could be that they were threatened by a patent troll.

[Pardel-Lux]

I am not sure I overestimate or you underestimate the magnitude of this AI innovation. This other thread may convince you that it is you who shoulld think again:

The next page in the book of AI evolution is here, powered by GPT 3.5, and I am very, nay, extremely impressed

Hi dopers. I used the word page in the topic deliberately because this thread is about an amazing text-based AI. You can try it yourself by making a free account at chat.openai.com. Here are some of my prompts. You can type normal english and make spelling mistakes and the AI will still understand you just fine! Write the first page of a gripping scifi novel in Asimov’s style. The robot stood at the edge of the cliff, gazing out at the vast expanse of the Martian landscape. It was the lates…

But perhaps you have not made your argument clear enough to me? Maybe I (and the people worried who wrote to ClipStudioPaint) are being luddites?

[DPRK]

It is possible I am underestimating or misunderstanding something, but, for the purposes of the users’ complaint, I doubt it matters if it is an AI or a human artist producing arguably derivative, yet not plagiarized, paintings (or novels) after having seen others’ work on the Internet (the “dataset”). You may rightly call me a talentless hack, but if I want to write “in the style of Cormac McCarthy” or paint (from @Folly 's prompt) like Artgerm and Greg Rutkowski, what is anyone going to do about it? As long as I am not representing it as other than it is, no forgery or plagiarism, in other words.

Have you heard of Jack Reacher? There is a series of novels featuring one John Puller…

[Pardel-Lux]

@DPRK

While the systems being considered in this thread are certainly capable of not-particularly-creative output (e.g., “in the style of Picasso”), the resulting output in no way violates Picasso’s intellectual property

I am not sure about that: the AI has taken Picasso’s pictures and has analyzed and stored them without asking for permission. It has taken many other pictures as well and digitized them. That is not the same as a human plagiarizing, being inspired by or wanting to parody Picasso (or whomever). But it is hard to explain the difference and I am sure lawyers will debate this for years to come and in the end the solution will be political, not technical (in the sense that stating that Facebook and Twitter do not have editorial responsability for the things the users post is a political, not a technical decision).
One way or another, we are not going to resolve this conundrum here. I just wanted to add a data point and express my insecurity.

@DPRK

Have you heard of Jack Reacher?

No, sorry, I have not.

[DPRK]

@Pardel-Lux

I am not sure about that: the AI has taken Picasso’s pictures and has analyzed and stored them without asking for permission. It has taken many other pictures as well and digitized them.

This is not correct, and that is easy to see without having to know what specific technique was used: the AI model (a few gigabytes) is far too small to have possibly stored the enormous amount of pictures it looked at during its training.

[Pardel-Lux]

But it has used them for training, and in classical AI fashion, not even the programers know what the algorithm is really doing: it is a black box. The pictures may not be in the program but they have made the program the way it is.
Imagine making a second copy of the algorithm, trained with exactly the same data set, except for Picasso’s pictures. Of those the algorithm gets not one. Then ask the algorithm to draw a bull in Picasso’s style. What will it do?
What is the difference between both algorithms? Something about Picasso, it seems to me. That must have some relevance for copyright issues.

[DPRK]

Absolutely; I have heard from some artists who are absolutely dismayed and angry that their data was used that way.

However, my naive understanding is that “you can’t copyright style”:

You Can’t Copyright Style — THE [LEGAL] ARTIST

The Grotesque Legacy of Music as Property

[Babale]

@Pardel-Lux

But it has used them for training, and in classical AI fashion, not even the programers know what the algorithm is really doing: it is a black box. The pictures may not be in the program but they have made the program the way it is.
Imagine making a second copy of the algorithm, trained with exactly the same data set, except for Picasso’s pictures. Of those the algorithm gets not one. Then ask the algorithm to draw a bull in Picasso’s style. What will it do?
What is the difference between both algorithms? Something about Picasso, it seems to me. That must have some relevance for copyright issues

By this logic an art student who studied Picasso paintings is also violating copyright.

The art student has used them for training, and in classical human conciousness fashion, not even the parents or teachers know what the student is really thinking: their mind is a black box.
The paintings in the textbook may not be in front of the student while they paint, but they have made the student the way they are.

Imagine a second student, taught from exactly the same art curriculum but with all references to Picasso removed. Then ask the student to draw a bull in Picaso’s style. What will they do?

What is the difference between the two students? Knowledge of and influence by Picasso, it seems to me.
That must have some relevance for copyright issues.

[Pardel-Lux]

Yes, I thought we would veer into questions related to vitalism and the difference between the human mind and a computer. I don’t know the answer, if there is one at all. For me a mechanical device is doing something cualitatively different from a human, but the argument of the black box undermines that difference somehow. The only prediction I dare make is that this issue will make many lawyers rich and some creators poor and angry.

[Babale]

@Pardel-Lux

Yes, I thought we would veer into questions related to vitalism and the difference between the human mind and a computer. I don’t know the answer, if there is one at all. For me a mechanical device is doing something cualitatively different from a human, but the argument of the black box undermines that difference somehow.

Yeah. The more advanced AI gets at replicating specific human functions, and the more we learn about the inner workings of the human brain, the muddier the waters become.

[Dr.Strangelove]

@Babale

the muddier the waters become

Not just in the sense of “can machines be creative?”, but also “what is creativity, anyway?” It might not prove to be such a special gift, after all.

[Chronos]

@DPRK

This is not correct, and that is easy to see without having to know what specific technique was used: the AI model (a few gigabytes) is far too small to have possibly stored the enormous amount of pictures it looked at during its training.

But it’s still possible that some of the input works, that the AI considered especially representative of a set of objects, ended up stored in the database more or less in their entirety, while other input works sufficiently similar to those were mostly or entirely discarded. If, for instance, the training set included a thousand photographs of a piece of broccoli, and the program essentially kept one or two of them in a compressed form and discarded the rest, then whenever you asked it for a photorealistic picture of broccoli, you’d get that same single piece showing up repeatedly. Which, from what I’ve seen so far, appears to be what happens.

[Babale]

@Chronos

If, for instance, the training set included a thousand photographs of a piece of broccoli, and the program essentially kept one or two of them in a compressed form and discarded the rest, then whenever you asked it for a photorealistic picture of broccoli, you’d get that same single piece showing up repeatedly.

I don’t believe that’s how these programs work. They don’t keep full images as part of the model in any way. Rather, the AI was shown a bunch of pictures of broccoli, and told “this is broccoli”, and it came up with some list of the properties of what a picture of “broccoli” is. And when you ask it for broccoli, it creates an image that scores highly in all of these “broccoli” properties.

If it always displays a very similar image, it is because that image, in the eyes of the AI, checks off all of the properties of “broccoli”.

[Chronos]

But that’s also how image compression works. If I show you a JPEG photograph of your mother, and you say “That’s my mother!”, and I were to then reply “No it’s not; it’s just an image that has similar Fourier phases and amplitudes as an image of your mother”, then I would simply be wrong. It is an image of your mother, and the Fourier business is just a way of storing that image (or at least, the important parts of it) efficiently.

[Sam_Stone]

I don’t think that is correct. There’s no image storing at all, and no ability to recreate a specific image.

Obviously we don’t know exactly what’s going on in these models, but neural net processing is more about coming up with tests for fitness and then tweaking the network until it optimizes. So it looks at a million pictures of broccoli then you tell it to draw broccoli. The neural net produces random glurge, then tests it for ‘broccoliness’. Test fails, so it tweaks its parameters and generates an image. This time there’s a bit of broccoliness, so it changes its parameters again and tries again. It keeps doing this until its image maximizes the tests that were set up by the output of the natural language interface.

I asked GPT3.5 “Explain how diffusion models are used by AI to make pictures.” This is what it said:

Diffusion models are mathematical models that are used in AI to generate images or other visual outputs. These models work by simulating the random motion of particles, or “agents,” within a system. These particles move around within the system and interact with each other according to a set of rules or probabilities, which can be adjusted to produce different visual effects.

For example, a diffusion model could be used to generate an image of a tree. The model would start by defining a set of particles, each representing a point on the tree’s surface. These particles would then be given rules for how they should move and interact with each other, such as rules for how they should respond to changes in temperature, light, or other environmental factors.

As the model runs, the particles will move around and interact according to these rules, creating complex patterns and structures that mimic the growth and behavior of a real tree. By adjusting the rules and probabilities of the model, the AI can produce a wide range of different tree shapes and appearances, creating a rich and detailed image.

Overall, diffusion models are a powerful tool in AI that can be used to generate a wide range of visual outputs, from realistic images of natural phenomena to abstract patterns and designs. These models are particularly useful for generating complex, organic-looking images that would be difficult to create using other methods.

Prompt: Do diffusion models store copies of the images they use for training?

Diffusion models are mathematical models that simulate the random motion of particles within a system. These models do not store images, but rather they use algorithms and rules to generate images based on the input and parameters provided to them.

In the context of AI, diffusion models are often used to generate images that mimic natural phenomena, such as trees or clouds. To create these images, the model may be trained on a dataset of real-world images that show these phenomena. The model uses this training data to learn the patterns and characteristics of the phenomena, and it applies this knowledge to generate new images that look similar to the ones in the training data.

However, the model itself does not store these training images. Instead, it uses the information it has learned from the training data to create new images on the fly, without needing to store the images themselves. This allows the model to generate a wide range of images, even if the training data only includes a small number of examples.

Overall, diffusion models are a type of AI algorithm that can be used to generate complex, organic-looking images without needing to store the images themselves. These models are useful for a wide range of applications, from generating realistic simulations of natural phenomena to creating abstract art.

However, it does agree that copyright violation may be taking place:

Prompt: Does training an AI with other artist’s images violate their copyright?

Training an AI with images created by artists could potentially violate the artists’ copyrights if the images are used without the artists’ permission. Copyright law gives the creators of original works, such as art, music, and literature, the exclusive right to control how their works are used and distributed. This means that if you want to use an artist’s work in your AI training dataset, you would need to obtain the artist’s permission first.

If you use an artist’s work without permission, you could be infringing on their copyright and could be liable for damages. In some cases, you may be able to use an artist’s work under the fair use doctrine, which allows for limited use of copyrighted material without permission for certain purposes, such as criticism, commentary, news reporting, teaching, scholarship, or research. However, whether a particular use of an artist’s work would qualify as fair use would depend on the specific circumstances and would need to be evaluated on a case-by-case basis.

Overall, it is important to respect the copyrights of artists and other creators when using their works in AI training datasets. If you are unsure whether a particular use of an artist’s work would be considered fair use, it is best to consult with a lawyer or seek permission from the artist directly.

The right being violated isn’t due to a strict copying issue, but rather a violation of the artist’s moral rights, which allows them to restrict how their copywritten images are used. So it’s not necessarily true that an image ‘in the style of Rutkowski’ is a violation of copyright, but the use of Rutkowski’s images for training an AI may be a cooywrite violation, even if no output remotely similar is generated.

But it’s a gray area. I would guess that using those images for research or scholarship,is ‘fair use’, but opening that data up commercially could be a copyright violation.