Why don't indie devs use AI-generated images as art?

[Davaris]

I was just thinking about the technology that is becoming obsolete. Game engines, 3D editors, 2D editors, IDEs, programming languages. Billions of dollars of R&D up in smoke.

I think you're way off on a temporal frame. Maybe like two decades from now something like what you're talking about might exist, but so far I can only render 1FPS at 512x512 on a 3080TI with 12GB VRAM under the best of circumstances.

Open Sourcing this instead of trying to lock it up on some proprietary website and filtering prompts was definitely a big move though. You can already make Videos or short Animations with something that's called "Deforum" and similar interpolation Tools: https://replicate.com/deforum/deforum_stable_diffusion

What it seems to do is render a frame, then shift the image and render the next. But so far it lacks consistency between frames and obviously any kind of active user input that would allow a player to directly interact in a "game world", someone else was working on a Blender Plugin so you can generate a 2D image from a 3D space:

Yeah I don't think its going to happen in a year, but in 10 years a lot happens in tech. It may not look as good as today's 3D engines, but it will be so easy to work with compared to 3D engines, no one will want to use them.

I saw a very good AI 3D animation today and the impression I got was it looked like a dream. There is a fellow who was heading up Tesla's AI department who just quit, he says similar about his AI when it is rendering.

 

[V17]

Apparently the result of: "fallout 5 tarkov stalker 2, canon50 first person movie still, ray tracing, 4k octane render, hyperrealistic, extremely detailed, epic dramatic cinematic lighting;
width:768 height:448 steps:50 cfg_scale:10 sampler:k_euler_a"

Spoiler

Look out Unreal Engine 5, AI is coming for you.

I was just thinking about the technology that is becoming obsolete. Game engines, 3D editors, 2D editors, IDEs, programming languages. Billions of dollars of R&D up in smoke. I also wonder will it make some/all math obsolete? AI doesn't need equations to get things done, since it works similar to our brains.

For the person who thinks I'm way out. I think algorithmic engines and related tools will become obsolete, IF AI rendering speed reaches a high frame rate. John Carmack just started a company to develop AGI, so massive performance increases in these programs could become available in the future. I have also seen AI make 3D animations.

Why do I say no math? Because while these AI are made using math algorithms, they do not generate these pictures using math . Math and algorithms are used to make non-thinking machines perform tasks. Photoshop 3D Studio Max, Unreal uses math and algorithms to make pictures. But an artist making an oil painting does not use math. He just does it. AI has no need for math and algorithms to perform a task, since it is modeled on the human brain. Like us, after training on a skill, it just does it. For example, its not doing calculations using triangles, meshes, matrix transformations, and light equations, like a game engine when it draws a picture. It just does it, like a human painter would.

I don't think this will happen as much as you say, especially with abandoning math, because there are a lot of technical fields where you need results that have known precision and can be provably correct. In fact I'd say that entertainment & art where this is not necessary are the exceptions in this case.

Neural networks are a blackbox, by definition, you have no idea how exactly the computations are happening, and this will only get worse as they get more complicated.

And secondly we are probably going to get specialized neural net accelerator chips, because using GPUs is incredibly expensive and inefficient and there's a demand to use something cheaper that's fast and can be used not only on home PCs but also in portable devices. And so far one of the most interesting directions these accelerator chips are going is reducing precision, sometimes by using analog matrix multiplication - a crazy but functional low-power idea. This is not exactly vaporware either, prototypes already exist. And it would only make the provable correctness problem worse.

 

[Davaris]

Copied some prompts from the MidJourney showcase section into Dezgo.com:

 

[Davaris]

Apparently the result of: "fallout 5 tarkov stalker 2, canon50 first person movie still, ray tracing, 4k octane render, hyperrealistic, extremely detailed, epic dramatic cinematic lighting;
width:768 height:448 steps:50 cfg_scale:10 sampler:k_euler_a"

Spoiler

Look out Unreal Engine 5, AI is coming for you.

I was just thinking about the technology that is becoming obsolete. Game engines, 3D editors, 2D editors, IDEs, programming languages. Billions of dollars of R&D up in smoke. I also wonder will it make some/all math obsolete? AI doesn't need equations to get things done, since it works similar to our brains.

For the person who thinks I'm way out. I think algorithmic engines and related tools will become obsolete, IF AI rendering speed reaches a high frame rate. John Carmack just started a company to develop AGI, so massive performance increases in these programs could become available in the future. I have also seen AI make 3D animations.

Why do I say no math? Because while these AI are made using math algorithms, they do not generate these pictures using math . Math and algorithms are used to make non-thinking machines perform tasks. Photoshop 3D Studio Max, Unreal uses math and algorithms to make pictures. But an artist making an oil painting does not use math. He just does it. AI has no need for math and algorithms to perform a task, since it is modeled on the human brain. Like us, after training on a skill, it just does it. For example, its not doing calculations using triangles, meshes, matrix transformations, and light equations, like a game engine when it draws a picture. It just does it, like a human painter would.

I don't think this will happen as much as you say, especially with abandoning math, because there are a lot of technical fields where you need results that have known precision and can be provably correct. In fact I'd say that entertainment & art where this is not necessary are the exceptions in this case.

Neural networks are a blackbox, by definition, you have no idea how exactly the computations are happening, and this will only get worse as they get more complicated.

And secondly we are probably going to get specialized neural net accelerator chips, because using GPUs is incredibly expensive and inefficient and there's a demand to use something cheaper that's fast and can be used not only on home PCs but also in portable devices. And so far one of the most interesting directions these accelerator chips are going is reducing precision, sometimes by using analog matrix multiplication - a crazy but functional low-power idea. This is not exactly vaporware either, prototypes already exist. And it would only make the provable correctness problem worse.

I watched a video where a scientist was talking about that. He said classical computers can't do AI efficiently because the CPU and memory are separate. He said what he is going to do is make each neuron have its own memory.

 

[CryptRat]

 

[Üstad]

Copied some prompts from the MidJourney showcase section into Dezgo.com:

Link redirected me to dubious looking golf cart site.

 

[Davaris]

Copied some prompts from the MidJourney showcase section into Dezgo.com:

Link redirected me to dubious looking golf cart site.

Thanks. Fixed.

 

[J1M]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I would guess the line art plus distinct colors would pose similar challenges as fingers.

 

[Davaris]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I would guess the line art plus distinct colors would pose similar challenges as fingers.

They had a nice B/W line art one on MidJourney yesterday. And the hands on their pictures look good, but they may have been touched up.

https://web.archive.org/web/20220908013809/https://www.midjourney.com/showcase/

MidJourney looks far better than Dezgo for some reason. Do they have access to higher quality art, or is it something else?

 

[Üstad]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I would guess the line art plus distinct colors would pose similar challenges as fingers.

They had a nice B/W line art one on MidJourney yesterday. And the hands on their pictures look good, but they may have been touched up.

https://web.archive.org/web/20220908013809/https://www.midjourney.com/showcase/

MidJourney looks far better than Dezgo for some reason. Do they have access to higher quality art, or is it something else?

How can I download images from MidJourney Showcase? I think we used to could download but for some reason I can't do it now.

 

[Davaris]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I would guess the line art plus distinct colors would pose similar challenges as fingers.

They had a nice B/W line art one on MidJourney yesterday. And the hands on their pictures look good, but they may have been touched up.

https://web.archive.org/web/20220908013809/https://www.midjourney.com/showcase/

MidJourney looks far better than Dezgo for some reason. Do they have access to higher quality art, or is it something else?

How can I download images from MidJourney Showcase? I think we used to could download but for some reason I can't do it now.

Its amusing they built the database for their AI to train on by copying artists' work from the internet, but now they are trying to make it difficult to copy from them.

If you have Firefox:
Click on the padlock icon next to the web address in your browser.
In the menu that opens up, click on the padlock with 'Connection secure'
In the next menu click on 'More information'
Then click on 'Media' and you will see a list of files you can save.
The images are of type webp, so you will need to convert them if you want to post them here. I don't know what program does that.

If you have Chromium. I don't know. It might be a similar process.

I have seen a program that downloads everything from webpages and websites, but I can't remember what it is called.

 

[Dexter]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I don't think it knows exactly what a Spawn is or that it has a lot of Todd McFarlane reference data/art to work off of. It definitely knows what Spider-Man is, might be a bit confused by its inhuman athletic abilities though. Also some examples for Superman. Could probably get something more workable playing around and refining prompts, but I can't be bothered. Also note that if you make the Output larger than the initial Training data of 512x512 you sometimes get the subject rendered double for portraits/objects or weird stuff might happen with torso or limbs:

Spoiler

I think in general it hasn't been trained on that much comics data as opposed to more generalized stuff and would probably benefit from a model specifically trained on a large amount of that like they're doing with Animu, if you wanted more stable and workable immediate results: https://huggingface.co/hakurei/waifu-diffusion https://github.com/harubaru/waifu-diffusion

Also stumbled upon these by Silent Hill artist:

https://automaton-media.com/en/nongaming-news/20220831-15350/

 

[Non-Edgy Gamer]

As with all professions, high skill jobs are probably fine. It's the low-skill, low-effort jobs that are in danger.

Of course, the more advanced the AIs get, the more jobs they will replace.

 

[Viata]

And that is why you should never make your passion into your job, because when you are replaced(and as any cog will be replaced), you can't deal with reality anymore.

 

[J1M]

How does it handle comic art?

Say Spiderman or Todd McFarlane circa 2000?

I don't think it knows exactly what a Spawn is or that it has a lot of Todd McFarlane reference data/art to work off of. It definitely knows what Spider-Man is, might be a bit confused by its inhuman athletic abilities though. Also some examples for Superman. Could probably get something more workable playing around and refining prompts, but I can't be bothered. Also note that if you make the Output larger than the initial Training data of 512x512 you sometimes get the subject rendered double for portraits/objects or weird stuff might happen with torso or limbs:

Spoiler

I think in general it hasn't been trained on that much comics data as opposed to more generalized stuff and would probably benefit from a model specifically trained on a large amount of that like they're doing with Animu, if you wanted more stable and workable immediate results: https://huggingface.co/hakurei/waifu-diffusion https://github.com/harubaru/waifu-diffusion

Also stumbled upon these by Silent Hill artist:

https://automaton-media.com/en/nongaming-news/20220831-15350/

So one of the only forms of art that AI can't properly imitate yet was destroyed by the politics of the artists moments before they needed those jobs. Ironic.

 

[Twiglard]

Anything AI-powered that can generate basic ground or wall textures? It could be actually useful for indie purposes unlike purely procedural textures. Basically something that could be laid down onto a flat quad.

 

[Dexter]

Anything AI-powered that can generate basic ground or wall textures? It could be actually useful for indie purposes unlike purely procedural textures. Basically something that could be laid down onto a flat quad.

Pytorch can apparently generate tiling textures, see here: https://www.reddit.com/r/StableDiff...ative_seamless_tile_generation_no_inpainting/
https://github.com/sd-webui/stable-diffusion-webui/pull/911

Should already be in some of the WebUI implementations, like this one that seems to get Updated with some of the newest Features and stuff a lot: https://rentry.org/voldy
https://github.com/AUTOMATIC1111/stable-diffusion-webui-feature-showcase
Just tell it what to generate, examples:

Spoiler

Hopefully this next ^^

 

[mkultra]

Anything AI-powered that can generate basic ground or wall textures? It could be actually useful for indie purposes unlike purely procedural textures. Basically something that could be laid down onto a flat quad.

Yes. I can create it in now in Stable Diffusion, we got a checkbox for it just days ago in Visions of Chaos..
Filter forge is awesome though, it can create them in 8K or whatever you need, seamless. Can even change lighting too, like if you want it come from a certain side, or straigt from above. Been using it for over 10 years. I have some old version, 7 if i recall correctly. It's up to 12 or whatever now i think.

any texture is possible to make, brain substance, blood splatter, any type of ground, any metals (rusty metal sheets, whatever), skin (human, snake skin), furs, skies, space etc etc

 

[Viata]

Using AI to make textures seems like an amazing idea.

 

[mkultra]

Maybe. Deforum has a fork now which can do pretty hi-res images from the looks of it. 512px is just too low, and upscaling with AI is still upscaling, i can tell rather quickly its upscaled now because i've looked at thousands of those images now (my own and others).
Stable Diffusion seemed so fast to me (going from Disco diffusion where you often waiit 5-10+ minutes for one image), then you try Deforum.. wow.

 

[V17]

Some dude on Reddit already tried a similar thing. Blocked out a pattern of shapes, overlaid a random "scratched metal" texture from google and let the AI convert it to an actual metal scifi wall panel with variations. Then modified the colors of a result he liked and created another generation of variations from that.
I tried doing a similar thing to add dirt to a clean render (so not a texture but a final render) and it worked just as well, but you need to give the AI something to work with - the dude added the generic metal texture, I added some procedural noise. Both got transformed into something non-generic and workable.

Didn't need it to be seamlessly repeatable in the case of the scifi panel, but that's just a matter of turning the feature on.
It's also not as simple as just running a txt2img prompt, but it's still pretty interesting and probably quite useful if capable of editing and finetuning the results.
And new forks of Stable Diffusion can do 1024x1024 px on 8 GB of VRAM I believe, which is not ideal, but should be enough for some purposes.

 

[mkultra]

eh. filter forge does sci-fi panels, doors, any resolution... it has some pretty nice ones and you can generate new variations endlessly.
This is from my old SS2 mod, original first then my textures... its all filterforge apart from the text which is done in PS.. it's probably 10 years old by now. it has better looking filters now though, this isn't spectacular by any means, something i did pretty quick since most of the game apart from this door (and some other few textures) was updated with hi-res textures, looked off to have something low-res..

 

[Dexter]

512px is just too low, and upscaling with AI is still upscaling, i can tell rather quickly its upscaled now because i've looked at thousands of those images now (my own and others).

Not all "upscaling" is the same with AI though:

Original Generated Images (512x512, 512x704):

This is a simple Upscale using "Real-ESRGAN 4x", trying to keep everything the same but blowing up the image, removes noise/grain, the edges and details are kinda blurred or smudged in the process, not exactly Optimal or what we preferably want, but it keeps the image composition the same:

Spoiler

But since we're using AI Tools we have other choices, we can use img2img "SDUpscale" with imported algorithms, what Stable Diffusion does in this case is not simply enlarge the image and try to keep it coherent, but goes over the image in blocks from Top to Bottom and recreates it using the same process used to generate our init image based on parts of itself. Try to use the same prompt and Seed as the Original image if you want it as close as possible. Other than that the important parameters to play around with here are Denoising strength (how similar the image will be to your input, whereby 0 is basically the same and 1 is an entirely new image, the Sweet spot here is somewhere between 0 and 0.3 if you don't want it to change markedly while giving it some room to change a bit or add new details, in some cases literally making them up by "finding" them in the noise) and CFG (how close the algorithm will follow your prompt)

Here are some results using "4x_foolhardy_Remacris" and "Lollypop":

Spoiler

Remacris (Denoise 0, 0.15, 0.3):

Lollypop (Denoise 0, 0.15, 0.3):

 

[Üstad]

Its amusing they built the database for their AI to train on by copying artists' work from the internet, but now they are trying to make it difficult to copy from them.

If you have Firefox:
Click on the padlock icon next to the web address in your browser.
In the menu that opens up, click on the padlock with 'Connection secure'
In the next menu click on 'More information'
Then click on 'Media' and you will see a list of files you can save.
The images are of type webp, so you will need to convert them if you want to post them here. I don't know what program does that.

If you have Chromium. I don't know. It might be a similar process.

I have seen a program that downloads everything from webpages and websites, but I can't remember what it is called.

Is there a way to search for images by text for MidJourney? Stable difusion has a site with this feature but I cant find one for MidJounrey.

 

[V17]

Soon there probably won't be a need to use the tile by tile method, as that can still leave visible changes where one tile fades into each other. Every few days there's a new optimization that reduces VRAM usage, I can now generate 1664x832 px images on 6 GB VRAM. Emad from Stability AI did a Q&A on Reddit and announced that firstly the hardware demands will go down further pretty soon (they'll release the model in native 16 bit precision, among other things), and secondly they have a more hi-res version ready, that uses 1024x1024 training data natively, instead of 512x512.

Currently what works for me is to generate a smaller image (where at least one of the dimensions is 512) and then upscale it using img2img to whatever fits in my VRAM. Since the img2img method provides some guiding, the common issues with generating large images purely from noise disappear. There is no need to use a neural net for the upscale, just normal bilinear upsampling will work since the details get redrawn anyway.

This is a panoramic "photo" of some hills near my place generated with that method. Honestly looks just like the real thing, I wouldn't have known. The AI is much better at generating this than for example hills of green grass for some reason, those usually look like carpet or felt.

Spoiler

But to be honest I enjoy those weird surrealistic accidents the most anyway:

Spoiler