I was tinkering around with ESRGAN and SFTGAN this morning and found a workflow that produces consistently
decentresults.
My new method involves appying a 1 pixel guassian blur to the vanilla texture then feeding it into ESRGAN, then applying a sharpen filter to the result, then upscaling that with SFTGAN. The results look a little better to my eye:
Then I got a bit curious what the finals results might look like in-game with normal, occlusion, and emission maps applied, so I manually created some to slap on there:
I still don't consider the results stunning, but it gets acceptable results without much work. I could probably churn out some good textures in little time.
The ESRGAN seems to put out a LOT of noise on these very small textures and I've been paying around with the interpolation to see how it reacts. Right now I'm using interpolation value of alpha = 0.1 to get a smoother output before feeding its result into the SFTGAN and it seems to work okay. The first images I posted to this thread using ESRGAN was using alpha = 0.8 and you can see how it added a lot of strange details into the texture. It was interesting, but not all that great. Blurring the vanilla texture by 1 pixel seemed to help a little in preventing ESRGAN from injecting more noise.
I think what would help this out the most would be for us to train the AI on a large set of HD textures so that it is trained for this specific task (I think the included models were trained on very general pictures). Does anyone happen to have a large set of HD textures (maybe 500 or more?) that we could run the training algorithm on? It would also help if said person has a Geforce 1080TI or two...
Looks promissing! Id like to see some enemy sprites, or npcs, or tree sprites...I will try this method on some NPCs and vegitation when I get home this evening, but I suspect it won't work very well... but I hope I'm wrong!