Bringing D&D/AD&D campaign settings to life with Stable Diffusion

[Zed Duke of Banville]

I've been messing around with the Stable Diffusion graphics-generating software, and among other things I experimented with creating portraits of characters from the D&D Known World / Mystara gazetteers from their written descriptions. For GAZ1: The Grand Duchy of Karameikos, I first created portraits based on the artwork of William-Adolphe Bougeureau and a few similar painters, but then also created alternative portraits using the Mucha/Rutkowski/Artgerm combination that seems to be popular:

Duke Stefan Karameikos:

Duchess Olivia Karameikos:

Adriana Karameikos:

Also from the same batch of five images as the latter portrait:

Alya Flameflicker:

Anton Radu:

Aleena Halaran:

Bargle the Infamous:

 

[rusty_shackleford]

damn she's a giraffe

 

[Zed Duke of Banville]

For GAZ2 The Emirates of Ylaruam, I used a few Orientalist artists, plus Orientalist art more generally, as prompts. This gazetteer did not include a section on notable characters, and few were mentioned in the gazetteer, so I just tried for a handful of setting-appropriate images:

Warrior (supposed to be wearing a hood and wielding a scimitar):

Belly dancer:

The Prophet Al-Kalim:

Female warrior in gray robes:

Female warrior without mentioning clothing:

Camel traders:

Intended as a merchant:

 

[Popiel]

Fuck's sake, the rate at which this is accelerating most of the digital artists/concept artists/so on will be begging on the streets in like, what, 2 to 3 years.

 

[Catacombs]

Warrior (supposed to be wearing a hood and wielding a scimitar):

Intended as a merchant:

The results still look cool.

 

[NecroLord]

Is it just me or does Olivia Karameikos almost look like Yennefer in her second portrait?

 

[FriendlyMerchant]

I've been messing around with the Stable Diffusion graphics-generating software, and among other things I experimented with creating portraits of characters from the D&D Known World / Mystara gazetteers from their written descriptions. For GAZ1: The Grand Duchy of Karameikos, I first created portraits based on the artwork of William-Adolphe Bougeureau and a few similar painters, but then also created alternative portraits using the Mucha/Rutkowski/Artgerm combination that seems to be popular:

Duke Stefan Karameikos:

Duchess Olivia Karameikos:

Adriana Karameikos:

Also from the same batch of five images as the latter portrait:

Alya Flameflicker:

Anton Radu:

Aleena Halaran:

Bargle the Infamous:

Left = Soul
Right = Soulless

 

[Zed Duke of Banville]

For GAZ3: The Principalities of Glantri, since Glantri City is inspired by Venice, I decided to use the style of several Venetian artists from the early-modern era (though not tightly bound in time or style). This gazetteer contains a section detailing the various families ruling the principalities, but I confined myself to the leaders:

Volospin Aendyr:

Etienne d'Ambreville:

Carnelia de Belcadiz

Jaggar von Drachenfels

Carlotina of Erewan

Morphail Gorevitch-Woszlany (vampire)

Innocenti di Malapietra

Brannart McGregor (lich)

Jherek Virayana (Ethengarian, hence Mongolian in appearance)

Vanserie Vlaardoen

 

[Old One]

Still having a little trouble with eyes, I see.

 

[orcinator]

For GAZ2 The Emirates of Ylaruam, I used a few Orientalist artists, plus Orientalist art more generally, as prompts. This gazetteer did not include a section on notable characters, and few were mentioned in the gazetteer, so I just tried for a handful of setting-appropriate images:

Warrior (supposed to be wearing a hood and wielding a scimitar):

Belly dancer:

The Prophet Al-Kalim:

Female warrior in gray robes:

Female warrior without mentioning clothing:

Camel traders:

Intended as a merchant:

Tits! Thought those would get censored like those monster with a "drawn by he giger" description I've been making, unless that's something they implemented recently.
What are the actual prompts for each one? I've been having trouble getting anything good out of the robots. Also do you use something specific for upscaling(which I need for background images)?

 

[Zed Duke of Banville]

Tits! Thought those would get censored like those monster with a "drawn by he giger" description I've been making, unless that's something they implemented recently.
What are the actual prompts for each one? I've been having trouble getting anything good out of the robots. Also do you use something specific for upscaling(which I need for background images)?

The NSFW filter seems dysfunctional, as it was blocking a number of images that had no reason to be NSFW while allowing some nudity through, but it wasn't too long before I edited out the filter.

Some examples for prompts:
Duke Stefan Karameikos, monarch, royalty, fiery red hair flecked with gray, fiery red beard flecked with gray, sea-blue eyes, gruff, candid, uncompromising, William-Adolphe Bouguereau, Edmund Leighton, Frank Dicksee, John William Waterhouse
Duchess Olivia Karameikos, a slender and graceful woman, middle-aged, dark brown hair, icy blue eyes, patrician beauty, royalty, perfect manners, unshakeable composure, art by Artgerm and Greg Rutkowski and Alphonse Mucha
warrior, scimitar, from the Emirates of Ylaruam, swarthy, black hood, red pants, D&D, fantasy, artstation, Jean-Léon Gérôme, Eugene Delacroix, Edwin Lord Weeks, orientalism, orientalist
merchant, from the Emirates of Ylaruam, middle-aged, thin black beard, camels, Robert George Talbot Kelly, John Frederick Lewis, Edwin Lord Weeks, orientalism, orientalist
Prince Innocenti di Malapietra, wizard, curly hair, goatee, fat, oily, paranoid, ruthless, Italian, fantasy, D&D, artstation, Titian, Veronese, Tintoretto, Tiepolo, Canaletto, Bellotto


GAZ4 The Kingdom of Ierendi is similar to GAZ2 in lacking a section on notable personages, but unlike Ylaruam there isn't much of an identity for Ierendi, so I just created three portraits based on Artgerm/Rutkowski/Mucha:

King Palfrit:

Queen Marianne:

Wert, a boy who collects seaweed:

Since these were the only three portraits I created for GAZ4, here are another six I created for Leranda from GAZ3, "a young lady steps into the room, a good-looking brunette with large blue eyes and a form-fitting robe":

 

[Heroic Liberator]

Amazing.

 

[Pocgels]

the eyes still aren't right

 

[Non-Edgy Gamer]

Intended as a merchant:

You should have added "profile, rubbing hands" to the prompt.

There you go.

 

[Non-Edgy Gamer]

the eyes still aren't right

You can correct them somewhat with masking (difficult) or with GFPGAN.

Before:

After:

It doesn't work on every art style, and there are tradeoffs, like smoothing the face.

You can also combine GPFGAN with Photoshop by doing an ok job at editing the eyes, and then having GPFGAN normalize it.

Stable Diffusion is also still working on their model and others are training models based on it, so there's a chance we will see improvements soon.

 

[Catacombs]

Was there a guide you followed to get a local version of Stable Diffusion running?

 

[Non-Edgy Gamer]

Was there a guide you followed to get a local version of Stable Diffusion running?

>Anon Guides
voldy guitard: https://rentry.org/voldy
inpainting: https://rentry.org/drfar
textual inversion: https://rentry.org/aikgx
AMD (docker): https://rentry.org/sdamd
AMD (onnx): https://rentry.org/ayymd-stable-diffustion-v1_4-guide
CPU: https://www.litechan.org/ran/67

There are also a couple of prepacks out there if you search for them.

https://mega.nz/file/bxYAlbLA#Qe2dffkISm3Zvozh4JMP3hwTCAHSif08tQYBjWBJY_E
This one is done by the guy who did KoboldAI. Just unzip the SD folder to your C drive and run it.

 

[Zed Duke of Banville]

Was there a guide you followed to get a local version of Stable Diffusion running?

You can find various guides online, but the basic steps:

1. Download Stable Diffusion and extract to your drive --- note the README file contains useful information
2. Download the latest Checkpoint data for Stable Diffusion (requires an email address, even if a fake one) and save it as [drive\mainfolder\]stable-diffusion-main\models\ldm\stable-diffusion-v1\model.ckpt
3. Download and install Miniconda (I first tried with Anaconda, but this caused problems)
4. Download and install the latest version of Python, if you don't already have it
5. Open a command prompt in Miniconda, change directory to [drive\mainfolder\]stable-diffusion-main, and then enter the statement conda env create -f environment.yaml (this should be necessary just once)
6. At this point, any time you want to run Stable Diffusion, you can just
   1. Open a Miniconda command prompt
   2. Change directory to [drive\mainfolder\]stable-diffusion-main
   3. Enter the statement conda activate ldm
   4. Start running your art generation commands, such as python scripts/txt2img.py --prompt "Duke Stefan Karameikos, monarch, royalty, fiery red hair flecked with gray, fiery red beard flecked with gray, sea-blue eyes, gruff, candid, uncompromising, William-Adolphe Bouguereau, Edmund Leighton, Frank Dicksee, John William Waterhouse" --plms --n_iter 5 --n_samples 1
   5. Note you can change the seed number by adding --seed #, where # is whatever number you want to enter
   6. Output appears in [drive\mainfolder\]stable-diffusion-main\outputs\txt2img-samples\samples; also, a single image containing all five images created from a seed appears in the folder above that

For GAZ5 The Elves of Alfheim, I continued using the Mucha/Rutkowski/Artgerm combination; although it should be possible to generate elves and dwarves in a specific artistic style, it would be more difficult. There is a section on notable personalities, which includes notes on appearance and personality, and I also requested a forest background in the prompts:


King Doriath Erendyl:

Brendian Erendyl:

Lynnwyl Chossum:

Dyradyl Feadiel:

Arloen Treelife of the Grunalf Clan:

Durifern Widefarer of the Grunalf Clan:

Beasthunter of the Longrunner Clan:

Coolhands of the Longrunner Clan (my favorite of the GAZ5 portraits)

Mealidan Mealidil:

Redsword Truetalker of the Red Arrow Clan:

Brightsword of the Red Arrow Clan:

Shurengyla is a stereotypical blonde elf-maiden, so a few of her better portraits could be used to represent different elf-women:

 

[Zed Duke of Banville]

I'll return to specific art styles with GAZ7 The Northern Reaches, but first comes GAZ6 The Dwarves of Rockhome. As with Alfheim, I relied on the Mucha/Rutkowski/Artgerm combination, now with a mountain or underground background.


King Everast XV

Bofin Everast (son of the king)

Noris Everast (elder daughter of the king)

Duris Everast (younger daughter of the king)

Dorto, head of the Buhrodar clan

Thoric Redhand, son of Dorto

Duric, head of the Hurwarf clan

Thrais, head of the Skarrad clan

Dwalur, head of the Syrklist clan

Korin, head of the Torkrest clan

Balin, son of Korin

Belfin Elf-Friend, head of the Wyrwarf clan

Gilia Songsmith, wife of Belfin

Morur Blackheart

 

[deuxhero]

Should include the descriptions the illustration is based on.

 

[Lagole Gon]

Good job typing stuff.

 

[Nortar]

Belfin Elf-Friend, head of the Wyrwarf clan

Yep, this is exactly the kind of dwarves who would make friends with elves.

 

[KateMicucci]

Yeah ok IDK why the fuck I wasted years learning to draw.

In a couple years from now image searches will be flooded with nothing but AI slop.

 

[JamesDixon]

Yeah ok IDK why the fuck I wasted years learning to draw.

In a couple years from now image searches will be flooded with nothing but AI slop.

Computer programs will never replace real artists. The uncanny valley effect shows up plus the fact that computers can't figure out how to handle fingers/toes.

 

[Zed Duke of Banville]

Should include the descriptions the illustration is based on.

Some of the gazetteer PDFs are only images, not text, and the more verbose descriptions of characters are lengthier than most people would want to read, anyway. For Stable Diffusion prompts, I extracted the pertinent facts about their appearance and added some personality descriptors gleaned from the text. I encourage anyone interested in these characters to read the original work.


For GAZ7 The Northern Reaches, I returned to a particular style, with the pre-Raphaelite artists Dante Gabriel Rossetti, Edward Burne-Jones, Evelyn De Morgan, and Walter Crane. Unfortunately, this gazetteer, as with Ken Rolston's previous work about Ylaruam, does not contain a full section on notable personages, but it does include a few pages about the court of Ostland (though details about appearance are rather lacking):

Hord Dark-Eye the High King of Ostland, middle-aged, violent, and impatient:

Yrsa the Young, Queen of Ostland, a strong-willed presence with blonde braids:

Asgrim the Bowed, a dogmatic, unforgiving cleric (should also have a hunchback):

Since the fully pre-Raphaelite style seemed more difficult to work with and perhaps wasn't appropriate for the Northern Reaches, I created some generic vikings and valkyries in the style of Konstantin Vasilyev, N.C. Wyeth, and Justin Sweet:

Bonus pre-Raphaelite image that didn't look like Yrsa but could easily be used another character: