AI Dungeon 2 - a bizarre procedurally generated text adventure

[Non-Edgy Gamer]

Right now GPT just really sucks for actually gameplay.

Well, if you mean stats, then yes. It always will because the AI has no actual understanding of stats or math, it's just making guesses. Especially with smaller models like NovelAI that don't really understand the difference between STR 5 and STR 100 or that it's more likely for a strong monster to have the latter and not the former.

You may be able to hack around this with scripts to keep track of stats like in AID, but if you want it to be really "game-like" instead of the do-anything-you want sim, you really need to create a separate program that merely calls the AI via an API for things like dialog and descriptions.

This is theoretically possible with NovelAI and Kobold AI, from what I understand, but I'm not aware of anyone who has done it or plans to.
https://api.novelai.net/docs/static/index.html

For example, in a rogue-like, you could generate characters with backgrounds, personality profiles etc. via an API and store the data locally. Then whenever you want to do a significant interaction with the character outside of combat, dialog for example, you call the AI again, feed it the locally stored data and then output the conversation text.

For example:

[<Character data>]
[Scene: Receive a quest to <kill goblins> at <location5> from <character name>. Current location: <current location>]
[Writing style: descriptive, purple prose;.]

This being fed to the AI would likely output a decent and very descriptive conversation with a quest giver.

This could also work for other things that need to be generated. Even if it's as simple as prose monster or character descriptions generated from randomly generated character data. It doesn't need to be the centerpiece of the game, rather, it can merely be something that gives more life to it.

It might change in the future, but for now you're best at using 1st or 3rd for it.

Or just set it to 2nd person in the AN's writing style. It works fine.

 

[Trans-Financial-Man]

I don't see it as having any future in gaming, at least not how it works now. You can get a valid sexting response about 1/3 of the time and a valid gaming response more like 1/10 of the time. It's incredibly obvious that they know what their real market is and that's what they're focused on. There's just not a lot of overlap between those two purposes. For sexting you can get away with just looking over the last 50 lines and guessing what the next one should be. For a real game, you'd need to store and process a lot more data.

They keep jacking up the prices so it's not affordable for me to come in and test out every once in a while anymore, but I'd be shocked if they're coming up with any actual gameplay anytime soon.

Lambchop used to run AI Dungeon CYOAs, but I always wondered how much of it was the program's actual output and how much was his own editing.

Right now GPT just really sucks for actually gameplay. Latitude kept trying to turn it into a dungeon master or CYOA, but that's one thing it really sucks for several reason, one of them being the amount of tokens it would need to keep track of relevant information. It doesn't mean you can't set up a game like setting as long as you take the time to fine-tune all of your settings and swap out relevant information as needed, but that's not its strong suit at all. First and foremost it is a writing tool and a lot of people never seemed to grasp the tools they already had at their fingertips for improving their stories. I mean you still have loads of people using /do and /say in AI Dungeon which is the exact wrong way to make a story even if you want a random outcome and a lot of people never figured out how to take advantage of the Author's Note or World Info.

NovelAI already has all the features of AI Dungeon except for a huge model and because it was trained on books and literature instead of just CYOA junk it's not that good at second person. It might change in the future, but for now you're best at using 1st or 3rd for it. The other big thing is no filters and no restrictions, NovelAI can't read your necrobesitality Sonic the Hedgehog\Star Trek fic except you. It's also about a hundred times easier to edit any part of your story in NovelAI than it ever was in AI Dungeon making for a much smoother experience.

I feel like we're coming full circle since I'm old enough to remember that Star Trek may well have created furries as we know them. So a Sonic the Hedgehog crossover isn't out of left field. Just incredibly horrifying.

 

[Zeriel]

Biggest problem NovelAI has is the godawful character/entry limit. It would be GOAT if they'd just up it, but I guess that's $$$.

 

[Zeriel]

I have fond memories of text based adventures from the 80's. They felt very open because they didn't have to rely on the very primitive or non existent graphical tech of the time. But frankly we grew past them. Whilst the actual quality of games nowadays is absolute shite I just can't get back into old text adventures where they wanted you to type specific verbs for specific stuff and it just becomes incredibly tedious. AI Dungeon for all it's tech advances was more of that annoyance for me and also I could never get it to spit out a coherant story. I'm very looking forward to how devs take similar AI's, combine them with proc gen tech and get an AI to become essentially an AI games master who genuinely reacts to your choices. I really think it could work for a Daggerfall spiritual sequel. Like the one me and my Jewish masters were making until we decided to wait for Covid's 99% survival rate to stop killing people. But we never got that far tech wise. It's still something we'd like to visit when we can.

Wouldn't hold your breath on that front. What devs can do and what they actually do have almost no relation to each other. It's almost 20 years later and no one has even tried to do what Daggerfall tried to do again. Not even close. Modern Bethesda isn't even trying, and the ones that do try and get so much hype for being "le innovative procedural tech" like NMS are much more primitive in their actual procedural gen than Daggerfall was for its time.

 

[Non-Edgy Gamer]

Biggest problem NovelAI has is the godawful character/entry limit. It would be GOAT if they'd just up it, but I guess that's $$$.

The what? There are no limits to character entry that I'm aware of. I've pasted in tens of thousands of words at a time.

Or do you mean the output limit? You don't really want a long limit, seeing as how the quality will decrease the longer the output is.

That said, they just upped the output for Opus tier to 100 tokens max, which means about 400 characters iirc.

 

[Zeriel]

Biggest problem NovelAI has is the godawful character/entry limit. It would be GOAT if they'd just up it, but I guess that's $$$.

The what? There are no limits to character entry that I'm aware of. I've pasted in tens of thousands of words at a time.

Or do you mean the output limit? You don't really want a long limit, seeing as how the quality will decrease the longer the output is.

That said, they just upped the output for Opus tier to 100 tokens max, which means about 400 characters iirc.

Just a personal preference thing I guess, but before the filter got applied to AID I preferred longer output. It's just annoying having to cycle through multiple outputs to get a decent amount of prose. Maybe it's because I have no problem editing the output that I don't mind a longer one. And I often found longer tidbits with AID could turn out fine if you gave it enough tries--which is happening anyway.

At the end of the day I think more options is always better. If it costs too much on the backend to be available for average user, just make it a specific ala carte tier I guess.

 

[Non-Edgy Gamer]

Just a personal preference thing I guess, but before the filter got applied to AID I preferred longer output. It's just annoying having to cycle through multiple outputs to get a decent amount of prose. Maybe it's because I have no problem editing the output that I don't mind a longer one. And I often found longer tidbits with AID could turn out fine if you gave it enough tries--which is happening anyway.

At the end of the day I think more options is always better. If it costs too much on the backend to be available for average user, just make it a specific ala carte tier I guess.

Given the way things are going, they'll probably get there soon enough. They're planning to make it possible for you to run your own small finetune if you have Opus tier, so I think a larger token limit will be on the table eventually too.

 

[rusty_shackleford]

so what's the one to use now?

 

[Zeriel]

AID is a literal joke because the filter will ban itself (the AI can generate content on its own that flags itself as a pedophile, kek). So it's not really an option anymore. Sadly NovelAI is your only real option. There's HoloAI and while it has some merits, its model is just too wonky and random.

 

[Non-Edgy Gamer]

so what's the one to use now?

Yeah, I second NovelAI. It's not as good as AID was last summer, but in overall quality, it's comparable with Griffon at least. Maybe better in terms of how well it writes.

If you just want to do a short game, you can still use AID's free mode though, just keep in mind that it is broken and they will IP ban you if they don't like what you write.

Btw, I have been testing NAI's AI modules and they seem quite good so far.

I used Lovecraftian and had it write the following from a blank prompt with this as the Author's Note:

[Plot: A young elf girl is lost in a dark forest.]

The trees were like the ones that had been at the foot of the hill. They grew close together and formed an impenetrable wall across her path, shutting out all light from above. The ground was hard and dry, with no sign of moss or undergrowth; it felt as if she walked on bare rock. She tried to keep track of time by counting the steps between each tree, but found this difficult because there seemed to be so many of them. In addition, they stood in such perfect alignment that their trunks made straight lines wherever one looked. There was something vaguely familiar about these rows—but what?

"Where are we?" whispered Nell. "And where's my horse?"

She stopped walking and stared into the blackness ahead. Something moved among the shadows. It might have been a branch swaying gently against another trunk, except for its size. Then she saw it clearly: a shape crouched in front of her, watching her intently. She took a step back, then realized that she could not see anything beyond the creature's head. Its body was completely hidden by the dense growth around it.

Continued...

Nell shivered suddenly. This place gave her the creeps! What sort of being lurked here? And why did it seem so intent upon observing her movements? Hadn't it seen enough already? Was it going to come after her now, just when she needed most protection? Her eyes searched the darkness until she spotted two more shapes lurking nearby. One appeared to be much larger than the others, although she couldn't tell how big it really was. Another pair of eyes glinted far off to her right. She turned quickly, expecting an attack at any moment. But nothing happened. All four creatures watched silently as she continued to walk. When she reached the end of the row of trees, however, she heard rustling behind her. With a start, she whirled around. Three pairs of glowing yellow eyes peered down at her from the gloom.

"Who're you?" demanded Nell sharply. "What do you want?"

There was silence again. The three pairs of eyes blinked once, slowly. Then they darted away, vanishing into the darkness before she could even blink. She waited patiently for some sign of movement, but none came. After several minutes passed without incident, she began to relax. Perhaps those things had simply gone somewhere else...

But the next thing she knew, there were voices coming toward her through the woods. At first she thought they were only echoes, but soon the sound became clearer. They were human voices, speaking in low tones. Nell tensed, afraid that whatever had scared away those other creatures would return. But instead of growing louder, the voices faded rapidly as whoever was talking got farther away. Finally, there were no more sounds to be heard. Nell relaxed slightly. Maybe she'd imagined everything; perhaps she should go home now. As she started to turn around, however, a new voice spoke up, sounding very close by.

"I'm over here," it said softly.

Nell spun around, still holding her staff tightly in both hands. There stood a man wearing a long cloak and hood, his face half concealed by shadow. He held out a hand and beckoned her closer. Startled, she hesitated.

"Come on!" he urged. "Don't be shy."

Reluctantly, Nell approached him. She could see little except that the stranger wore a ring with a large red stone set in the middle of it—a ruby or garnet, she guessed. It was hard to make out details because of the shadows cast by the tall bushes all around them. Even so, the man's face was clearly visible: pale skin, high cheekbones, a hooked nose, and a sharp chin.

No other input from me. Rerolled a couple of times, but not every time. Probably 5 rerolls total given that I rerolled the whole story initially to make sure it wasn't a fluke. No edits at all.

I don't know if that's quite Lovecraftian, but it's a step in the right direction without much prodding. Using Lovecraftian as a writing style and/or genre would probably up the creepiness further.

 

[Non-Edgy Gamer]

https://novelai.medium.com/custom-ai-modules-dbc527d66081

Introducing Custom AI Modules
Today, we’re excited to launch the long awaited AI Module training feature to Opus tier users.

What are AI Modules?

AI Modules menu in NovelAI.
AI Modules directly influence the behavior and the generations of the AI. You can use the default Modules we provide or you can train your own modules on the training data you provide! For example, you can create a module for the AI to mimic the style of your favorite author, or you can do a module for your favorite novel series to make the AI learn about your favorite characters!

Who can train their own custom AI Modules?
Custom AI Module training is exclusive to Opus tier subscribers — but sharable & usable by all tiers.

Why is AI Module training exclusive to Opus?
Our initial plan is to release custom AI Module training as an Opus only feature since it is a very resource heavy procedure to provide this feature. We are going to monitor demand, feedback and suggestions in order to evaluate how we continue forward.

This will help us determine if we have a need to provide options for single time purchases for a potential rollout to lower tiers further down the line.

How many AI Modules can a user train?
Custom Training is defined and limited through Training Steps. Training steps shows how much of the material text the AI will read/train over.

Each Opus tier subscriber receives 8000 Training Steps a month that will reset on the next billing cycle.

The maximum file limit for your uploaded training material file is 10MB per module.

Examples for default module’s training:
Style: Edgar Allan Poe is trained on his collected works, a 3.6mb file and took around 1000 steps to train.
Theme: 19th Century Romance is trained on 9 files from 6 different authors, at 4.7mb and took around 3000 steps to train.
Inspiration: Crab, Snail, and Monkey is trained on 20 files at 5.8mb and took around 3000 steps to train.

How can I start training models?

Open the NovelAI menu and click on Module Training. If you don’t see such an option you might need to refresh your page.

After this, the AI module trainer screen will pop up. Next you should upload your material and select how many steps you want to train for. The default of 1000 steps is a good place to start, don’t forget to give your module a name and description too. Also remember that you can select multiple files!

After done, click Train. If there is people ahead on your queue you will have to wait for that, after it’s your time to train it will show you the progress as percentage.

When this screen comes up, it means your module training is done! Your module is already imported to your modules menu, but if you want to share it with others you should download the model right now, and after that delete your training data.

Important Note: Trained modules are only compatible with the current AI Model, if we change the AI Model in the future they might not work on the new models.

How long does it take to train an AI Module?
Every second around 5 steps are processed. This means an average module with 1000 steps would train in ~3 minutes.

Once you have used up your 8000 training steps you will need to wait for the next billing cycle in order to reset.

You can only train one AI module at a time and cancelling the training of a started AI module is not an option.

Recommendations for Training/Cleaning Data:

• Biggest rule of thumb: put in your data formatted the way you want the AI to generate back.
• Plain text in .txt format, avoid formatting tags/markdown or html.
• Use prose.
• One paragraph in one line, no paragraphs split onto multiple lines.
• No empty lines between paragraphs.
• Empty lines could be used for chapter break, but we highly recommend using *** instead.
• No leading/trailing space, tab or other whitespace (easy to clean with programs like Notepad++).
• Ideally, use regular quote and single quote characters “ and ‘ not fancy ones (Easier for on the AI).
• Make sure all included material is focused on what your module should achieve.
• If you want to avoid the leaking of character names/other details, keep the data balanced and include many different stories with different characters.
• If you want the AI to learn character names/other details, train it on specific series.
• Feel free to experiment with short data. Nothing stops you from turning a short prompt into a module, which will require much less steps too.
• Avoid using encyclopedic data unless you plan to generate encyclopedia entries.

How do I share AI Modules?
Once you have trained your module you will be able to download the file to share it.

Other people can download this module and import it to easily start using it. Training is Opus only, but all the other tiers can import trained modules.

To import, you can drag and drop the module to the story and after doing that it should be visible in the AI modules menu or the dropdown.

You can also use the import button on the right menu to import modules.

Will AI Module training data be kept by NovelAI?
Training Data is deleted after training is done and not logged by us.

Updates to default AI Modules
We have improved these default modules:

• Inspiration: Crab, Snail, and Monkey
• Inspiration: Witch at Level Cap
• Theme: Hunter Gatherer
• Theme: Magic Academy

tldr: if you have an Opus tier subscription to Novel AI, you can now train your own custom modules. Eg: your favorite book series, scripts from TV shows, wiki articles, whatever.

 

[Tweed]

This thing just gets better and better all the time.

 

[Zeriel]

https://novelai.medium.com/custom-ai-modules-dbc527d66081

Introducing Custom AI Modules
Today, we’re excited to launch the long awaited AI Module training feature to Opus tier users.

What are AI Modules?

AI Modules menu in NovelAI.
AI Modules directly influence the behavior and the generations of the AI. You can use the default Modules we provide or you can train your own modules on the training data you provide! For example, you can create a module for the AI to mimic the style of your favorite author, or you can do a module for your favorite novel series to make the AI learn about your favorite characters!

Who can train their own custom AI Modules?
Custom AI Module training is exclusive to Opus tier subscribers — but sharable & usable by all tiers.

Why is AI Module training exclusive to Opus?
Our initial plan is to release custom AI Module training as an Opus only feature since it is a very resource heavy procedure to provide this feature. We are going to monitor demand, feedback and suggestions in order to evaluate how we continue forward.

This will help us determine if we have a need to provide options for single time purchases for a potential rollout to lower tiers further down the line.

How many AI Modules can a user train?
Custom Training is defined and limited through Training Steps. Training steps shows how much of the material text the AI will read/train over.

Each Opus tier subscriber receives 8000 Training Steps a month that will reset on the next billing cycle.

The maximum file limit for your uploaded training material file is 10MB per module.

Examples for default module’s training:
Style: Edgar Allan Poe is trained on his collected works, a 3.6mb file and took around 1000 steps to train.
Theme: 19th Century Romance is trained on 9 files from 6 different authors, at 4.7mb and took around 3000 steps to train.
Inspiration: Crab, Snail, and Monkey is trained on 20 files at 5.8mb and took around 3000 steps to train.

How can I start training models?

Open the NovelAI menu and click on Module Training. If you don’t see such an option you might need to refresh your page.

After this, the AI module trainer screen will pop up. Next you should upload your material and select how many steps you want to train for. The default of 1000 steps is a good place to start, don’t forget to give your module a name and description too. Also remember that you can select multiple files!

After done, click Train. If there is people ahead on your queue you will have to wait for that, after it’s your time to train it will show you the progress as percentage.

When this screen comes up, it means your module training is done! Your module is already imported to your modules menu, but if you want to share it with others you should download the model right now, and after that delete your training data.

Important Note: Trained modules are only compatible with the current AI Model, if we change the AI Model in the future they might not work on the new models.

How long does it take to train an AI Module?
Every second around 5 steps are processed. This means an average module with 1000 steps would train in ~3 minutes.

Once you have used up your 8000 training steps you will need to wait for the next billing cycle in order to reset.

You can only train one AI module at a time and cancelling the training of a started AI module is not an option.

Recommendations for Training/Cleaning Data:

• Biggest rule of thumb: put in your data formatted the way you want the AI to generate back.
• Plain text in .txt format, avoid formatting tags/markdown or html.
• Use prose.
• One paragraph in one line, no paragraphs split onto multiple lines.
• No empty lines between paragraphs.
• Empty lines could be used for chapter break, but we highly recommend using *** instead.
• No leading/trailing space, tab or other whitespace (easy to clean with programs like Notepad++).
• Ideally, use regular quote and single quote characters “ and ‘ not fancy ones (Easier for on the AI).
• Make sure all included material is focused on what your module should achieve.
• If you want to avoid the leaking of character names/other details, keep the data balanced and include many different stories with different characters.
• If you want the AI to learn character names/other details, train it on specific series.
• Feel free to experiment with short data. Nothing stops you from turning a short prompt into a module, which will require much less steps too.
• Avoid using encyclopedic data unless you plan to generate encyclopedia entries.

How do I share AI Modules?
Once you have trained your module you will be able to download the file to share it.

Other people can download this module and import it to easily start using it. Training is Opus only, but all the other tiers can import trained modules.

To import, you can drag and drop the module to the story and after doing that it should be visible in the AI modules menu or the dropdown.

You can also use the import button on the right menu to import modules.

Will AI Module training data be kept by NovelAI?
Training Data is deleted after training is done and not logged by us.

Updates to default AI Modules
We have improved these default modules:

• Inspiration: Crab, Snail, and Monkey
• Inspiration: Witch at Level Cap
• Theme: Hunter Gatherer
• Theme: Magic Academy

tldr: if you have an Opus tier subscription to Novel AI, you can now train your own custom modules. Eg: your favorite book series, scripts from TV shows, wiki articles, whatever.

I can attest to the fact that this is REALLY good at improving short or non-existent prompts. You can probably achieve comparable results by writing manually for a long time, but taking even 1-2 chapters of your own prose and putting it into a module seems fantastically good at letting very short prompts not be retarded.

I thought at first the idea was you needed to feed it a huge body of work to get a good model, and this may be true as something you'd use for everything--but they also mentioned the way it works you kind of want it to be targeted. So if you're going for a specific theme or style or even a specific scene, having a very targeted but brief module training content may be superior to 1000s of pages of more varied fiction if it's not aimed at what you want.

 

[Non-Edgy Gamer]

I thought at first the idea was you needed to feed it a huge body of work to get a good model, and this may be true as something you'd use for everything--but they also mentioned the way it works you kind of want it to be targeted. So if you're going for a specific theme or style or even a specific scene, having a very targeted but brief module training content may be superior to 1000s of pages of more varied fiction if it's not aimed at what you want.

Yeah. There's only so much it can do with the roughly 205kb of data it boils it down into.

Some kind of consistency is better than nothing though. If you make it too short and all from one source, you might have characters show up from that story a la Count Grey or Kyros' frequent unwanted and frequent appearances in AID.

 

[Multi-headed Cow]

That sounds cool, but I can't really justify $25 a month for AI text fuckery. When AID was $5 a month I was on board (Until that was due to expire and Dragon kept getting shittier, at least) and even that was a bit of a stretch for me since even though I was playing an hour or two a night and I realize that AID uses an ungodly amount of horsepower for Dragon, I never did anything with it I felt like saving and the vast majority of it was fucking around with the AI and seeing what it would do rather than trying to make a story. I do miss it since I had fun thinking of some stupid prompt and giving it a whirl but even $10 a month would be stretching it. And I'm dubious on if NovelAI's as good as Dragon, especially Dragon before the damn dirty mormon dumbed it down. Which people swore up and down never happened but I tested a few prompts many times and saw they'd give fairly consistent results based on world info and everything, then a week or a month would pass and then it couldn't remember jack shit. Admittedly that might've been OpenAI changes in the background and not the mormon, but the mormon gets the blame regardless.

Also tried some of that JAX shit using colab like the dark ages, but it felt too stupid to be enjoyable. Either I was remembering colab AID in a better light (Though I do remember the fucker crashing after only a few inputs) or I'd forgotten just how rough non-Griffon GPT2 is. Especially after tasting the fruit of GPT3 and being able to tell it to rewrite blocks of text as though it was in Max Payne.

 

[Zeriel]

IMO NovelAI is pretty good, maybe even better than Dragon, but there is definitely some idiosyncratic "flavor" to the Dragon model. Sometimes NovelAI is better, but I can still think of things Dragon did that were more interesting sometimes.

NovelAI really needs a certain amount of effort in setting it up to be good though--for the casual user I feel like the old Dragon will always be better. But that's not a real-world comparison, since modern AID is broken due to the filter which will generate content on its own it then bans itself (meaning you, but heh) for writing. So it's just not even really a choice anymore. I'm only glad NovelAI is now at the point where it's legitimately as good as Dragon was, rather than being in a situation where Dragon is still theoretically the best but it doesn't matter because the filter ruins it.

I thought at first the idea was you needed to feed it a huge body of work to get a good model, and this may be true as something you'd use for everything--but they also mentioned the way it works you kind of want it to be targeted. So if you're going for a specific theme or style or even a specific scene, having a very targeted but brief module training content may be superior to 1000s of pages of more varied fiction if it's not aimed at what you want.

Yeah. There's only so much it can do with the roughly 205kb of data it boils it down into.

Some kind of consistency is better than nothing though. If you make it too short and all from one source, you might have characters show up from that story a la Count Grey or Kyros' frequent unwanted and frequent appearances in AID.

Didn't they say it's up to 10mb? Or do you mean the derived data from that?

 

[Non-Edgy Gamer]

That sounds cool, but I can't really justify $25 a month for AI text fuckery.

The $15 tier is more than sufficient if you aren't going to train modules. You can still use the modules that others have trained, which people are adding all the time.

The $5 tier is ok, but the context will be smaller. (Though actually, that will still be bigger than AID's context, so it might not matter if you're used to that anyway.)

Also tried some of that JAX shit

I ran JAX as well. It's quite a bit better than JAX.

IMO NovelAI is pretty good, maybe even better than Dragon,

It really isn't better than Dragon or even as good as it. It matches old Griphin at times (new Griphin seems unusably stupid now).

Dragon's GPT-3 is just superior when it comes to making connections and being inventive without being dumb. Of course, its larger context size helps it make up for that at times - giving it the ability to remember things from much further back in a story and stay on track, but I can't ever equal it to the stuff I tried on old Dragon or even the later, more limited editions (the new one is a hit or miss that bans you, but it's still a bit more on the ball than NAI most of the time). I think most of the people who compare it to old Dragon are just using it for porn, and even then, I really doubt it compares. (Mind you, I'm talking about what it DOES and not how it writes. The quality of the prose itself is usually very decent, even if it messes up details now and again. I can understand how you'd think that was better than Dragon, especially if you didn't use the Author's Note to make the writing better in Dragon.)

Does that mean it can't be worked around? Of course not. But the fact is, that old feeling letting Dragon finish your sentences or of just thinking of something offhand and having Dragon do it and surprise you with how good it is will rarely happen.

That said, they're still developing it and there are going to be MUCH larger models released by Eleuther AI soon that might actually compare with Dragon. And we now know that NAI's team is a lot more competent when it comes to training and modifying models than Latiude/AID ever was.

Time will tell.

Didn't they say it's up to 10mb? Or do you mean the derived data from that?

I meant the result. No matter how much or how little data you squeeze in there, it will always end up about 205kb.

 

[Zeriel]

Yeah, you put it pretty well there. For what it's worth I've never considered any of these AI-generated-content tools to be good at complex storytelling though, so I am indeed just judging it based on the quality of writing it puts out minute to minute. I think these apps are all best at generating some decent descriptive writing of something--if you want it to create the complex connections of an actual plotted novel... I dunno, even Dragon wasn't good at that, and structurally the AI is not even capable of that, since it is just trying to create what it thinks you expect based on what came before, it's a prediction algorithm not a work of creativity.

Ironically, yeah... that does make, despite the Mormon's salt, the best suited application for these things writing porn.

 

[Non-Edgy Gamer]

I dunno, even Dragon wasn't good at that, and structurally the AI is not even capable of that, since it is just trying to create what it thinks you expect based on what came before, it's a prediction algorithm not a work of creativity.

It could do it if it had enough guidance. That's part of what the summary feature of Dragon does. Actually, NAI can do it if you give it a plot outline. Will it be good without you helping it though? Not really.

I'm not really talking novels though. I mean that on a input by input basis, Dragon was superior. It "got" things a lot easier.

By "connections" I mean the number of parameters the AI can factor into an output. That's something that's mainly limited by the size of the model, from what I understand.

You know I'm non-edgy gamer, you know that this is the Codex, you know that my avatar is male and that it's from Avernum. Now, imagine that you couldn't connect a few of these things properly and that it made you think the wrong thing about me. Maybe you'll keep thinking I am playing a video game instead of on a forum, talking about them. That's the difference between Dragon and NAI. NAI is over 6 billion parameters. Dragon is over 130 billion. It's just going to end up better at some things for now.

Ironically, yeah... that does make, despite the Mormon's salt, the best suited application for these things writing porn.

If you knew how to use it, it could do a lot more. Most companies that use GPT-3 are using it for writing articles, social media posts or creating chat bots. There are entire companies built around using it to help with writer's block, and they censor the porn even harder than AID, blocking all of it.

Writing porn just provides the most instant gratification, so that's what most people try and what most people keep doing rather than bothering to learn how to use it for other features.

Writing new quests for your favorite game characters and keeping the writing and dialog style, inserting yourself into your favorite story and having the characters respond at least close to how they logically would, adding a chapter to a finished series or writing a story that you have a rough idea about, but need help getting past your writer's block - these are all things AI can do.

But of course "> You fug da girl" will always be what 90% of people will stick with. Especially those who couldn't be bothered to learn how to write World Info or use Memory, which I suspect was the vast majority of AID users.

 

[Zeriel]

I dunno, even Dragon wasn't good at that, and structurally the AI is not even capable of that, since it is just trying to create what it thinks you expect based on what came before, it's a prediction algorithm not a work of creativity.

It could do it if it had enough guidance. That's part of what the summary feature of Dragon does. Actually, NAI can do it if you give it a plot outline. Will it be good without you helping it though? Not really.

I'm not really talking novels though. I mean that on a input by input basis, Dragon was superior. It "got" things a lot easier.

By "connections" I mean the number of parameters the AI can factor into an output. That's something that's mainly limited by the size of the model, from what I understand.

You know I'm non-edgy gamer, you know that this is the Codex, you know that my avatar is male and that it's from Avernum. Now, imagine that you couldn't connect a few of these things properly and that it made you think the wrong thing about me. Maybe you'll keep thinking I am playing a video game instead of on a forum, talking about them. That's the difference between Dragon and NAI. NAI is over 6 billion parameters. Dragon is over 130 billion. It's just going to end up better at some things for now.

Ironically, yeah... that does make, despite the Mormon's salt, the best suited application for these things writing porn.

If you knew how to use it, it could do a lot more. Most companies that use GPT-3 are using it for writing articles, social media posts or creating chat bots. There are entire companies built around using it to help with writer's block, and they censor the porn even harder than AID, blocking all of it.

Writing porn just provides the most instant gratification, so that's what most people try and what most people keep doing rather than bothering to learn how to use it for other features.

Writing new quests for your favorite game characters and keeping the writing and dialog style, inserting yourself into your favorite story and having the characters respond at least close to how they logically would, adding a chapter to a finished series or writing a story that you have a rough idea about, but need help getting past your writer's block - these are all things AI can do.

But of course "> You fug da girl" will always be what 90% of people will stick with. Especially those who couldn't be bothered to learn how to write World Info or use Memory, which I suspect was the vast majority of AID users.

Specifically what I mean is text porn is mostly based on describing some sort of stimuli in detail. So "write something based on what came before decently" works really well for that. Anything based on ideas and human judgment is gonna get really iffy, and I stand by that even in regards to the best Dragon results.

It's kind of like the spirit of an advertisement versus a movie. There are movies made by guys who started in advertising, and they are really good at glitz and action scenes, things that catch the eye. There is no substance though. That's basically how I see AI content generation.

The writer's block thing is a good point. That's initially what drew me to it--it's definitely interesting as something to spit out a bunch of random concepts or ideas at you if you're looking for something to give you broad-based ideas, and sometimes you're not feeling creative enough to come up with those ideas on your own.

 

[Non-Edgy Gamer]

It's kind of like the spirit of an advertisement versus a movie. There are movies made by guys who started in advertising, and they are really good at glitz and action scenes, things that catch the eye. There is no substance though. That's basically how I see AI content generation.

This effect diminishes with increased parameter count.

"If this, then X" is something parameter count helps with. And that's what gives logic and substance to writing, beyond just imitating pretty prose. The larger the model, the better it gets. Within reason, and with proper training.

On OpenAI's end, Dragon can even explain why it made a specific choice when it came to content generation, and though NAI can't do this (because it lacks knowledge graphs iirc), it makes similar decisions. "X character is an old man. He's old because he's 70." So it isn't just picking things at random based on the words before it.

I realize that the commonly understood explanation is that it's just a text prediction algorithm, and on a basic level, it is, but it's also more complex than that. A lot more went into GPT-3 than just jamming a bunch of text together and training a model off it.

 

[Non-Edgy Gamer]

Update: Novel AI just released a Text Adventure module that basically replicates the Do/Say functionality of AI Dungeon. It's surprisingly functional.

 

[Non-Edgy Gamer]

Latitude's drama continues:
https://latitude.io/blog/a-new-story-experience/

Now they're giving people a their own AI (probably just GPT-JAX) whenever Open AI's filter gets triggered. Not sure if it still triggers a review process still, but the autobans are gone it seems.

If your story triggers OpenAI’s filters it will be handled by our own models rather than resulting in account suspension. We will still use and continue to work on our own filters to enforce our content policies, but they will be much more carefully targeted, ensuring that you aren’t unfairly penalized for incorrectly flagged content.

So basically, you get forced on the slow bus for the rest of your story instead of being banned completely. From 175 billion parameters to 6 billion, and probably untrained or poorly trained. Ouch.

The company is basically done at this point imo. NovelAI's lead dev says NovelAI has 10,000 subscribers now and that all of them probably came from AID's paid userbase. That's a lot of money to lose overnight.

 

[rusty_shackleford]

I don't even understand the rationale behind this sort of censorship. If we follow it to its implication that writing about something means you want to do it, shouldn't they ban all stories that feature violence or murder?
I can dislike what someone is writing and still think they should be allowed to write it.

 

[Zeriel]

They're just bad at their jobs is my assumption. I heard some scuttlebutt that most of the company's employees were on another project--so maybe what happened was the combination of a moralistic diktat from the higher-up and not enough employees on the project to implement it in a smart fashion. The main issue if you set aside a libertarian argument was that their implementation was so terrible it made people just writing normal stories get banned. It's like those chat filters on some MMOs where writing "grape" is censored, because it contains the letters "rape", only in this case the AI would generate bannable content on its own, then ban you for what it wrote. Or hilariously famous things like it interpreting "a 12 inch socket wrench" as being associated with a 12 year old, and hence, BAN.

They were also interpreting words like "youthful" and "young" to mean underage, which means that just about any fiction writing from the 90's (and even now) that used such words to refer to younger adults in their 20s, or just attractive people, would get banned. Trying to write word for word any normal book that even young adults (i.e kids) read for school assignments would get you banned for child pornography. They just fucking shanked themselves, no simpler way to say it than that.

I'm curious about them saying its driven by Open AI. Does Open AI just want their model exclusively used to generate columns for Vox for something? Because the stuff they banned would be completely normal stuff to find in fiction books in the library. If I was a customer of Open AI and wanted to use it to generate ideas for writing normal fiction to be published, and Open AI told me I couldn't have anything violent or even PG-13... jesus christ, what is left as a viable business model? I don't know whether to assume Open AI is truly more censorious than Big Brother in their attitudes, or if Latitude is just using them as a scapegoat for their own incompetence (likelier). Either way it doesn't matter since open source models allow a way forward that isn't chained to this retardation.