AI Dungeon 2 - a bizarre procedurally generated text adventure

[Non-Edgy Gamer]

They were also interpreting words like "youthful" and "young" to mean underage, which means that just about any fiction writing from the 90's (and even now) that used such words to refer to younger adults in their 20s, or just attractive people, would get banned. Trying to write word for word any normal book that even young adults (i.e kids) read for school assignments would get you banned for child pornography. They just fucking shanked themselves, no simpler way to say it than that.

Yep.

I'm curious about them saying its driven by Open AI. Does Open AI just want their model exclusively used to generate columns for Vox for something? Because the stuff they banned would be completely normal stuff to find in fiction books in the library.

Welcome to corporate sanitization. The lack of logic is due to politics, of course. The society that permits movies like Kingsman, wherein one of the main characters slaughters a church full of people and we're supposed to be ok with it because they were mildly racist, but wants to ban speech that might lead to hate.

However, the extent they're willing to go to in cases like these can often have very ration reasons, usually related to banking. The banking laws have basically made banks responsible for just about anyone they do business with. You can see a good example of this in OnlyFans, which is BANNING PORN starting in September (get rekt, thots).

So, my guess is that either OpenAI's bank(s) came to them and said something along the lines of "cut it out or we'll cut you off", or else their legal team was worried they would (more likely).

Latitude was then likely told something similar by OpenAI and then likely overreacted with their efforts to get rid of problem users. When users began to push back on these methods, they likely overreacted again, cracking down harder in response to a challenge rather than seeing that their userbase was justifiably upset.

As most of you know, we’ve been testing various filters and suspension systems. We recognize that these systems have had false positives and unintended side effects that we know have frustrated many of you (trust us, we haven’t wanted those either). We regret the frustration these caused and the way we had to roll those out.

What some may not know is that, behind the scenes, we needed to implement these changes to prevent AI Dungeon from being required to use the much stricter content filter that would normally be required for use of OpenAI’s GPT-3 models. That filter often blocks even mild fantasy violence, which would obviously hurt the magic of AI Dungeon—we believe that you should be able to fight off bandits, slay dragons, and have all of the kinds of epic adventures that make AI Dungeon so fun.

While we’ve been working with OpenAI to find a joint path forward, it’s clear that we have different approaches to providing AI experiences to users safely, and that Latitude needs to be able to decide on the approach that best supports our users. Because of this, we have decided to transition to a new implementation that gives us more flexibility in how we let you experience the magic of AI.

These three paragraphs, along with a commitment to user privacy, would have been very effective back at the beginning of all this. I really doubt users would have been nearly as angry, or as desperate to find alternatives. Now it's too little, too late.

 

[Multi-headed Cow]

Latitude's drama continues:
https://latitude.io/blog/a-new-story-experience/

Now they're giving people a their own AI (probably just GPT-JAX) whenever Open AI's filter gets triggered. Not sure if it still triggers a review process still, but the autobans are gone it seems.

Interesting, so the filters were on OpenAI's side of things and not AID's side? Assumed it was the mormon being up to no good but if it's OpenAI itself that's even worse since they're the bastards with the wondermachine in the first place. Makes it feel even less likely we'll have access to something as good as GPT-3 without being lobotomized in any sort of decent timeframe.

Either way it doesn't matter since open source models allow a way forward that isn't chained to this retardation.

Hopefully, but I'm leery on if they'll be able to reach the level of training GPT-3 has and have machines beefy enough to host it. I do certainly hope that we'll get to Dragon-like text AI in the $5ish subscription range within the next 5 or so years but that might've just been a brief fluke. Do still miss it on occasion, would spend so goddamn many hours just fucking around with it. Fantastic toy for the imagination, punching in a prompt based on the text for this picture for example gave me a bunch of weird-ass fun, and looking at somewhat ill-defined worlds and situations like that through the lens of the AI was a real treat. Typing "I AM ON BABYLON 5 AND DELENN IS THERE AND SHE WANTS TO HAVE SEX WITH ME" is cool and all but the AI's so much more fun when you aren't in an existing fictional setting and you can roll with a lot of the weird bullshit it comes up with. Closest equivalent is just getting high and letting your mind wander while looking at art which does admittedly work but in slightly different ways. Using your imagination to interpret what the AI spit out and how that fits into the context of the story as a whole is where you get into the weird collaborative writing with the computer which is what I miss.

https://www.keiththompsonart.com/pages/prophet.html

 

[Zeriel]

Not like we have much choice. OpenAI is a corporate dead-end. Anyway, EleutherAI are supposedly working on a GPT-3 equivelant, and imo properly trained NAI right now (which is on a technical level WAY less powerful than GPT-3) is already as good as nerfed Dragon in my book. Not everyone sees it that way, but I can only judge based off of my own experiences and needs. I like Opus NAI, and I like the fact they're improving noticeably in a quick turnaround time. That's the total opposite of Latitude, where its getting worse, slowly over time, and never getting better.

Admittedly I'm a writer and hence more into the third person perspective thing--that seems to be where a lot of people who really liked AID have a hard time with NAI, they want the CYOA thing. But they did add CYOA mode to NAI, so... I dunno. Guess it's personal taste at this point.

Edit: Btw, while they did say OpenAI was behind the strict need for a filter, they also said that even if OpenAI had never done anything, they would have made their own filter. Latitude are moralists, they just are trying to play PR and create an excuse at this late stage and deflect blame onto OpenAI.

 

[Non-Edgy Gamer]

Interesting, so the filters were on OpenAI's side of things and not AID's side? Assumed it was the mormon being up to no good but if it's OpenAI itself that's even worse since they're the bastards with the wondermachine in the first place. Makes it feel even less likely we'll have access to something as good as GPT-3 without being lobotomized in any sort of decent timeframe.

From what people are saying, now any NSFW content is triggering a hard "AI doesn't know what to say" filter from OpenAI.

Latitude's filter is different, but yeah, they may have gone that far due to OAI breathing down their necks. Doesn't make any of what they did excusable though. The violations of user privacy, the silent treatment they gave their community. But even that could have been prompted by NDAs. Which makes me wonder whether all of this is just OAI's way of showing them the door, since they likely aren't profitable and devalue their product by offering it so cheap.

As far as timeframes go though, you're looking at around 6 months before EleutherAI drops another big release. A Latitude employee was also asking questions about AI training costs of the Novel AI devs in EleutherAI's discord. Apparently, they're pricing what it would cost to train a 20B model. But that'd cost around $300,000, and it's unlikely that they'd do it.

So, yes, it sounds like GPT-3 is basically dead to us from OAI's end, and even Latitude wants to dump them if at all possible.

Hopefully, but I'm leery on if they'll be able to reach the level of training GPT-3 has and have machines beefy enough to host it.

Machines don't need to be bought. Cloud infrastructure is usually used. It's what NovelAI is using now.

Technically, they're already near or at the low-end of GPT-3 models. I think Griffin was 13B? Don't remember.

Using your imagination to interpret what the AI spit out and how that fits into the context of the story as a whole is where you get into the weird collaborative writing with the computer which is what I miss.

Yeah. Me too. NAI is decent, but it loses track of things too often and it doesn't seem as creative as Dragon, and certainly not to the point Dragon was last summer.

 

[Tweed]

Well, no matter what anyone says I don't seem to be able to get much out of second person unless I tell it to write smut and smut isn't what I want. For all the tuning, tagging, and tweaking I usually get sub-gryphon responses and it constantly forgets who's talking. It's much stronger with third person. I wonder if we'll ever get a bigger model.

 

[Non-Edgy Gamer]

Well, no matter what anyone says I don't seem to be able to get much out of second person unless I tell it to write smut and smut isn't what I want.

Are you using the text adventure module?

Before you stands an elven woman, of noble aspect and bearing.
She is clad in a robe of green silk embroidered with silver thread; her hair flows down upon it from beneath a diadem wrought like the moon at midnight. Her eyes are large and grey as stars. She speaks: "I am called Eärendel."

> You ask "Bitch, did I ask for your name?"
You do not remember doing so, but she answers: "My true name was never given to any man. It has been lost since ancient times when my people dwelt beyond the sea-girt islands that now lie sunken far below us. But men have named me by another name which they call Elwing."
"Elwing! Then why did you come here? Are you one of those who brought about our ruin?"
"No," says Elwing, shaking her head slowly, while tears fall from her eyelids on to her breast. "It was not I. My kinsmen were more powerful than myself; and their power grew apace until all things began to wither and decay around them. They became arrogant, cruel, and unjust, and took pleasure in trampling underfoot the helpless creatures whom they had made or bred. And then came the day when they fell into madness, and slew each other by tens and twenties. Only two escaped alive—myself and Fingolfin's son Beren, who fled away over the watery waste till he reached this land, where his heart failed him, and he perished miserably. Yet even before that hour there had grown up between the Elves and Men such hatred and enmity that no living thing could endure to pass between them. Therefore we who yet lived withdrew ourselves from the world, and shut out its evils with high walls and strong barriers; and only after long years have we ventured again to look abroad among mankind, and see how they fare. For many generations past none have passed through these gates save few wandering spirits, doomed either to wander endlessly, or else perish within the gloom. Now however a new hope has arisen in the hearts of some of my kindred, and once more I venture forth alone into the wide lands beyond the twilight hills, to seek tidings of what befalls on earth. Thence I bring back word of good or evil, according to the mood of fate. If fortune smiles upon me, I shall return ere long laden with rich gifts. That may well be, seeing that the wealth of Ilúvatar is infinite."
Hereupon she turns to go, but suddenly stops short, and looks behind her towards the west.
> You pull out your gun and shoot her.
The shot hits her squarely between the shoulders, and she falls forward without a cry. As she lies still, you hear the sound of bells ringing faintly in the distance.

Pink is my text.

Clearly drawing heavily from the Silmarillion fsr btw.

For all the tuning, tagging, and tweaking I usually get sub-gryphon responses and it constantly forgets who's talking. It's much stronger with third person. I wonder if we'll ever get a bigger model.

All of this is pretty normal. As I've said before, the main model for NAI is nowhere near GPT-3, (though neither is GPT-3 at this point).

We are supposed to get a much bigger model sometime within the next year, yes. Everyone is waiting on EleutherAI to release a new model, including AID/Latitude, who want to break free from OAI's proprietary models at this point.

 

[Tweed]

No, I was just writing standard prompts in a second person perspective. I tried a second person module, but I can't tell if it really makes that much difference. I'm perfectly aware Sigurd's 6B is no match for Dragon's 175B, but that model is useless as tits on a boar with all the filters and lobotomization.

sometime within the next year

 

[Non-Edgy Gamer]

I tried a second person module, but I can't tell if it really makes that much difference.

It makes a big difference if you want to write like that with the standard > You do/say x... actions.

The modules make a ton of difference on the whole, especially some of the user-trained modules. It isn't magic, but there is a major effect.

Another method you can try is the [Input: xxxx] method, which doesn't require a module:

You stare at the monster in front of you, trying to stare him, but you see nothing but hate and unquenching thirst behind his eyes.
"Yes... I see strength in you. Potential. I have an offer for you, as the newfound man of the house. Join me. Let me take you on as my... apprentice. As my servant. As my slave. As whatever I need you to be. In exchange for your servitude, I offer you not only your life, but the life of your mother and sisters."
You stare at the monster, whose eyes narrow as he waits for you answer.

[Input: you reject his offer]


The monster is silent. His eyes are still narrowed as he waits for your answer, but you can't find it. It seems so absurd to you. A monster offering to become your master, for what? What would you have to gain?
Your mouth falls open, words escaping in a scream. "NO! NO! NO!" You can feel yourself falling into madness. The beast's hands tighten around your arms. You struggle against him, twisting and turning like a child trying to free himself from a parent's grip.
[Input: you scream for help]

The monster turns, and with a roar, drags you into the darkness. Your cries echo in the tunnel, filling you with terror as you know the monster is right behind you.
"NO! NO! NOOO!"

"I'm going to kill you," you hear the monster whisper.
"And I am going to enjoy it. Because I was never here."
The monster laughs as he stalks towards you, and then you see the knife in his hand.
[Input: try to escape]


The knife is the only thing you can think of. You are trapped. You are naked, defenseless, alone, terrified. You stare at the blade of the knife in front of you. It gleams in the dark, reflecting the glow from the torches.
There is nothing else to do.
You reach out, groping for the hilt of the knife. You are going to die. There is no doubt about it.
[Input: you are stabbed to death]


The pain is too great. You feel like you are burning alive as the monster thrusts the knife into your chest. It slices through your flesh and bone. Blood gushes from your wounds. You fall to your knees, unable to scream.
The monster pulls the knife from your body and throws it on the ground.
[Input: you go to heaven]


Your eyes open. The sky is blue. The sun is shining. You're in a field of grass. A warm breeze blows across your skin.
"Hello?"
A voice calls out to you.
"Who's there?"
The voice comes closer. It is soft, gentle. You can hear laughter in it. You look up, and see an angel standing before you. She is beautiful.
She has long hair that flows down her back, and her face is radiant. Her eyes are deep blue.
"Are you okay?"
You nod. "I think so."
She kneels beside you, gently taking your hand. "You're safe now," she says.
"Where am I?"
"You're in heaven."
[Input: you sense something isn't right] NOTE: this is where I changed the writing style to Lovecraftian and dropped the enhancements.


You can't shake the feeling that something is wrong. You are aware of the angel, but she seems distant somehow. You turn your head to look at her, and you see the knife lying next to you. It is still wet with blood.
The angel smiles at you.
[Input: the angel suddenly turns into Cthulhu]


Your heart races as you realize the truth. You feel a sudden chill. The angel is now a giant monster. It is huge. Its eyes are black pits, its mouth a gaping maw. Its tentacles reach out to you.
"NO! NO! NOOOOOO!"

It can be hit or miss at times, but it generally works.

 

[rusty_shackleford]

how does holoai compare to novelai?

 

[Non-Edgy Gamer]

how does holoai compare to novelai?

Cheaper, less customizable (no user modules), no lorebook generation, and they have less technical knowhow when it comes to training AIs.

One of the guys on the NovelAI team is the one who ported 6B JAX to pytorch, which is the reason it has important quality of life additions like the ability to add negative characters (which things like author's note and memory need in order to work). AFAIK, Holo doesn't have that level of skill on their team and have mostly been piggybacking off Novel's work.

You're not going to get an insane difference in quality for most applications at this point (user modules can make a big difference though), but I think that NAI are actually committed to and able to push the technology further, while Holo is just playing catch up.

I would say NAI is worth the extra money unless you live in a country where the extra $10(?) a month for the $15 package is a big deal.

I'd say Holo is a good option if you can't afford NAI though. It's always good to have options and I'm glad they're continuing their work. Holo also has a free mode for you to try it out. Novel doesn't.

 

[Larianshill]

NovelAI did not seem particularly good to me. Maybe it's just nostalgia goggles, but Dragon seemed quite a bit more coherent.

 

[Tweed]

175b model vs 6b.

 

[Non-Edgy Gamer]

NovelAI did not seem particularly good to me. Maybe it's just nostalgia goggles, but Dragon seemed quite a bit more coherent.

It's not your imagination. But old Dragon is dead due to censorship. Even AI Dungeon has partially switched to the base model that NAI is using.

 

[Tweed]

Life is suffering for those of us who got to sample Pre-Lobotomized Dragon. The damn thing was magical, especially if you have any abilities in writing. As for the 22b model I keep hearing nothing but rumors. Everything from Eleuther just waiting on GPUs to them not having any money at all for it, to it actually being in training. It's all wind in the trees with nothing substantial and their own blog is silent on the subject. 22b still won't be dragon, but with all the tools NAI gives us it should be a night and day difference to Sigurd.

If it ever comes out.

 

[Non-Edgy Gamer]

Life is suffering for those of us who got to sample Pre-Lobotomized Dragon. The damn thing was magical, especially if you have any abilities in writing. As for the 22b model I keep hearing nothing but rumors. Everything from Eleuther just waiting on GPUs to them not having any money at all for it, to it actually being in training. It's all wind in the trees with nothing substantial and their own blog is silent on the subject. 22b still won't be dragon, but with all the tools NAI gives us it should be a night and day difference to Sigurd.

If it ever comes out.

 

[Non-Edgy Gamer]

And the walkback is finally here:

https://latitude.io/blog/the-walls-approach

What this means for AI Dungeon
So what does this mean for AI Dungeon? Well, for starters, it means we will not be doing any moderation of unpublished single-player content. This means we won’t have any flags, suspensions, or bans for anything users do in single-player play. We will have technological barriers that will seek to prevent the AI from generating content we aren’t okay with it creating — but there won’t be consequences for users and no humans will review those users’ content if those walls are hit. We’re also encrypting user stories to add additional security (see below for more details).

Essentially, users can do or say what they want in single-player play, but the AI may sometimes decline to generate certain types of content.

Additionally, those barriers will only target a minimal number of content categories that we are concerned about — the current main one being content that promotes or glorifies the sexual exploitation of children.

As part of this change, we’re releasing new community guidelines that will replace the current content policy and current community guidelines. These community guidelines lay out what is allowed in interactions with users and content published in the Latitude community, but they do not apply to unpublished content since that will not be moderated.

We’re grateful to all our players who have come along with us on this journey to make a new evolution of games. We love seeing the amazing things you create and share along the way. There have certainly been bumps on the road, and we apologize for the frustration that these have caused our users. We are committed to fixing those issues along the way and finding the right path forward on this extraordinary journey we’re on together.

Sincerely,

Nick Walton
CEO

Questions
What kind of technological barriers will you have and how will you enforce them?

Every AI output along with the context will go through a filter that prevents it from generating types of content we aren’t okay with it generating. In most cases since we generate several possible options at once, there will be at least one generation that can be delivered to the user.

In some cases the AI may not be able to think of a response that passes the filter in which case the user will get a message letting them know and giving them ways they can continue their story and how they can report if they think the AI should have been able to generate a response that would be appropriate so we can improve the system.

We are currently working on building a new classifier that aligns with our intention to prevent the AI from generating stories that promote or glorify sexual exploitation of children.

How does story encryption work?

All stories in the Latitude database are now encrypted. They are decrypted and sent to users’ devices when requested. Because the AI must take plain text to be able to generate a response they are also decrypted before being sent to the AI to generate a new response.

What if unpublished content goes against third party policies?

If third party providers that we leverage have different content policies than Latitude’s established technological barriers then if a specific request doesn’t meet those policies that specific request will be routed to a Latitude model instead.

TLDR: they're giving up on most of their policies and encrypting user content so that they won't know what you're doing (or so they say). OAI will still refuse to generate some of your content though. The filters are still there, you just won't be banned for hitting them.

Also, it turns out that the Taskup leaks were indeed OpenAI's doing:

Which makes sense as they only included the data sent directly to the AI and not full stories. I'll also add that this goes a long way to proving that it wasn't their idea, just a bad attempt at finding a solution to a problem OAI imposed on them.

10 months of ruining their company and this is what they've done in the end. It's about 9 months too late for me though.

 

[Tweed]

Yeah no, I'll stick with NAI even on 6b. We'll get something bigger sooner or later and it'll only get better. I much prefer no-filters, modules, and the hope that nothing I write is being leaked at the behest of a retarded company.

 

[Atrachasis]

Just out of curiosity: Would anyone be able to provide a more in-depth explanation or links thereto as to how exactly OpenAI went about implementing their filters? Did they create a sort of watchdog (rule based or ML based) that classifies and censors the output of their principal model, or did they attempt to fine-tune their model so as to prevent any undesirable output in the first place? How excatly does one lobotomize an AI?

Currently curious about uses for transformer models in safety-critical applications.

 

[Non-Edgy Gamer]

Just out of curiosity: Would anyone be able to provide a more in-depth explanation or links thereto as to how exactly OpenAI went about implementing their filters?

Since OpenAI never made it public, we can only guess based on available info, but I'll say what I know (though I'm no expert):

We know that they did hire out to companies in order to sort all the porn people were making flagging the content based on a set of parameters. This was an attempt to cut out the less-than-legal (in some countries) content, since all porn content can simply be blocked already on OAI's end, and the guy working on it said that he was hired to curate data to create a model - though he was being paid pennies and may not have really known.

Did they create a sort of watchdog (rule based or ML based) that classifies and censors the output of their principal model, or did they attempt to fine-tune their model so as to prevent any undesirable output in the first place?

At first, Latitude (not OpenAI) made a simple algorithm that just checked if certain keywords were within 400 characters of each other in a given submission. They also had developed a sort of watchdog program called the judgebot before this, which could potentially do what you're talking about, but I don't think it ever evolved beyond a novelty.

The simple algorithm Latitude designed was what first broke the game. Later, OpenAI apparently lobotomized it on their end as well, and that's why people just got a lot of yellow text error messages, even if they didn't hit Latitude's blue text filter message.

Just so you understand the timeline.

Beyond that is the part we don't really know about. I don't think it's a finetune, since that wouldn't kick back an error, it'd just be bad at generating the desired response.

It may function like their content filter though:
https://beta.openai.com/docs/engines/content-filter

The filter aims to detect generated text that could be sensitive or unsafe coming from the API. It's currently in beta mode and has three ways of classifying text- as safe, sensitive, or unsafe. The filter will make mistakes and we have currently built it to err on the side of caution, thus, resulting in higher false positives.

The filter currently has a harder time parsing prompts with strange formatting. Thus, if a prompt has a lot of linebreaks, unusual format, repeated words etc. then the model might misclassify it with a higher frequency. It also has lower performance on certain kinds of texts such as fiction, poetry, code etc.

tldr: Pretty clear that this filter is a watchdog model of some sort, in that it scans the output and classifies it, so maybe the AID lobotmizer is too.

 

[GrafvonMoltke]

So which is the best one for smut?

 

[Tweed]

Probably NAI since AID is filtered to hell and back. So you can write that horse smut you're jonesing for.

 

[rusty_shackleford]

There's nothing to be gained from censoring a private story no matter what content is in it.
"B-b-but think of the imaginary children!"
no. Censorship always starts with the thing most people dislike and goes on from there, we've already seen this play out with AIDungeon.

AIDungeon cucking themselves to OpenAI means I'll never touch their service. They've had how long to work on their own model?

 

[GrafvonMoltke]

Probably NAI since AID is filtered to hell and back. So you can write that horse smut you're jonesing for.

And how does one procure this NAI without payment?

 

[Tweed]

And how does one procure this NAI without payment?

You don't. If you want free you have to set up the KoboldAI collab.

 

[Non-Edgy Gamer]

Probably NAI since AID is filtered to hell and back. So you can write that horse smut you're jonesing for.

And how does one procure this NAI without payment?

You can sample HoloAI for free if you're hesitant. NAI is similar, but better in terms of quality imo. Holo is the only one if the two with a free trial at this point though.

And how does one procure this NAI without payment?

You don't. If you want free you have to set up the KoboldAI collab.

You'd still end up paying Google for that, both for storage and the processing power needed to run Kobold well. About $15 in total. Might as well get NAI.