If the dialogue checks were presented in a way that it was clear that failure was not a Boolean state & you could proceed but with a penalty, I'd treat it much more like a non-optimal combat result, like if I lost health or used consumables - not perfect, but just keep going.
I disagree on this.
Years ago, I wrote an
article about why RPGs should impose consequences on the characters that are less than death but more than trivial. I'm now somewhat skeptical that this is possible; player psychology may simply be too rigid on this point to shift, at least as far as it comes to narrative. My supposition is that RPG players will accept any random penalty
other than (1) closing off access to narrative segment that they want (which would include, e.g., loss of a companion due to permadeath, but also loss of access to a quest, an area, etc.) or (2) a permanent reduction in a character attribute, skill, or level. These penalties can be imposed
non-randomly (e.g., the Pillar of Skulls, where you have to choose one of several hard-to-accept penalties), but not randomly. All of the Goldbox-era draining powers of enemies are no longer feasible with player psychology; they will simply trigger a reload.
Take a dialogue that goes:
"I offer lore that is the start of an interesting side quest."
1. [Persuasion 60%] As a fellow knowledge-seeker, I would love to hear it.
2. [Intimidate 20%] Tell me or die!
3. So what.
Totally worthless. No player will take #2 (as its odds are lower); if his stats are such that the odds are flipped, then he will take #2 rather than #1, of course. And no player will take #3. Furthermore, there is a 40% chance that the player will reload -- if he takes #1 and fails the check.
(In fact, AOD suggests that even if you replace these rolls with threshold checks, players will go berserk about their inability to get this sidequest without foreknowledge of the necessary thresholds.)
If you replace the fail states for 1 and 2 with something stupid like, "Ah, well, I'll tell you anyway -- but for 10 gold" then the whole thing feels pretty dumb. You're right that having this a Biowarean choice with a small tax is less likely to induce rage-quitting. But it also doesn't really accomplish anything meaningful from the standpoint of the player's experience. You should just use thresholds and a bypass option or something.
Put otherwise, it seems to me that in the post-Planescape: Torment RPG community, for narrative RPGs the only acceptable use of dialogue skill checks is (1) to let the player choose
which key opens the door (not whether the door gets open) or (2) to let players skip a few squares of the Candyland board that they otherwise would have to grind through without having passed the skill check. What is
not acceptable is a door that you can't open and what is
definitely not acceptable is a door that you
can open, but only if you roll the right number. The last scenario will just lead to save-scumming or, in TTON's case, a mix of save scumming and Effort.
Imposing a dialogue tax is ultimately no solution because if the tax ever reaches a point where players cannot go through the door they want to go through, they will rage-quit and reload, but if it never inflicts a meaningful loss, then you are just wasting an enormous amount of designer time in fail states that are irrelevant.
The insanity of TTON is that it is sold to players as basically, "This is a story about controlling your destiny and legacy." If the players buy that, then they are never going to want dialogue outcomes that prevent them from (1) hearing the story they're interested in or (2) being able to achieve the destiny/legacy they want. As with PS:T itself (where there was a robust AD&D character sheet but where anyone normal simply maxed dialogue-oriented stats like CHA, WIS, and INT, since anything involving the other stats could be dealt with by grinding or save scumming), the player will sacrifice anything for narrative goodies and narrative control. In other words, you have a game where players are
even less likely to accept dialogue fail states, and then you make dialogue fail states a central feature.
On top of this is the problem you note, which is that once Edge or whatever it was called comes into play, there is a weird thing where you don't really need the stat that ostensibly defines your character.
It's a sad state of affairs. I think in the perfect world, RPG players would accept fail states because they make the world more reactive and interesting. But here, TTON's design achieved the worst of all worlds: all character types can brute force through all dialogue checks. From a narrative standpoint, the key value of dialogue checks is that they are how the game defines the outline of the player character. What the character can and can't do in dialogue is the way the game confirms the existence of the character your made in the context of the game's narrative* (i.e., it is what allows for a player
character without LARPing). TTON's design obliterates the distinctions between character types, while imposing on the designers all of the effort of scripting multiple paths (with fail states!) through every dialogue. Changing the fail states from a vague, seemingly binary narrative outcome to some kind of tax wouldn't solve this problem IMO.
(* Obviously, what you can do in combat, etc. defines your character in other contexts.)
Sounds like you're trying to min/max the system to pass everything but without any actual understanding of it, so of course you're going to fail.
