Nearly every RPG is a combat RPG, that is, they have a dedicated combat system as the focus of gameplay, while other things are relegated to skill checks. There's no reason you couldn't turn it around, by auto-resolving combat based on a combat skill check, while having, say, a dedicated dialogue system. Or have both, if you have time and resources.
The first obvious obstacle is that social interaction is far more complex and non-linear than combat. The things you can say are limitless and will be have a different effect based on who says it, who hears it and the circumstance. An impossible task at first glance. Yes, there's AI to help with complexity, but any complex system is useless if it can't offer feedback to the player in a way that's logical, consistent and understandable. Otherwise you'd fail an interaction and wouldn't understand why.
There's also consensus in representing combat in terms of spatial relations with quantifiable elements. Characters have HP, strength and resistance; they use weapons with damage modifiers; they interact with other characters at specific "ranges", etc. Dialogue has... charisma? But what does charisma interact with? Is there a stat to represent, say, how much a character is influenced by others? how strong their convictions are? their ability or willingness to understand arguments? We quickly find ourselves in uncharted territory.
There's the problem of personality. Being fit and strong obviously helps with combat, but what's the purpose of being stubborn, persistent, delicate, sensitive, sensual, fearful and so on? Most games have the hero as a blank slate and this aspect is entirely left to roleplaying. This would be unsatisfactory in a real dialogue system. Should these character traits be treated a part of a gradient, as stats with values attached, or as perks?
If such a system is feasible, it must not have too much complexity. Yet it must be able to recreate a consensual model of social interaction with a high degree of consistency. Naturally, you must severely limit the scope of possible interactions. One may look at dating sims for inspiration, or detective games. These are games where characters have defined roles and are interested in one thing only. Yet the ones I've played are just adventure games. You find the thing, use it on something else, open a dialogue option. They don't really have a dialogue system. And in any case, what people are wondering is how such a system would work in game like Fallout.
Someone mentioned Deus EX: HR. Iirc, that was a primitive dialogue system where you could trial and error different approaches, while getting hints of what arguments certain characters were susceptible to. If you were more right than wrong, you would basically win the conversation. That's something like what I'm talking about, but there's no reason you couldn't have a more involved approach with stats and percentages. I may do that in another post, as this already turned into a wall of text which no one will read.
