This is a good discussion. If you're curious about where we at Obsidian stand on this issue, I can shed some light. (Although I'm just speaking for myself here.)
- Concise is only the undisputed goal of writing to high school English teachers. (No offense, teachers! Those kids have to learn self-editing.) Many of what are widely considered the greatest works of literature of all time are so dense with words that they are unreadable to much if not most of the population. One of the hallmark traits of literature as a genre is that the authors commonly push the form in a new direction with a distinctive prose style. Hemingway did this with conciseness. Joyce is more at the other end of the spectrum with his more adventurous works. Both are considered masters. "Good writing" is completely independent of verbosity. (And frankly I never loved Hemingway, because concise, taken too far, becomes dull.)
- That said, we want our games to be readable. We are not looking to enter the literary canon by displaying our comma splice or stream-of-consciousness prowess. Concise isn't always the answer, but I think "clean" is maybe a better target. Does the prose have a nice flow to it? Does it avoid redundancy with its descriptive words? Are we breaking it up enough with dialogue that it's not wall after wall of text?
- We often were not clean in the Pillars base game. There are a ton of reasons for that. Rust, experience level (maybe 10 different people contributed writing, all with varying levels of experience), a lack of time to edit, a new setting that demanded a large amount of exposition, and a plot that exacerbated the problem by requiring that a lot of it be front-loaded. I cringe at plenty of my own stuff when I look back at it.
- I do believe we're getting better at writing cleanly, and hopefully those who've played White March would agree that we're trending in the right direction. (I'd be curious to hear your impressions, though.) Some of the above root issues still persisted, so we weren't perfect, but I personally felt in playing the game that it was quite a lot easier to read through it without getting fatigued or rolling your eyes at overwrought prose.
- There are certain unfortunate requirements of the form that handcuff you as a writer. I don't think many people are aware of them until they actually try to write RPG dialogue for themselves.
One of the biggest is that branching dialogue encourages NPC monologuing as a device to mitigate the amount of branching that you do. If every time the player has an opportunity to respond, we have to give them 3+ things to say, the more the NPC can say between those player responses, the less enormous your dialogue file is. (There are other measures equally as controversial - having more inconsequential lines that all funnel to the same NPC response, or giving the player less opportunity to make choices of what to say, forcing them to say some lines from time to time.) Writing an RPG dialogue is a balancing act of trying to find the structure that will make the player hate you the least that you will also be able to finish on schedule. But you will see NPC monologuing dating all the way back to BG and PS:T, and that's why. It's unnatural and it reads kind of silly. It's not how I would write dialogue in a book or screenplay. You'll note the dialogue in our short stories reads quite a lot differently than our game dialogue.
The other major issue is that we have to cater to a player base with a broad range of attention spans. Some read everything, but many skim and miss stuff. Sometimes that stuff is very important. Unfortunately, as the goal is for everyone to understand what's going on, sometimes important information has to be restated several times or in several different places, or else people will miss it or fail to understand it. Maerwald was a victim of this. (He was also overloaded with exposition, which was a separate problem. Don't give crazy characters exposition, kids.) I did a first pass of Maerwald, and testers were not understanding what his deal was. So I dumbed it down, and still the same problem. By the time people understood his story, the dialogue had become a slog. The testers were bright people that were doing their job properly, so that's not to lay blame on them. Just making the point that what's redundant to one player will often be a minimum requirement for another to follow what's going on.
- Going forward, the choice of how much text we use will continue to be defined per-project, I think. With Pillars, if we were to do a sequel, I don't think the goal would be less text, per se, but we would want to be more economical and readable, hopefully with more time carved out for a real editing pass.