Upon further testing, it turns out you actually have to pick the "lie" option as the first thing you say. Can't bullshit the guard if you say "just passing by" first, for whatever reason.
Still shitty that aggressive players get punished that hard. From what I've seen online it also left a very bad taste on a lot of people. Seen a fair number of them complaining about thw dungeon crawl at the end. Oh well. What can you do.
You know, the game did suggest you multiple times (almost to the point of screaming in your face) that there were other routes beyond the full frontal attack way. The NPCs told you how to go in without fighting (pretending to be someone else) and that there are also other, more stealthy routes that would let you avoid a lot of battles.
If you didn't listen to any of it and fucked it up you can't protest that there are too many fighters when you are storming the hidden stronghold of a god, whose members value their secrets above anything else and have spent centuries protecting them from strangers.
In fact, the Abbey is the closest dungeon to Readric's Hold, which was rightfully praised by pretty much everyone because it allowed multiple pathways.