Because the devs ran out of time/money/ideas/whatever so it ends up either being a rushed copy-paste job or a mess of new elements or mechanics with no build-up or context. It makes sense that the bad guy/group would live in a heavily fortified place, storming it should be climactic and the ultimate test of your abilities and knowledge.
Final dungeons don't suck as a concept and most of them suck because game development with finite resources sucks. They don't have to suck just because most of them do.
I find it to be a recurring problem in those sprawling rpgs that take 2498457 hours to complete. By the time you reach the so called final dungeon you just wanna be done with it, probably the same as the developers.
I think it's both of these.
Let's say as a developer you think the game should have 5 main areas. It seems like what most do is work on the game in a linear fashion, starting at the beginning and working until they reach the end. Then what often happens is the time and budget (and creative energy) start to run out, leading to a rushed and unsatisfying ending.
Imo this can be fixed by prioritizing areas properly.
It still makes sense to do the starter area first, since if it's not fun then players won't be invested and will probably quit the game.
But the next area that gets developed should be the ending, so that the game is brought to a fun and exciting conclusion.
The least important and lowest priority are the 3 middle areas. As time and resources wane, you can thin these out or maybe even cut one entirely, rather than do a half-assed ending.
This approach is admittedly complex and you have to be careful not to run into a situation where everything is out of whack at the end because you were planning on X, Y, and Z in the middle and now those things are cut.
However I think it's something that can be worked through and would effectively solve the problem of bad ending dungeons / bosses / etc.
