I've never finished the main story in any of the elder scrolls games I've ever played. And I imagine I came into the series earlier than most of you.
In the 90s I played Dagger Fall, then...
morrowind
oblivion
skyrim
I think I enjoyed skyrim the most, even those it had fewer options than each preceding elder scrolls game. You want to talk about dumbing a series down, there were attributes and options in daggerfall that it makes skyrim look like a mobile game. You could increase athleticism and jumping to the point where you could leap into the air like superman, you could crawl up walls, you used to be able to craft a spell that allowed you to absorb mana, and other AoE spells where when you fired them at the ground, you did both damage AND restored your mana due to the AoE. But even though skyrim had less of the "freedoms" of earlier installments, it came across as more fleshed out and engrossing than its predecessors.
But even then, I've always found games with more narrative to be more engrossing, like the baldurs gate 2, and kotor I/II, Dragon age origins.
I've finished every one of those games, and enjoyed them even more. Because at their core, they kind of resemble an interactive novel, and the elder scrolls model cannot replicate that without crafting more narrative to drive things forward.
Now it may be that most people are less interested in that, games like skyrim and GTA are quite popular, but then so are shooters. But stop trying to pretend that the ever increasing freedom has the highest appeal. I had far less freedom to play around in baldurs gate 2 than any of the elder scrolls games, but that game was far more engrossing and impactful, because was not doing the equivalent of doodling with my character, I was on a ride where I controlled some of the outcomes and fought through some of the challenges, but also had a strong narrative that DOES NOT EXIST in the same way in any truly open world game I've ever played. They can probably blend it all together, but it must take more time and money.