There are many reasons why Oblivion is bad, but the number one thing I hate the game for is destroying The Elder Scrolls as it had been before. Daggerfall and Morrowind were very different games, but they shared their basic design principles, and they both provided a good roleplaying experience in an interesting world with cool lore.
Oblivion utterly destroyed that.
All the terrible gameplay and balance issues, such as horrid level scaling, can be fixed with mods, and I enjoy modded Oblivion as a decent-tier action RPG. Heck, Oblivion even gave birth to the total conversion mod Nehrim, which is legitimately good.
But it absolutely shat on EVERYTHING The Elder Scrolls stood for, and what's even worse, it introduced horrible decline features into the action RPG genre as a whole, features which crawled into non-RPG genres like the open world shooter, too. I'm not 100% sure on the timeline of it all, but I think Oblivian started some of these trends.
Let's begin with how it ruined the systems and gameplay of Elder Scrolls games:
1. Daggerfall and Morrowind had incredibly detailed and varied equipment systems. In Morrowind, you could wear: boots, pants, skirt, greaves, cuirass, shirt, belt, robe, left gauntlet, right gauntlet, helmet, left pauldron, right pauldron, amulet, two rings. That's 16 equipment slots. You can wear clothes together with armor, and you can enchant every piece of equipment there is. Daggerfall also had a huge variety of equipment slots, allowing you to wear clothes and armor together (and up to this point in history, I think Daggerfall is the only RPG that lets you play a fashion criminal who wears socks with sandals!), and the only difference to Morrowind was that gauntlets came in pairs - so Morrowind expanded on Daggerfall's equipment system, giving you even more slots to mix and match. This was extremely conductive towards exploration, since getting the best armor in the game wasn't just a matter of finding one piece - you had to find multiple pieces before your armor was complete. You could play a heavily armored warrior wearing a daedric cuirass, greaves, boots, gauntlets, left pauldron... but you're still missing the right pauldron! And then, finally, in some dungeon you find it at last! It's a great sense of achievement to finally complete a piecemeal armor set, and the fact that you have to hunt down all the seperate pieces motivates you to explore.
Oblivion removed that. Not only have the amount of equipment slots been drastically reduced, you now can't wear armor and clothes together anymore - it's only either, or. This reduces equipment slots from Morrowind's 16 to torso, legs, feet, hands, head, rings, amulet - which is 8, only HALF the amount you could equip in Morrowind. Of course, enchanting also got dumbed down so you can no longer create your perfect magical enchanted apparel that gave you permanent bonuses to everything. You could experiment less. You could play around less. And of course, there was less to discover as a consequence. In Morrowind, when you're already wearing enchanted greaves and enchanted pants, you'd still be happy about finding an enchanted skirt! In Oblivion, you'd go "meh" because equipping the skirt would automatically un-equip your greaves. Fucking decline. They went from the coolest equipment system in RPGs to something horribly crippled and dumbed down. It's the one feature I miss the most in post-Morrowind Elder Scrolls games.
2. The quest and location compass. Oh boy, this is probably THE feature that ruined exploration-focused open world games the most. I may be wrong, but I think this was the first game to introduce this feature (GTA may have been earlier, but in GTA it made sense because GPS is a thing that really exists in our world... in a fantasy world, however?), and it fucking sucks. I replayed Morrowind only a few weeks ago, with the Graphics Extender mod that extremely expands the view distance. Exploration consists of spotting a thing in the distance, walking there, stumbling upon other cool things while you walk there, and you can actually get lost in Morrowind. You can end up in some wilderness area surrounded by mountains and there are no roads nearby and even if you check your compass and look at the map, you're not sure how to get back to the nearest city. And, heck, cities only appear on your map once you've visited them! You have to explore the world on your own, and the game allows you to get lost. Nothing points you to the nearest dungeon. Some of the coolest places in the game are, in fact, quite out of the way and you might never find them at all. You might miss them completely - unless you explore and keep your eyes on your surroundings, and go off the beaten path once in a while. One of the best cuirasses in the game is hidden in an underwater cave at the very edge of the map. Nothing points you to this location. Even some quest locations require some effort from the player to be found. The Ashlander camp you have to visit might get marked on your map, but you have to find your own way there. There is no compass marker to follow. Many dungeons you have to visit in the main quest don't even have a map marker, you're just told to "go west across the river, turn right after the bridge, walk up the mountain, the entrance is there". The Imperial Cult faction even has an entire questline that consists of giving you cryptic hints about the locations of artifacts you have to find. Their very concept would be ruined by the existence of quest markers on the compass.
In Oblivion, you're not allowed to get lost. You're not even allowed to explore on your own, to be surprised by a dungeon door you just spotted in the mountain side but didn't expect to be there, to wander around and stumble upon a cool place you didn't know was there. You have markers on your compass that always show you the nearby dungeons, and 99% of the quests in the game has a marker you can follow, always pointing you to the location of your quest goal no matter where you are. You don't even get descriptions like you got in Morrowind. Nobody tells you how to get to your quest location. All you get is that fucking marker to follow. Exploration is, therefore, essentially nonexistent. You don't get to explore anything. You only follow those stupid fucking markers.
3. The level scaling, and especially the unrewarding, shit-tier loot. Others have already talked about how terrible it is, and luckily mods fix this, but in vanilla Oblivion it was just utterly, extremely shit. Worse than ending up with level scaled bandits wearing glass and daedric armor was the fact that every dungeon, no matter how cool it looked, contained only random, level-scaled loot. Morrowind was fun because you could find a unique artifact while delving into a dungeon, and it felt like exploring these places was worth it. If not an artifact, at least you'd find something expensive at the end, like a diamond or an emerald. In Oblivion, you'd open the chest at the end of a dungeon and find... 10 gold and a wooden fucking spoon. Fuck this. There's no point to going into a dungeon when you know for certain that you're not going to find anything worthwhile in it unless there is a quest connected to it. The only cool items you could get were quest rewards. Randomly exploring never gave you anything interesting.
But more than just the gameplay, Oblivion utterly destroyed the extremely cool, exotic and unique lore of the Elder Scrolls world. Daggerfall started the cool lore by having interesting political relations between all the factions, and having all those cool daedric lords. Battlespire really made the daedra much cooler than they already were. Redguard added massive amounts of lore about the Dwemer. And Morrowind had one of the most visually interesting game-worlds ever created in an RPG. You have the barren ashlands, the dangerous swamps of the west, mushroom trees growing everywhere, crazy wizards building their fucking houses from giant mushrooms, huge insects serving as the main means of transport... it's just such a cool place to explore.
There were also books, both in-game and out of game (as feelies in the box), that described the other provinces of Tamriel. The Imperial Province, Cyrodiil, was described in some detail in Redguard's supplemental "Pocket Guide to the Empire". It's a province with political intrigue like we've seen in Daggerfall, it's a province with cool visuals like Morrowind was, with the Imperial City being a pseduo-Venice with canals going through it, where the main way of moving from place to place is by boat. Large areas of the province were supposed to be a tropical jungle. It was a cool place, and whenever I read about the province as it was described in pre-Oblivion lore, I really want to explore the place because it sounds so interesting.
But Oblivion didn't transport the player into that place. Instead, it just gave us the most generic fantasy world ever created. Everything about Oblivion's visual design was generic. There was nothing fantastical about it as there was in Morrowind or its expansions - heck, even the Bloodmoon expansion with its nordic forests looked unique and interesting thanks to having a giant fortress made of ice and a gigantic glacier in its northernmost area. You don't have anything like that in Oblivion, no "holy shit this looks awesome!" moments as you explore the landscape. Everything looks just like the forest next to your town, and you could see the same kinds of landscapes by driving around with your car for half an hour. By the time of Morrowind, The Elder Scrolls had gone through almost a decade of intricate lore writing and cool design, and the result was an extremely interesting and exotic fantasy world that really beckoned you to explore it. Oblivion killed that. Oblivion retconned a cool province into the most generic fantasyland ever made, and everything is banal and boring - there isn't even the basic political tension between lords that we've seen in Daggerfall. Even that was taken out.
Oblivion killed one of my favourite fantasy RPG series of all time. I still wonder what Oblivion and Skyrim could have been like if Bethesda had stayed the same company it had been when it made Morrowind.
If I ever become a billionaire and for some reason Bethesda's IPs fall into my hands, the first thing I'd do is create a new "Elder Scrolls 4", but in the way it's meant to be created. With Cyrodiil being what it was described as in the pre-Oblivion lore. An actual cool place that invites you to explore it, rather than bland and boring genericland.
tl;dr: Oblivion killed my favourite fantasy RPG series. That makes it a terrible game because the previous games in the series showed a potential that Oblivion snuffed out while it was at its highest. Fuck Oblivion.