Ultima 9, easily.
Runnerup? Ultima 8.
First disappointing game: Eye of the Beholder 3.
I got into the Ultima series at Underworld, followed shortly by Ultima 7. My mind was blown by both games, they were massive, they were well written, they were loads of fun. Then both games got sequels and I was overjoyed. Then I heard of Ultima 8 being in development and waited eagerly. And it came, and I was... surprised.
It wasn't anything like the other Ultima games, that was certain. None of the "usual" Britannian stuff was around. OK, I could get behind that. Keep in mind that I played most of it on a 386 DX 40Mhz processor. Technically it wasn't supposed to run at such a slow computer, but it did. When no monsters were around the came worked fine, but even a single monster on-screen sent me into 5 FPS-land. Playing the game like this, I got as far as the Sorcerer's Enclave. Then I got a Pentium computer, and read that a patch had been released. Then I had another go at it and finished it. It felt anticlimatic. It was a de-evolution of the predecessors. No party, much of the world interaction was gone, the Avatar had a static... well, avatar, and no weather. But by then (1996) I heard talk of Ultima 9 being in development and my hopes picked up a bit. For 3 years I waited, and when I heard it was close to release, I saved up some money just to buy it.
Let's just say that I understand every single time Spoony says "BETRAYAL!" about this game. I've played it twice since then: Once with the latest patch just so I could finish it and be done with it, and again some years later because someone released a fan patch that cleared up much of the plot inconsistencies and tied the game closer to the prequels (aforementioned patch requires that speech be disabled, which is actually a good thing). But never again.
But EOB3 was the first game to disappoint me. It took a while for me to realize it, but it was an inferior game in almost every aspect. Instead of texts being short and to the point, they were long-winded and dull. Instead of challenging you, enemies merely annoyed you to no end. The game is a step closer to AD&D rules, but not in a good way. Nevertheless, I played it through, but after I micro-analyzed the EOB trilogy back in 2007 I haven't touched them since.
But after Ultima 9, there has not been a single game that has disappointed me. Mainly because I became so jaded and bitter after that experience that I made sure never to get my hopes up again. At least I can be surprised by some games in a good way, like Anachronox, the Gothic games, Psychonauts and Eversion.