I'm also a fan of the Bloodlines system. And as Grunker suggests, it can be adapted to open-world games.
One way to do this is to reward exploration and certain achievements with XP, ala Deus Ex. Defeated a powerful type of enemy for the first time or a unique enemy? XP+. Snuck through a heavily guarded bank to reach a vault within? XP+. Talked an enemy into giving up his mission? XP+. And so on. Of course, limitations would have to be put in place to prevent abuses and keep jack-of-all-trades types from becoming overpowered.* Raising skills at higher levels might require more XP than at lower levels, and the XP bonuses would increase for more difficult tasks, discouraging spreading your skills too broadly. A limit on the number of skills you can raise past a certain point could also help mitigate this, as could something akin to cross-class skills where it's costs much harder to raise skills you haven't tagged or which don't jive well with your attributes (e.g., someone with low STR and END would require more XP to raise his combat skills, someone with low INT and CHA would require more XP to raise his persuasive skills, etc.). In the end, it might be easier just to make a raise-via-use system, but it's also difficult to prevent abuses in those and, I'd imagine, doubly so for an open-world game.
* I don't care so much about balance in single-player RPGs, but it is somewhat important in the sense of preventing TES-syndrome, where every decent character eventually gravitates towards a certain archetype. (In TES's case, this is the fighter/mage/thief that everyone eventually becomes unless actively gimping their character and/or LARPing.)
Another option is to limit experience rewards to quests and such, and encourage exploration via rare items, backstory and information that could prove useful like Vree's report in FO, picking up useful techniques, etc. This seems to favor finishing as many quests as possible in order to get the most out of exploring, but this problem could be lessened somewhat if whatever is gleaned from exploration proves necessary/extremely useful for completing the quests. I don't like this method as much, though, as it seems rather arbitrary to reward XP for actions performed during a sidequest but not for the same actions performed outside of one.
In the end, I'm (tentatively) of the view that the Bloodlines system works best in the context of linear games and smaller open-world games like FO, but I'd lean towards learn-by-doing in the case of larger open worlds with many optional areas, like Morrowind's or Gothic 3's. In any case, anything is better than the XP-per-kill system unless the game is a dungeon-crawler or similar type of game where 90+ percent of what the player does is either combat or preparing for combat.
