can an assassin install kill a mage
only if he duals into the installation wizard
Gibbing mages is very viable, and the assassin's poison (which you can enchant ranged weaponry with) passes through Stoneskin for further mage debiliation. Even if you're playing a solo Thief and are somehow disadvantaged in the anti-magic department, a lot of the time you can just wait for already cast buffs to disappear while invisible in a corner (Rayic Gethras can be done that way, IIRC). When you get to the hardcore mages, you will have UAI and dispel spells from scrolls, Staff of the Magi backstabs for dispel on-hit, and more. Should be doable, though generally backstab-heavy playstyles are a labor of love; the whole trickery aspect of the playthrough is awesome for your own personal fantasy, but some fights can get a little tedious. Though I recall having a lot of fun with nuking Sarevok in BG1 with two super-powered Staff of Striking backstabs (it was vanilla Shar-Teel, an ex-Fighter/Thief, however).
Though frankly, a 10 level ex-Assassin/Fighter doesn't really offer that much beyond his poison and -1 THAC0 / +1 Damage rolls, because his backstab modifier is stuck at x4 -- dual at 13 instead to bump it up to x5. Might as well go Bounty Hunter/Fighter in that case (dual at 11th to get a new trap) if you want to preserve some stabby flavor while having a semblance of fun toys (still you ain't getting Maze trap, which, again, is locked at 21). Or a Stalker, since a dual-class AssFighter ain't getting UAI anyway... unless you dual at level 21. The only other Rogue that is worth dualling early than late is Swashbuckler, and a Swashbuckler (11) / Fighter can be an alright glass cannon DPS (pick up some Swashie utility and bonus shit), but then you ain't stabbing much.
Ex-Rogues are generally better with Mage, I'd say, unless you're ready to dual really, really late; Assassin levels give you more HP, THAC0, and let you hurl poisoned Melf's Minute Meteors at 5 APR (10 APR with IH), each with bonus +1 damage, regardless of how late you dual, and you always get the Mislead M/T backstab cheese. 17th level for an Assassin/Fighter, at the earliest, if you want an x6 backstab. Unless you're happy with just Poison, but that, again, seems to go against the flavor you're going for (heavy stabs). For going full on TF2 Spy, I'd just go for a multi F/T or plain Assassin for party play. For solo, Assassin (24) / Fighter or Cleric is quite potent.
Item Upgrade features a dagger called Quietus which increases backstab modifier by +1x, but whether you want IU is, of course, very debatable (only if you can leverage the increase in your power level with other mods).
Also, if you pump Detect Illusion a lot before dual, at high enough levels you essentially can use True Sight spell. Locks can be Knocked open or 25 STR bashed in, and Traps can be detected. I'm just saying this because the Assassin has to make a lot of hard choices with his meager skillpoints. Luckily there's plenty of Potions of Thieving Mastery, but for ease of use I'd just travel with someone like Imoen or Jan - dual rogues make for fun parties, too.
tl;dr because this fucking post took too long: Assassin (10) is only a good place to dual at for a bonus Poison charge but it might be better to dual at 13 for the stab; Assassin/Mage is IMHO better for lower level dualing; you have to figure out if you want to focus on poison or stabs in your playstyle to make a good assesment.