What exactly you don't like about XCOM cover (not saying it's great, just want to understand your concerns better) and what is the best way to implement cover in your opinion?
NuXcom's implementation of cover is atrocious. It's a popamole shooter from a top-down perspective. Hidding behind a park bench or a dumpster somehow protects your guys against PLASMA FIRE!? Lol. Xenonauts provides a decent implementation of cover IMO (at least with mods it does). I remember the color coding: red for solid cover, green for visual and yellow was smt in between.
Implementation ideas:
It's absolutely imperative to differentiate between visual and solid cover. Hidding behind a car only makes you harder to spot, but your enemies should have the option to just fire at the cover and hit whatever is behind. Decent FPS games like Insurgency do this really well. If I see a guy running up to a car and taking cover I'm just gonna empty a clip on the car and I KNOW the other guy will drop dead on the other side. HOWEVER if I DON'T see him running to the car and he shoots at me I'll have a much harder time "zeroing in" his position. So in order for this to work you would need a "likely position" system like in Silent Storm. Eg: I had a visual on the enemy during his turn, saw him taking cover behind some piece visual cover, I should get a "ghost siluette" during my turn, unless he fires at me from his cover, thus giving away his current position.
In the end, visual cover would work hand-in-hand with LoS and stealth mechanics. If you get to a good position without being spotted AND you use a silenced + muzzle break weapon then you can score several kills before the enemy even realizes wtf is going on. HOWEVER, if you go loud and you don't care whether or not you get spotted/heard, then visual cover is almost as bad as standing in the open. Just adding a smaller or a bigger penalty to enemy THC (like nuXcom half/full cover system), IMHO, doesn't cut it.
Also, being able to spread bullets around like in JA 2 1.13 would be REALLY cool. That and suppression is what brings the JOY of using a machinegun in an RPG to life. Machineguns in Wasteland 2 felt like garbage because they were essentially rifles/smgs with different stats.
Magical buff smoke, most likely. We don't have time or resources to do anything ground-breaking when it comes to smoke and LoS but we can do a defensive AoE "spell" that lasts 2 turns and makes you harder to see and thus harder to target.
Xenonauts has a pretty decent smoke system. My problem with magical buff smoke is that it has absolutely
nothing to do with how smoke is used IRL. You toss smoke towards wherever you think the enemy might be hiding in order to block HIS vision, not your own. Standing inside a smoke screen makes no sense, since you're blinding yourself as well. I wish I was good at drawing, but you'll have to use your imagination: suppose your party reaches a fork in a long corridor. To the left is where they wanna go, but they suspect enemies might be hidding to the right, laying in ambush. The proper place to toss smoke is not in the middle or in the left, but to the right, thus blocking the shots of would-be-ambushers while STILL allowing the party to take a clear shot at whoever comes from the left path. If you have "magical buff smoke", then the proper place to toss the smoke would be in the middle, putting the party in magical smoke while they shoot at both sides (and not get blinded by the smoke for god knows what reason
).