I am not a big advocate for respeccing builds, but it quickly becomes obvious late game that magic is going to totally outdo anything a physical class can do and it is frustrating
That's not true. Physical classes have three advantages over magical ones:
1. They act faster. Melee attacks, one way or another, get down to minimal 30 recovery time. With grayface patches that's spear & sword combo. Without them, that's just two handed axes, eventually turning into ax & dagger combo - somewhat worse but still efficient. Spells, on the other hand, range from 80 to 90. Efficient spells, that is - there are lots of duds with 120-140 recovery range. So you can say that for every single cast your fighter is doing three strikes. And, as most good spells are point-blank range, it's not like they have much of a distance advantage.
2. Vampiric weapons are broken, period. They drain 20% of everything that they do and, if you dual them, this one increases to 40%. There's a point of time in each game when even a solo character can hold down attack button in real-time and that's gonna be enough to win the fight as no one will overtake him. Well, solo-melee is not really doable in VI but not due to the lack of damage - unescapeable insta-death really ruins its day.
3. They don't need mana, duh.
The thing is, from my observations, people don't understand how to optimize melee parties. In VI, it goes like this:
1. The party is either Knight-Knight-Sorcerer-Cleric or Knight-Knight-Paladin-Sorcerer. Knight-Archer-Paladin-Sorcerer may work if you don't wish for repeats. Whatever the choice, you dedicate one Light caster as your main engine and dump all the spare skill points into him - all the horseshoes and all the kriegspire well exp.
2. Don't be greedy with teacher-instructor-banker shit in the early game - simple healers are your best friends. You have a large health pool and replenishing it once per day means a lot in the early game.
3. Don't spread your points thin - if your light caster is wizard, it's something like air 7, water 12, dark 10, everything else into light magic. If it's a cleric, it's even simpler - dark 10 and everything into light magic. You wanna reach 30-40 light magic asap. Likewise, your knights should prioritize weapon skills. +1 damage per skill increase may not sound as much, but that'll eventually turn into +30 and that's a bit of a breaking point. For knights, bodybuilding of at least 15 is also adviseable.
4. Remember that artifact rings of magic stack with the common ones multiplicatively. So double-ringed 30 skill Hour of Power is +72 attack and damage for everyone.
5. Once your party is equipped with vampiric duals, Shared Life becomes the best healing spell. Due to bodybuilding, your knights will be overwhelmingly fat. This just shaves some of that off their sides and waxes the entire party.
Here are the examples from my old playthrough: