Except you still didn't explain how will you scale physical spell damage when access to that is worse than access to elemental or cold damage nodes. Especially when there are awesome scepters and item mods for them that will give lots of elemental or cold damage boosts and same don't exist for physical damage except as local modifiers to base weapon damage.
I had the same problem when coming up with how to further boost Bladefall damage by using Physical Damage boost and could not find good solutions in Breach league.
Every damage interaction is either buffed or unchanged. It simply cannot be a nerf. Impossible.
Physical damage increases will now affect GCs damage with 100% of its value, instead of 50%.
Elemental/cold damage increases will now affect GCs damage with 100% of its value, instead of 50%.
Spell/AoE damage increases already fully affected GC, and will continue to do so.
Damage conversion effects (hatred, added fire support, phys to lightning, cold to fire etc.) will also have their effect on GCs damage doubled, since all of them previously just affected 50% of the damage.
Damage penetration now effects 100% of GCs damage, instead of 50% (unless you managed to convert 100% of it to fire previously, in which case nothing changes).
Edit: All of this considers GC to have 100% conversion to cold, which is manageable by either picking up Winter Spirit or having the uber-lab helmet enchant. Even without a full conversion it is still better than before.