In Cadia alone there are slightly under eight thousand Pylons known to the Imperium, and there may be others yet unknown as about one out of four of those known is still partially buried. Even if the Necrons were to sit down and let us try there is no way we are forcing our way out the planet.
Even if we were to force our way out the planet without the light of the Astronomican to guide us and its relative position known we aren't going anywhere other than blindly and stumbling around like idiots.
And about "closing" the Cadian gate Pylons are known to be present in many other worlds near Cadia and around the region so it wouldn't be happening anywhere soon even if we were to somehow manage to destroy all, or most, of the almost eight thousand in Cadia and then jump to each, or at least most, of those other planets, blindly, and repeating it.
So we are going to stay in Cadia for a long while.
Options as to what to do while we are here:
1. Look for the Eldar and ask them, nicely and very politely, to allow us into the webway, or to take us away by means of it, or we just force our way through and hope we do not somehow end stumbling into Commorragh while going through it. Yeah.
2. We pray to one, two, three, or all four the Chaos Gods and hope they can, and want to, offer either guidance, help, or a magical Chaos Gate to take us elsewhere. Anywhere. Fast.
3. We find a Necron ship, find how it operates, then steal it, after killing (several times) everything on board. Or one of those magical Necron gates that link their Tombworlds together, then we fight our way to it, jump in, fight out way out of whatever we find in the other side, and hope it isn't a deathworld. Yeah.
While we are at it, each of those (almost) eight thousand Pylons leads to a massive underground complex that links them all and somehow goes under all of Cadia. While nothing that went down there, regardless of it being the Deathwatch or the smallest and stealthiest servitors available to the Imperium, ever came back, it isn't like we have nothing to do but die horribly. Maybe we should explore them?