Also, bear in mind that the link between INT and wizardry comes from Vancian magic - i.e. from the idea that you need high intelligence to be able to bind and outwit the entities that do the actual magic, to do research, and to keep the brain-bending spells in your memory till discharged.
Without those ideas, magic doesn't have any particular link to intelligence and it could just as easily be a natural ability (as with Gandalf, being an Aesir or whatever the term is, an entity naturally able to do magic) that a jock could have just as easily as a nerd.
Not that I'm particularly defending the idaea of the "muscle wizard", since I think that's a canard (the situation is as I posted earlier, Might was supposed to represent a generalized "puissance" that could be mental or physical) - but even the "muscle wizard" isn't that weird of an idea in and of itself, the only problem is that it would be OP (if you had both high martial ability AND high magic ability).