It's worth noting that the squishy wizard in robes is to a large extent a Dungeons & Dragons thing, from where it trickled down everywhere else. In a whole lot of myth and literature, magic is largely used by the same people who do any sort of heroic or villainous things, and anthropologically speaking, in societies that believed in magic, the people considered to have magic powers tended to be big, powerful and charismatic people - certainly not the guys who were picked last in gym class.
Physically weak wizards are logical if you consider they spent most of their time sniffing glue in libraries and meditating. Having next to no physical activities or learning related skills takes a toll. In RL a person like that can't just grab a sword and start swinging (also trying to punch someone would result in a broken hand).
Well, to begin with that starts from the assumption that magic requires a whole lot of book-learning, which in itself is largely a D&D trope (taken from Jack Vance, admittedly). But even then, historically the kind of monastic and ascetic people that were ascribed with magical powers and
did do a lot of meditating and book-learning often did a whole lot of physical training as well. That's somewhat an Asian stereotype, but even in Europe, monks - for instance - did much physical labour, and actually one of the few existing medieval fencing manuals derives from a German monastery and depicts priests and *gasp*
women in the technical drawings of combat wards.
D&D wizards are appealing from the perspective of character diversity, but practically speaking it makes very little sense for people who regularly participate in combat and need to endure a lot of physical hardships just from traveling by foot around the world adventuring to be physically weak and have no combat skills. It's another thing if either casting spells literally enfeebles spellcasters (which actually makes some degree of sense) or if wizards are so awesomely powerful that they can curbstomp everything without even bothering with any of that nasty fighting, but neither of those things is really true of adventuring low-level D&D wizards. So in the end it's more of a gameplay contrivance. In real life, if I were a wizard running out of spells, I sure as hell would rather have a sword to defend myself with than some pesky dagger, no matter how much that breaks character class conventions.