I must admit I've never felt I really fully understood the rationale of "touch attacks" in terms of simulation or quasi-realism. So far as I understand it, they're something that requires contiguity from body to body (i.e. a bridging from body to body, either via a ray conceived of as a continuous beam of matter, or via hand touch). Is that right?
But then in terms of quasi-realism, the idea of a touch attack ignoring armor armor only really makes sense in some cases: e.g. heat (enemy is cooked) and/or electrical attacks (enemy is shocked through the armor), or maybe something like a fungal attack that spreads across the armor and gets in the cracks. A diseased touch, for example, should not affect armor at all (it would have to touch flesh). A corrosive touch might weaken armor, but it would still need to be followed up.
Is the abstraction then that, say, with a diseased touch, the toucher is touching the face, or touching flesh through a joint in the armor or something like that? But then that makes it less likely that a diseased or sickening ray would be accurate enough to do that. You can picture someone up close grabbing an exposed part of the body, but a ray would have to be super-accurate to do the same sort of thing. And then there's the problem that if we're taking it-has-to-be-flesh-touch seriously, light armor would grant less protection (less covering, more exposed flesh) in that sense. But also, the close-touching thing would require DEX and speed, whereas aren't touch attacks based on STR? (although I suppose that's conceivable if someone is super-strong, they could just force a touch on flesh, but then why aren't they just a warrior? why piss about with magic?). It's an autistic tangle that's similar to things like DEX being for bows, whereas even with a non-composite longbow you require big STR. It's all a bit unsatisfactory really from the point of view of simulation.
But I suppose it's the same as with all these things, you have to draw the line somewhere, simplify and even gerrymander to some extent, as the reality (mundane or magical-with-rules) would be kind of too complex and "lumpy" (some things way better than others) to be able to make it gamey enough to be fun. You have to have enough of a sense of realism to fool the hindbrain while engaged in the combat in the moment, but not so much realism that you're sidetracked into minutiae and get pulled out of immersion.