Yes. I think I'll go back to the staff and robes even if it makes him very vulnerable. At least, with the staff, he was able to deal melee damages from second line. It's just that I looked at the excellent nerdcommando vid for follower's build and for a while he convinced me to go with sword + shield. But years of Ad&d programmed me that priest using blades just feels wrong.
No wait, I can keep the armor. Heavy armor was never wrong for priest :-p I'm a sheep all right but sheep are responsible for great inventions like plaids, pullover kebab and stuff !
I'm not saying there aren't niche builds (in PoE there always are), but generally in PoE you should think of Priests as "divine mages."
And in PoE, most mages go for Hatchet + Light Shield and light - if any - armor. Don't know why you would use a sword.
All I'm saying is for purposes of optimization on PotD of course. If you're playing on other difficulties, I certainly would advise you
not to optimize like that and pretty much go with whatever you bloody feel like - it's all viable on Hard and below, more or less