This sort of vestigial combat takes place in a lot of post-FFVII JRPGs
Wnat do you mean 'post-FFVII jRPG's'? It's not as if most jRPG's before that were hard, or weren't narrative-driven (apart from a couple of the earliest NES jRPG's, I suppose). The formula of these games has largely stayed the same - if anything, it has been somewhat improved through the years.
OK, this is difficult to answer concisely, so wall of text it is. I'll start by saying that I like FFVII - it's a good, if troubled game that admittedly already has a lot of "vestigial" elements, but there's still a proper
game there. The way these things go, though, is that a good game comes along that has a new feature that it somehow adequately integrates into the game, and is seen as novel and becomes popular. The inevitable imitators (which are often its sequels) try to have the same feature and don't do a very good job - but, nonetheless, the feature becomes a convention, even though it's actually a poor fit to the genre as a whole. And so, decline has taken place.
Before FFVII, FFIV was the exact same sort of sinister turning point - it was highly linear, narrative-driven and constricting. On the other hand, it is one of the best paced narrative RPGs ever made, the Active Time Battle system that it pioneered actually worked pretty well, and the game's main design feature - altering your party constantly as the plot moves along - actually did its job, because it kept the battles mostly fresh and allowed the game to saddle you with strange parties consisting of characters with very different abilities and power levels, including completely useless characters that you have to train right from the beginning. It's a game about working around your deficiencies - which is what also managed to make the dungeons non-trivial, since you rarely went in with an optimal party for the job. It's one of the FF games I like the most. But what happened then? Many subsequent JRPGs were mired in the linearity and the emphasis narrative, but didn't manage to imitate the pacing or the interesting party compositions, to the effect that you're that much closer to a game in which combat is a liability.
FFVII did a lot of harm to JRPGs as a whole, particularly because it was a good game and so damn popular that it got a genre-defining status. It was, in parts, even more linear than most games before, to the point that you often can't even return to many past areas - but like FFIV, it did that with such flair and good pacing (especially in Midgar) that it ultimately works in context. It had characters that were, indeed, "coathangers for Materia", and a gimmicky magic system - but, the thing is, the Materia system is probably the best gimmicky JRPG magic system ever - it rewards diligent, patient cultivation of your Materia resources and it allows you to essentially respec your characters on the fly as you gain new, interesting Materia and think of new combinations. FFVII already has barely any attrition, but it has a gradually opening world that is pretty fun to explore, largely due to story but also because finding new Materia is actually kind of exciting, so it wasn't a total loss. The game isn't difficult, but it still encourages mastery of the system and is pretty engaging, is what I'm saying. Also, one of its new features was that it had a ton of weird minigames, which... well, I guess the motorcycle chase game was cool? They weren't major enough to actively hurt the game, though.
The problem is what came after. Already in FFVIII you had a mess of a magic system, a largely broken combat system with awkward level-scaling, basically no worthwhile loot whatsoever, and an optional card game that was actually more strategic and more fun than the actual combat. Not to mention all the cinematic scenes with awkward pacing. Since then, a great many JRPGs were needlessly linear, with pretentious magic systems and awkward subsystems and minigames that only serve to pad the length of the game, and battles that are boring and pointless in context. And somewhere along the line JRPGs became synonomous with 80-hour games that, for the most part, had no business being 80 hours long. The sad thing is that this is what happens when a good game is imitated ineptly.