Of course systems matter, they factor heavily into moment-to-moment gameplay and flow of the game, especially in a combat heavy one. They're not some unimportant elements of the game you can abstract.
This is how I look at things re: PoE's system and systems in general.
IMO, PoE's system can be described as essentially a 3E/4E D&D hybrid plus Sawyer tweaks.
First, I think PoE's solid foundations in RPG design have gone under-recognized. People look at it and see a horrendous mutilation of the AD&D they remember from Baldur's Gate, rather than a game that follows many of the same steps that WotC themselves did in the 2000s. It's easy to see why - we never got a proper tactical RTwP 3rd or 4th Edition D&D game in the 2000s (IWD2 was an easily ignored, partially implemented black sheep of a game, and NWN2's combat was a joke). There's no really good frame of reference for properly evaluating what Sawyer did with his systems design tweaks.
But, more to the point of this discussion, I think systems are moderately important to an RPG's quality. I think they're especially important for replayability. You play the game once, you're mostly thinking about the content. Play it again, now you're looking more at how you can tweak your experience via the systems. But that's looking at the system
as a whole. As I said, PoE's system isn't just Sawyer tweaks. It's that D&D foundation PLUS the tweaks. The tweaks alone, IMO, just don't really matter that much. They're quality-of-life stuff that people have obsessed about to a disproportionate degree.