Rules are fundamentally the arbitrary judge in games like physics are in real life. If I play in a game and a GM "fudges" a roll, it's basically the same as that GM changing the rules to go easy on me or alternative change the rules because I did too well. It's like switching from Hard to Easy because I can't handle it, or vice versa. We agreed that the world works in such-and-such a way, but suddenly the world works differently because it's more convenient? Fudged rolls are to rule-heavy P&P what deus ex machina are to literature.
A GM "fudging" a roll is like an invisible hand or awesome button gently guiding me through a video game.
Rules are the one thing in P&P that needs 100% clarity and adherence from both players and GM, otherwise what's the point in simulating the experience at all? You can change the rules of course, edit them however you'd like, even play games with no rules at all, if it's just about story and player interaction with that. But if you agree to adhere to a common structure, deviating from it because "it'd make for a better story" or somesuch convenience is the number one destroyer of any immersion for me.
The key aspect of P&P is player interaction with a GM construct. If the rules are not set in stone once agreed upon, then the player interaction is nullified. GM manipulation with rules is negation of player influence.
(If everyone in the group agrees to throw that influence to the wind and prioritize convenience over the simulation, then fair enough - you should do whatever works for you. This does not change the objective fact that manipulation is negation, though, and you need to be aware of that. For my part I would not enjoy playing in such a game.)