Björn said in one of his videos that this is done by design, i.e. they purposely make the early game, and to some extent the mid game, hard as ass on purpose.
If only it could remain hard the whole game.
Agreed, I do also think the end-game difficulty could use some tweaking. But all in all, it does feel very rewarding to become so powerful, that you can one shot enemies that used to rape you. I just really enjoy how Piranha Bytes gives the player the feeling that all of their hard work leveling up and exploring didn't go to waste.
Another similar example, to name a popular game, would be Fallout. In Fallout you will get brutally murdered if you dare take on a Death Claw in the early-game, but in the end-game you can one shot those bastards with a plasma cannon. I just absolutely love that kind of game design, the only difference is that in a PB game, the rats from the starting area would rape you as well.
Still, the difficulty is why the combat is much better than in The Witcher 3. TW3 was quite shit, at the hardest difficulty you could just mash the quick attack button and win everything, no timing or strategy necessary. And that was with a build that put almost no points in sword fighting or fast attack. Quite a decline from TW1-2.
At least with ELEX, you need to pay attention or you get fucked. In the early game, you have to treat enemies as obstacles to bypass, or get an advantage on them to be able to defeat them. You need to time your shit. And you can actually fight enemies harder than level-appropriate ones, you can take on a challenge, and you feel accomplishment for doing it. TW3 prevents this with their almost-invulnerable monsters if they're not level-appropriate (the Skull-rated ones). So in TW3 you either have fights that are too easy, or ones that are almost impossible and frankly quite boring because they become a slog due to their artifically inflated high damage resistance.
TW3 did a bunch of things well, but combat was the worst.
I would like to add that combat in ELEX is also much more interesting simply due to the huge arsenal you have to choose from.
In The Witcher 3 you just have two swords (Witcher Gear) with varying amounts of damage output and the crossbow. Oh, and some bombs and a bit of magic. That just isn't the same as having flame throwers, spear guns, great axes, shot guns, swords, great axes, hunting bows, plasma cannons, grenade launchers, rocket launchers.... to choose from like in ELEX.
CDPR obviously planned to give combat much more depth, by adding more weapon types, but nearly all of it was scrapped. Some of the weapon types are still in the game and can be wielded, but they are completely useless in comparison to Witcher Gear.
Still, I quite like The Witcher 3 despite its faults. The Bloody Baron quest line is possibly the greatest quest ever created in a video game.
I'd take jank ELEX combat over polished shit TW3 combat any day.
I don't like jank, I like smooth animations, just as much as I like nice graphics, but gameplay is KING. That said, graphics fags and animation fags can get bent.