What's actually holding them back is a very interesting mystery to me.
I think the balance between castle sim and rts is fucking them up.
The very first game had a very deliberate balance between sim elements and combat elements, to the point where the peacefull campaign is genuinly really fun to play.
Crusader has a good degree less sim elements, and the peacefull Crusader campaign is boring by comparison. Very interestingly Stronghold Crusader sold significantly worse than Stronghold I, despite now being remembered as the best Stronghold game very often. So there were definitly core fans which were fine with taking a detour from castle sim elements, but a majority liked them.
Stronghold II sold less than both Crusader and Stronghold I, and it definitly oversteered towards castel sim elements. The peacefull campaign is literally more fun to play than the military one. Quite interesting is that presumably the sales of Stronghold II and Crusader were not that far apart, both a good deal below Crusader.
I have no idea how well Strongold Legends sold, but since they never published the numbers anywhere I assume it did not sell well. Stronghold Legends interestingly plays least like a Stronghold, and the most like a generic rts.
Stronghold III is a clusterfuck and nearly killed the series. I don't think this can be related to the castle sim/rts balance, it is more a consequence of the retarded idea to cut AI opponents.
Stronghold Crusader II feels like a strange mashup of Legends and Crusader I. It introduces very "gamey" units like the armor aura seargents, which were a trait of Stronghold Legends. Also it dials the castle sim aspect a bit from Crusader.
Warlords seem to follow Crusader II's path, and take the game even closer to classical rts, and it seels and reviews mediocrely.
I think of you look at this as a development the issue is as follows:
Stronghold 1 is the best mix of castle sim and rts that the series ever had.
Stronghold Crusader is as high on the RTS / Castle Sim aspect as you can go. A Stronghold should not try to be less castle sim than Crusader.
Stronghold II is too far on the castle sim scale. A Stronghold should try to be less castle sim than Stronghold II.
So to make a good Stronghold again, they need to place the amount of castle sim mechanics between Stronghold and Stronghold II, and not put the amount of rts elements above Crusader of all things.
Then again it is always easier to critique stuff from the outsite than to make it, and I do not really envy Firefly's position of sitting on a prestigious IP without the skill to make it justice.