The richer the colony, the bigger the raids - it even makes sense.
Not really though. It would make sense if there was more to easily steal. But if my colony has 5000 granite blocks and a big cooler full of raw corn, it doesn't make sense that more raiders come to attack it than a colony with two guys with sticks defending a small pile of silver. It's even worse if your wealth is in the form of things like skilled colonists, their equipment, walls, floors, buildings, etc. And even if you're talking about stuff they could steal, it doesn't make sense that the pirates know about my stash of medicine or my awesome dining hall full of marble statues. It'd make more sense if it was related to how much you trade, since you can't really hide the fact your colony traded half a dozen masterwork golden turd statues for a huge pile of slaves and weapons.
Ultimately the dumb thing is that your base is so difficult to defend in the first place. Walls should be basically indestructible, you should be able to shoot through arrow slits that give 95% cover instead of like, 50% cover, and people should go down after taking a gunshot instead of charging through 12 revolver wounds to start firing back point plank from behind the same cover you're using. The only decent advantage you have as a defender are turrets and they fucking explode, assuming they don't get turned off before the raid by bullshit.