Nobody knows that you're worth anything more than dirt and, therefore, its only at the end that people start to actually court you. Starting with Ming.
The Tremere primogen personally invites you over at the very beginning of the game, and his riddle even includes the line "Such power I sense in one so young." The leader of the Anarchs personally saves you and then invites you to talk to him as soon as you leave the first hub, and tries to sway you over to his side. The Kuai-Jin agent comments how you were the only vampire worth a damn that he ran into. Even LaCroix, who was likely trying to get rid of you at first, starts using you as his personal agent with regards to everything involving the Ankaran Sarcaphogus. When he wants to open it, the two people he leaves it to are you and Beckett.
Like I said, being LaCroix's lackey in Santa Monica wasn't that bad to me (even though people notice how strong you are back then). But once you start talking with primogen about how terrible LaCroix is? Once the anarchs start courting you and you tell them that you agree with them? After the Giovanni mansion, you have the Ankaran sarcophagus in your possession (which you single-handedly took from the Giovanni), you've learned about LaCroix's deal with the Kuai-Jin, you have a good relationship with the leaders of several powerful anti-LaCroix vampires, and you're quite powerful in your own right? Despite all this, the game forces you to give the thing to LaCroix and to keep doing his bidding before it finally allows you to go after him.
Keep in mind that when you leave the Giovanni mansion, you don't know that the sarcophagus won't open (you haven't yet fought through a fortress of hunters to get the all important piece of information that "it needs a key"). For all you or the other factions know, you could be transporting a sleeping Antedeluvian to LaCroix, one that he'll immediately diablorize on arrival. That's something you'd think someone might care about.
Later you go on to wipe out the Sabbat on your own, wipe out the hunters on your own, and Ming comes up to you telling you how LaCroix had a deal with the Kuai-Jin (which her agents told you about before) and framed Nines (the way Nines was initially framed was stupid too, but that's a separate issue). And again, the game forces you to walk right up to LaCroix and tell him everything. The only thing that happens between the Ming encounter and when the game finally allows you to decide things for yourself are the werewolf section (your character insisting on staying in the middle of a forest fire was stupid, but again, separate issue) and LaCroix framing you for killing Nines (framing you is yet another stupid but separate issue). There's no good explanation for why you couldn't go after LaCroix after Hollowbrook - there's nothing about the werewolf section that makes LaCroix vulnerable or that suddenly would turn the other factions against him. The game just forces you to because it can't think of a better way to get you to the end point.