First time the writers told me that elves of Solasta were turbo-racists that used to their mastery of magic to enslave everyone in the past I also lifted an eyebrow lol.
Its "realistic", and not the only setting doing it, folks with high magic skills will rule kingdoms and enslave others, if they are stronger they will abuse their position of power . It make sense when worldbuilding . Elric of melnibone 's people really are close to elves, whole decadent civilization thriving on slavery . In primeval thule setting elves are slavers too depicted as cruel and decadent.
The problem with that is it's not "realistic." Slavery is highly inefficient as a means of sustaining a leisure class, the only reason it was widespread in the past for that purpose was because
there was nothing better. In the real world, slavery fell out of use as soon as technology and industry improved. Also, free people who are working for themselves (understandably) work harder than slaves, so taxing hard-working people is always going to be more lucrative and sustainable than owning lazy malingerers in the long run.
If one substitutes "high magic" for "technology" one can see the nonsense in the idea.
Also, while it's certainly realistic that a high magic society would use its magic to maintain military dominance, it needn't necessarily be expansionist, it could be insular and only use its military might to ensure control over its own lands, and keep other groups in check (such was the case with the empire we call "China" - once the technologically superior Han had conquered the other local nations, they were done with expansionism, built a wall, and were satisfied with roughly the same territory for a thousand years).
Of course a high magic society might hold slaves if they're "just eebil" but in that case they're probably being used as a Nazi substitute by a lazy writer