If grimoires were fragile objects, sure. They are not fragile objects. Grimoires are all enchanted capacitors of soul energy that happen to be made in the shape of a large book. Every time a wizard casts a spell, he or she gathers and channels soul energy through the grimoire before releasing it. Even without the wizard using it, a grimoire is a magic item.
If a wizard uses Grimoire Slam, he or she is not just swatting the target with the weight of a large book, but hitting the enemy with a magical capacitor. It happens faster than most spells are cast and, if it hits, can knock the enemy back. In a melee situation, that's why a wizard would use it.
It's also an optional class-restricted Talent, not a core class Ability. If you don't like the flavor of Grimoire Slam, don't buy it. You'll have spells you can cast with more traditional magic effects that could accomplish similar things, but they will require you to cast a spell to do so.
