there are 5,000 people in your town who are as smart as or smarter than you are.
i don't believe you
And you are right. Because his methodology suck and we don't have enough data or information.
We don't know who he thinks ''people'' consist. or where his 100k town is or what this town is. But lets ASSUME things.
1. We assume that by 'smarter than 95%' he mean being smarter than 95% of genus Homo(sure you can say that all of other branches are extinction but looking outside window I am not so sure) and by smart he mean getting high IQ score(but that is big assumption)
2. We assume than 'a town of population of 100k' he mean some idealized ''average'' town that pretty sure don't exist.
Now if you live in 100k town somewhere in Bumpkina Faso which average IQ for a country is around 70 then you being in top 5% of Humanity would probably make you smartest guy around with nobody smartest than you in miles, on the other hand if you land in some 100k research/high tech city like Siliconoid Valley or some old style academic city(outside of USA and before 90' - which mean before diversity hits) or some Soviet style closed city with population consisting of highly educated and smart people then its probable than you actually are in bottom 20%.
But sure in general if population distributions was uniform over the globe, all cities consist of uniform population samples then sure.