The first game I think of when the word "atmosphere" is mentioned, is Thief: The Dark Project.
In that game it's mostly from the visuals. That the game didn't have state of the art graphics enhances the atmosphere IMO. It's like how the old Star Wars movies feel much more convincing, with a lived in universe, than the clean, soulless, antiseptic CGI universe of the newer movies.
The writing also contributes, even though it's minimal, or maybe because it's minimal, and leave lots of room to fill in with your own imagination. There are no 20 page tomes on the origins and story of the Hammerites and the other factions, or about The City, or even about Garrett himself.
And of course the cutscenes. Like the writing they are on the minimalistic side, but highly effective in setting the mood.
So, if I should rank the factors:
1. Visuals.
2. Writing.
3. Sound (I never have music on in single character games).