I'll admit I'm in that category.
When really pressed I'll just say something like "I don't go to any church" or "I don't believe in those things". Only in trusted company will I say the words "I am an atheist".
It's easier that way, and people don't need to know anyway. To me, atheism means complete separation from religion, and caring about it enough to tell everybody is akin to practicing a religion. I think one should consider what they gain from using the word atheist, and what harm it might cause, and make a decision based on that. If you really don't care about religion, then there's little need to talk about it.