Oh, agreed, it very much depends on the interlocutor. I was mainly thinking of an average phone call, but I've had people with whom I used to spend hours upon hours just chatting over the phone or, as you've said, voice chatting during work.
But that, to me, requires a high degree of familiarity, to the point where I don't need to think about what they meant when they said that in that particular way, because I already know their "tropes," so to speak.
You're right, he's a very complex asshole, indeed!