I was on the same track when I read this - Morrowind was the first one that came to mind for me. It just had so much in it. Don't get me wrong, I've enjoyed oblivion and skyrim, but there's something special about Morrowind to me. Never tried daggerfall, your post inspires me to try to find it again :)
For me, I think the nostalgia of the first iteration is part of what holds me. Halo 1, for example, is my fav of the Halo games as a whole. Sure, I enjoyed many of the others, but there's just something about that first game and the experiences I had in it that have been hard to top.