No idea what kind of wish.com etymology book you have, but "to each his own" and variations have been a common saying in English (and "Jedem das Seine" in German) since the 1500s, it's a calque of the Latin phrase "suum cuique". And it is still a common saying in English that is not associated with Nazis by normal people. It being plastered on the gates of Buchenwald has absolutely nothing to do with common usage of the phrase.

Even after the German variation was used in Buchenwald, it didn't become very controversial until a neonazi published a book of the same title in the 90s – still, most people speaking German won't think of Nazism if you use the phrase, and it's the motto of several German government organizations (including the Feldjäger/military police, who also have the Latin version on their insignia). Either way, it doesn't affect the English language at all, it is not a "Nazi slogan".