The main thing that concerns me about a fully cashless society is that the means of buying and selling stuff shifts fully into the hands of the for profit, private company payment processors.
If cash is no longer an option, then ever increasing payment fees can become a growing profit center for those banks, credit card companies and payment processors as they gouge the public worse than they already are.