At first I thought you need to replace the spring, contact plate assembly, but then looked at some pictures online and it looks like there is a plate that slides up and down that you broke off as you described. I'd consider sending it in for repair eval or replacing the controller. If you are good with electronics repair then maybe you could find the parts needed, disassemble the controller and perform the replacement.
Edit: lol at me for not understanding this community is a full repost of that sub and it's comments...
Also VGA: Don't forget to unscrew both sides of me at the same time if you ever want to free me again.