So I wanted to follow up on this with what I did, and think I found the solution. This way if someone like me trawls the internet looking for something that is the same problem, they might have resolution.
Basically I isolated the problem to podman. What it looks like is I had an existing podman directory pre-3.5.1 update and it had issues with distrobox. So what I did;
Went to .local\podman - deleted the contents of that folder.
Then followed the emudeck distrobox guide to reinstall podman to .local\bin\
Made sure to rename the podman launcher to simply “podman”.
I also edited the .bashrc in home\deck which looks like it had leftovers from the previous podman to have the following (per guide)
#Uncomment the xhost line below if you know that you are using xhost #xhost +si:localuser:$USER export PATH=$HOME/.local/bin:$PATH
I only re-installed podman, and left distrobox alone. You may also have to use the following commands, as I did;
sudo touch /etc/subuid /etc/subgid
sudo usermod --add-subuid 100000-165535 --add-subgid 100000-165535 $USER
For reference, this is the guide - https://emudeck.github.io/community-creations/steamos/third-party-emulation/?h=distrobox#how-to-set-up-distrobox
Hope this helps some stranger out there figure things out