I fiddle around with podman some more and found that changing these two things will make debugging work with podman. However this will force the container to use the host network and therefore the container will not get its own network space.
Not sure if this is a good way to go about it for a PHP development environment or if there is a better way to achieve this. The problem with docker setup in the original post compared to podman is when using those steps for podman, the debugger cannot connect to the IDE (VSCode) since the podman container network is isolated as its own network.
If anyone else knows of a better way to go about this, please do share
Terminal command to create container
podman container create --name my-container -v ./app/:/usr/src/app/ -v .:/usr/local/etc/php/ -w /usr/src/app/ --network=host -it my-php-image
php.ini
xdebug.client_host = "host.containers.internal"