Selfhosted

You'll have to use nginx's ACL feature. Like you've discovered once someone has access to the proxy itself, by its very nature it's acting on behalf of the client making the request so firewall rules won't help much there. You'd have to get into packet inspection, and deal with https man in the middle stuff and all that, likely not worth it.

I'm not familiar with nginx proxy manager which you seem to be using but in the config files it'd look like this:

nginx.conf

location / {

    include /etc/nginx/acl_file_name.acl;
    deny all;

   # rest of location config

}

the deny all; at the end means everything that wasn't listed in the ACL file is denied. You could alternatively put the deny all directly in the acl file at the end of the file.

the acl file itself is just a list, i create it in a file for easy re-use, and made up the ".acl" extension because it makes sense.

acl_file_name.acl

allow 192.168.1.0/24;
allow 192.168.2.101;

If they are on the same l2 network then your options are basically either to use nginx ACLs or a local firewall on the nginx host since the traffic wont traverse the firewall.

Something like iptables -I INPUT -s 1.2.3.0/24 -j DROP on the nginx host should work

This is an interesting problem and probably one I will run into when I eventually divide my LAN into multiple vlans.

Unless traffic can be filtered based on hostname then the only solution I can think of is to run two instances of the proxy/ingress controller.