I did not say that only women or PoC get this suggestion, just that it's common for their issues to be dismissed. I don't know your personal medical history, but sometimes it is just that people need better diet and exercise. That does apply to women and PoC too. It's possible that advice is or is not salient to your health, but I can speak from personal experience that it is used to dismiss life threatening conditions.
I don’t know where you live, but 1/3 of Americans don't have a primary care physician and almost half of Americans didn't get medical treatment due to costs in 2022 from a cursory search. This is not a population that can afford frivolous medical visits. I don't know where in the medical field you worked, but your assertion does not seem evidence based. That may well be your personal experience, but that is subject to so many biases and if you were not giving people a full range of tests, how could you even know you weren't turning away legitimately sick people. Maybe the medical field was not right for you if you truly believe it's possible that most issues people seek treatment for are trivial or psychosomatic.
Then they should potentially be even more likely to be correctly diagnosed in people that are overweight. Having issues exacerbated by your weight does not mean that your weight is the issue. Additionally, PCOS and cancer can both cause weight issues, so it's even less helpful to suggest that the weight is the issue if the weight could be caused by an underlying disease.