IMO you've hit the nail on the head.

These conflicts and confusions are the consequences of gendered leagues, but because the issues come up in the context of transgender people, it makes it seem like an issue with transgender people.

Buuut then there is the issue of the representation of women in (most) popular professional sports is much lower, and would be worse without this delineation along gender lines.

So what we need is to make up our mind if we want either:

All sports to be as fair at all times as physically possible

Or

If we want to see all genders fairly represented in popular professional sports

If it is the first, we should ungender the leagues. If it is the second, we should stop worrying about it and let transgender people compete wherever they want.