The social construct part of Gender is that the terms were constructed by society - they represented the stereotypes of how different sexes would express themselves. It's in modern times that these are more explicitly detached due to our better understanding of Gender and sex.

People perceive themselves based on their Gender as much as other people do, and it is not necessarily true that a person's perception of themselves (their gender) lines up with their sex - hence Gender dysphoria.

It is one thing to accidentally misgender someone because gender affirmation isn't always outwardly obvious (particularly for non-binary folks), but refusing to make that consideration at all, even when aware of it, would say more about you than it would them.