Why is it that when Men's Rights advocates are accused of being racist or anti-socialjustice or anything else of that nature, do they reflexively claim to be in favor of affirmative action for people (regardless of gender) of different races?

Speaking in terms of genetics, physiology and evolutionary psychology, there's a much greater difference between men and women than there are between members of different races?

Shouldn't the intellectually honest response be to say "We believe in treating all people equally and with respect", rather than simply focusing on gender issues and the different treatment each group is given, weighed on exactly the same sort of scorecard comprising "White/Male/Cis-gendered/attractive/wealthy/whatever privilege"?

I might be willing to accept an answer of "Women are out-performing men in both the job market and education right now, so why are we continuing to give them extra assistance in doing so, rather then striving for equality", but that can't possibly be the entirety of the answer to the question.