I'm not saying this is some huge secret revelation, but from what I have seen, physical differences between men and women usually get brought up from the female perspective, focusing on how men are different from women, like how men are bigger and stronger and more threatening and less pregnant.

So in a similar vein, women are smaller and weaker in general, they have smoother skin, different body shapes, pregnancy makes them more vulnerable, and also their voice is higher pitched. Higher pitched voices are associated with cuteness even in the animal world, because babies have high pitched voices, and they need protection and care. I heard this is the reason why house cats developed such high pitched voices, so that humans will like them more. Same goes for when a dog is "whining", trying to garner sympathy.

I would say that it's about as understandable that this would color how women are seen in contrast to men as the opposite. As much as women are constantly aware of being smaller and more vulnerable, men are also constantly aware that they are less cute and less valued. On a "lizard brain" level, these physical differences simply kind of matter. It's just a different vibe. Heck, if we manage to deconstruct everything that's really societal, we would still have these differences.