Think about it, in most animal species for which the male and female look significantly different it is usually the male which is more beautiful (e.g. compare peacocks with their tails to peahens, lions with their manes to lionesses or male vs female mandrills if you want an example with primates).

So why is it that for most of human history it was the woman who was expected to be beautiful so a man would choose her while the man's looks didn't matter as much despite the fact that it should have been the opposite way around if anything?