Is it social pressure and shame? Is it due to hypergamy?

Shouldn't it generally be less of a concern for women since their children are for sure... theirs?