Both sides of this argument have it wrong.

It is easy for women to claim that sex shouldn't contribute to your self-worth, because the majority of women receive or have received minimally sustaining (sexual) attention at some point in their lives. You don't have to be anything special to get cat-called or simply looked at. Whether women consciously realize it or not, they have been sexually validated since they first grew tits (and society tells them via media "tits are sexy") and got looked at.

Men, on the other hand, don't. (Sexual) attention, both direct and indirect, is not a given unless you are at least of average attractiveness. It's a blessing and a curse ⁠— men can be valued for their abilities alone, whereas historically women have been expected to be decent looking in order for their accomplishments to be taken seriously...

For the vast majority, sex is a basic human need. Being sexually desired is validating, and it should be, given that it validates the competence to fulfill a basic human need. Thankfully, though, it is not the only human need. As such, our self worth is determined by a myriad of other contributing factors. See Maslow's hierarchy of needs we all know and love.

In a world where casual sex was not so normalized and rampant, I might argue that the obsession to accomplish casual sex is a symbol of mental illness, but it actually makes sense in this day and age. However, those who argue that the "only" thing that would make them truly happy and fulfilled is "ramming hoes like Chadwick" either have never fucked a pussy in their entire life or simply have a few screws loose, perhaps has a result of that scarcity. When you are able to fulfill your basic need of attaining sex and intimacy, perhaps "ramming hoes like Chadwick" is an extra benefit you'd like to experience, but to hinge your entire mental wellbeing and sense of self worth on just that ONE thing very clearly shows you have bigger problems to work through. Ramming will not fix those problems.