If he cums too quickly, he's ruined the sex. If he doesn't cum at all, his sex partner will get offended. If he can't make her cum, it's his fault. Society unilaterally judges men for being bad/good at sex, making the man's performance higher pressure than the woman's.

Once he does cum, he has zero control over whether a baby will be born with his sperm. His next 18+ years will drastically change depending on the whims of whoever he has slept with. This makes sex higher pressure for men, since they have less control over the consequences of sex.

If sex is so high pressure, why not avoid it at all together? Lacking a sex life is culturally stigmatized for men too since they'll get shamed for not being able to attract women.

Thus sex creates a higher pressure for men since, unlike women, they have zero control over the final outcome and they are heavily judged for their sexual performance or their lack of sex.