I can think only of upsides.

No more awkward feelings caught for your female friends (because why the Hell would you ever catch feelings for someone who you believe is incapable to reciprocate them), no more misread indicators of interest ("is she smiling because she is in a good mood or flirting?" - she is not flirting; case closed), no more romantic advancements misinterpreted/misreported as sexual harrassment (because why would you "advance romantically" on someone who you believe can't have romantic feelings back).

What would be the downsides?

Putting aside the "how" (how we convince so many men that women are barely capable of romantic love). Just assume we could.