Why do people act like men are lesser victims just because they're usually murdered and assaulted by other men?

People seem to overwhelmingly focus solely on violence against women. When you mention that men are actually more likely to be victims of violence so they need attention too, the response is something like "men are victims of other MEN" as if somehow that means that they're lesser victims. I just don't get it. Why should violence be treated as a more serious deal when it's committed towards another group than when it's committed towards the in-group?