I'd say you are more sad than him. He may be racist but at least he doesn't have nothing better to do then look through someones post history then make generalizations.
Remove the whole gender factor completely from your argument. If you had someone twice your size hit you in retaliation after you hit them you would have a problem with that. This guy didn't even hit the girl. He just held her wrist still. I think you have a legitimate point, but it doesn't pertain to this situation at all.