Yesterday I was eating dinner at this West Indies place when a homeless guy comes up to me asking for $. I told him I didn't have anything for him and he began saying that I was racist. I told him skin color has absolutely no bearing on whether or not I would give him money, it doesn't. He then started saying that I was a white piece of shit. I told him not say that about me, and he essentially says what are you going to do about it. I said a couple other things back to him, but that was about it.

My question is, how could I have handled that better? Should I have asked him to step outside, or said shit back to him when he was calling me out?

Also how do you guys handle altercations with people much bigger than you?