Yesterday I went out with 4 of my closest friends. I was near a bar and saw a girl walking the dog. I said to one of my friend ( we were both drunk) that it would've been funny if he went there and talked to the dog. A shirtless guy from a balcony on the other side of the street yelled "who else would he talk to" to me and I didn't hear it at first but he repeated it and I responded "the girl"(she was the only girl in that area) he asked which girl and I pointed her. He started talking and I understood that he was a thug and that he was crazy, really crazy protective so I just said ok bye. He continued and I said ok bro. Then when we got in the car and left I saw him coming down probably to fight. We were 5 and he was alone but nonetheless thinking about it got me super insecure while coming back home. I never got in a fight before and never trained martial arts(I'm 19). I got the idea that I couldn't win that fight because he was a thug. It shattered my frame. And he was so crazy that he could've come down with a knife.

What shuold I have done? Ignore him from the start when he yelled at me? Confront him? What is the proper response to verbal aggression? How can I stop being scared of people that grew up in the streets?