okay so i've been practicing my social skills for a while now(first time i come here, though).

anyways i have no problem making eye contact with anyone whether they're male/female, whatever. i always show confident body language. something that bothered me recently though is i was at the gym i went to get a drink of water and this old man probably 30~y/o standing behind me using the lat pull down a few feet behind the water fountain whispered stupid shit in my general direction. he said it in spanish but what he basically said was "bugger. calm down." I just ignored him like i didn't give a shit about him i didn't react at all but now i think i should have asked him something along the lines of "you said something?" or "you mad?" fuck. keep in mind i'm not big i'm 5'8 or 5'9 150~lbs i don't look scrawny as shit but i'm not the biggest guy at the gym.

i'm while i'm not rude to anyone, i do have a high sense of self-worth and confidence even though i'm small and younger than most people at the gym. i look at people if they look at me and i don't care how big and strong you are i will go up and ask how many sets you have left/etc. etc.

this is the first time it happened to me and i was just wondering.

TL;DR can confidence back fire and make people dislike you?