What do you do let's say if you go out with a girl that some guy happened to date once, and you're in the same venue and he subtly makes fun of you.

How do you hold frame there? Do you need to walk up to his face and ask what's wrong in an assertive way, or just ignore and use amused mastery to the girl or group if they notice?