Just came back from a party, and there was a guy there who was trying to pick fights with everyone for little reasons

At one point he was getting up in my face talking about how he'd punch me, and I just stood my ground and told him assertively to calm down, defusing the situation. I wasn't worried about being beaten up as I do martial arts and all my friends there also do various fighting sports

Did I handle this correctly? Or should I have been more aggressive towards him?