I've recently had a married male co-worker of mine messaging/calling me during and off-work hours. He asked for my phone number stating he wants to ask me questions about work. I reluctantly gave it to him (I'm still working on establishing strong boundaries) since he stated it would be only for work. Soon after providing my personal number, he started messaging and calling daily including mornings/evenings/weekends. Sometimes asking about the smallest thing related to work. Other times to ask me about my personal life. I always kept it professional but when he started joking about coming to his place and other more inappropriate (nothing overt, usually followed by "I'm just kidding" after making an uncomfortable joke - not sure if he realizes it's weird coming from a married man) jokes, I didn't respond. Then, because I started to distance myself, he started messaging me less usually later in the week and would start off with... "I'm messaging you mid-week because you're mean during the beginning of the week." or "Hope you aren't grumpy today." And, even when asked, "I didn't realize I was mean, could you care to elaborate what I did that came off as mean?" He would respond with, "Oh, I guess you didn't DO anything but I FEEL it and I know you." All coming from a person that barely knows me. Anyway, I distanced myself even more and only responded to his correspondence via our work email. Then, he started popping up in my messages again asking why I've been acting weird. I responded, "I'm sorry for not being communicative but I did not feel comfortable with some of the jokes you made. I also do not want to make anyone inadvertently uncomfortable including your wife." I would think it's odd for a married man (who works the same position as his wife) to be messaging another single woman in the same position when he can check his own resources, his wife, or management. He then responded with, "Wow, I'd never cheat on my wife. I was going to hook you up with my friend. You're like a little sister to me. I can't believe you'd think I'd cheat on my wife. I'd never." And, I never insinuated that he would cheat on his wife but wasn't sure the best way to respond about "Why are you acting weird?" from a co-worker and responded that way. I'm wondering if I should have just continued to only communicate via work emails and have disregarded that message. Now, he is saying that I am the weird one for even daring to think he would cheat on his wife. Which I didn't say, so now I am embarrassed in a work environment.

I would appreciate any advice, thank you.