I have a lot of canned material:

"You're so funny/cool/fun to be around/whatever, it's too bad you aren't my type."