Don't invite him. Seriously though, "competing" for a girl is stupid. If this so-called friend has literally ruined a past relationship, why the hell are you still friends with him?
Maybe you're being too obvious that you're looking for a hook up? Tough to pin point exact reasons why it isn't working, maybe just switch up your approach a little.