I get it don't shit where you eat, but I work in a large 400 person organization. What if the girl does not work directly with me on a regular basis?

I chatted her up walking out of the building one evening and we hit it off. A good ending or bad ending, neither would affect my career. Thoughts?