A mistress; a woman with whom a man carries on a physical affair (not emotional affair) without having the intention of forming a serious relationship.
How often have the words fille de joie and side piece occurred in a corpus of books? (source: Google Ngram Viewer )