To make a memorable impression or have a strong effect on someone; to impact someone's feelings, especially when such an effect is not expected or wanted.
twist someone's balls
verb
To annoy someone.
How often have the words get under someone's skin and twist someone's balls occurred in a corpus of books? (source: Google Ngram Viewer )