take it out of vs try the patience of

take it out of

verb
  • To enervate or make tired; to sap the strength or vitality of. 

  • To exact payment from; to punish. 

try the patience of

How often have the words take it out of and try the patience of occurred in a corpus of books? (source: Google Ngram Viewer )