stick by vs stick with

to follow or adhere to somebody or something, to remain loyal and unswerving in one's attitude towards somebody or something 

To follow or adhere to. 

To endure in the memory of. 

To remain close to. 

Used other than figuratively or idiomatically: see stick, with. 

To follow loyally. 

To persist in using or employing.