lay a finger on vs manipulate

To merely touch. 

To move, arrange or operate something using the hands 

To influence or control someone in order to achieve a specific purpose, especially one that is unknown to the one being manipulated and beneficial to the manipulator; to use 

To handle and move a body part, either as an examination or for a therapeutic purpose 

To influence, manage, direct, control or tamper with something