To fall asleep slowly.
To lose concentration.
Used other than figuratively or idiomatically: see drift, off.