To step or turn aside; to deviate; to swerve; especially, to turn aside from the main subject of attention, or course of argument, in writing or speaking.
To turn aside from the right path; to transgress; to offend.
talk off top of head
How often have the words digress and talk off top of head occurred in a corpus of books? (source: Google Ngram Viewer )