A small shop and/or cafe, often part of a larger shop, that sells fast food and a range of dairy based beverages such as milkshakes and ice creams; a deli or delicatessen.
tea room
How often have the words milk bar and tea room occurred in a corpus of books? (source: Google Ngram Viewer )