The book that shows harry think about to distract himself from aunt Magie's insults was at the third one: Harry Potter and the Prizoner of Azkaban. The setting was a dinner time, where Aunt Margie, a relative of the Dursley's visits and kept insulting Harry to the point of provoking him.