Holden is not a confident man and is at a point of emotional breakdown towards the end of the book.One such factor was death of Allen.Another point could be that he was fired his prep school and another factor could have been his incident with Antolini.These incidents made Holden weak and vulnerable from inside.But Allie's passing away could have been the starting point of his mental instability.