The main ideas of love and loss are first expressed together in the third stanza. Before that, the speaker only talks about their love. Sure, he mentions that it happened in the past - but it still doesn't mean that he lost her. The third stanza makes it clear that Annabel Lee died because the heaven was jealous of their love. He also mentions the sepulchre, making it clear that she died.