I think because they are the pioneer species that are used to create soil. Thus they are replaced by larger organisms during the secondary succession.
Pioneer species are the first organism that live in a previously uninhabited area, they break down rock more and when the die their remains and rock pieces form soil. This happens in primary succession which involves the establishment and development of an ecosystem in an area that was previously uninhabited.