The soils in the rainforests are very labile, and they are soils that evolved in environment where they are protected from erosion by the very dense network of root systems.
When the vegetation is cleared, and the soil is exposed, having no natural protection from erosion, the surface layer is washed away very quickly, as it is soft and unprotected. The erosion is mainly caused by the rains that are present almost every day, so the soil can be used only for a couple of years, and than it is useless for farming.