There are two scenarios. One is the weak acids that are released by the plants and that slowly eating throught the rock, dissolving them. Eventually they destroy them and they break. The same happens with roots. Roots enter cracks and spread and grow and go through rocks and break them. These are quite common.