It is due to a lack of providable energy to the next trophic level of the energy pyramid. Primary consumers only obtain around 10% of energy that producers have, and the energy depletes as you move further and further up the food chain, until you reach the tertiary consumers which have the least amount of energy at their disposal, meaning there are less of them than other organisms further down the energy pyramid.
Hope this helped!