The important point is that the surface area to the volume ratio gets smaller as the cell gets larger. Thus, if the cell grows beyond a certain limit, not enough material will be able to cross the membrane fast enough to accommodate the increased cellular volume.
Hope this helps :)