Longer pendulums swing with a lower fequency than shorter pendulums. The force of gravity accelerates the pendulum down. The momentum built up by the acceleration of gravity causes the mass to swing in the opposite direction to a height equal to the original position. So therefore I think the answer your looking for is gravity.