They worried because they were afraid of the Soviets. The missile gap was a false belief that the Soviets had many many more missiles than the US and that if they decided to attack it from far away it would win easily. It turned out in the end that it was fake and that the Soviets actually had fewer ballistic missiles than the US.