This depends on the time period. However, here are some common themes seen in American history, regardless of time period.
1) Southern whites were scared that black citizens would vote into office other black citizens- This irrational fear was fueled by racism in the South. This was especially prevalent during the era of Reconstruction (1865-1877).
2) Southern whites were worried that black citizens would vote for candidates who would help blacks gain equal rights in American society. - This was true in several different time periods, including the Antebellum era (1820-1860).