I think it's because we once went to war with china. Many people blamed them for the labor unrest that had spread across the country in the late 1800s.
They especially feared that foreign anarchists and socialists might undermine American democracy.
hope this helps:)