I believe the answer is: they had been born in the United States and considered it home
Despite the horrible treatment that they received, they still see United States to be the environment that they are the most familiar with. On top of that, a lot of them believe that the amount of job opportunities in united states far exceeded those in African continent.