As usual, it was about slavery. While they were together on immigration, that is against immigration, they split on slavery since the southerners wanted to keep slavery intact while the northerners supported the Union and wanted to abolish slavery, and after that they weakened and stopped existing after the war.