The US Capitol is located in the South because of the Compromise of 1790.
Thomas Jefferson famously convened a dinner party and invited James Madison and Alexander Hamilton and no one else.
By the end of the meeting, the US Capitol would be in DC, the US financial system would be in NY, and the US would assume all of the State's war debt as a war to build credit.
This is famously chronicled in the musical Hamilton.