Factor 1: Who decides to run for office and what advantages candidates have over one another.
Factor 2: Incumbency. Usually the incumbent (the person already serving in the office) has a large advantage over challengers, unless negative events have the voters wanting to get rid of incumbents.
Factor 3: The way congressional districts are drawn. There have been court cases recently about "gerrymandering" -- attempts by a party in power to draw district boundaries in ways that will serve to perpetuate that party's power.