That is because they were rich and wealthy and powerful. They were mill lords because they had their own sugar mills. People had sugar cane farms so they depended on the mill lords to have the sugar milled. That's why the people who owned the sugar mills easily controlled neighborhoods and had a lot of money.