Videos of it are mostly simulations and documentaries since there were no cameras then, but the people who built it were builders and regular workers who worked for the feudal lords who owned the property. They would build a castle and in return the Feudal lord would give them something. In Europe, it was mostly land or gold or things like that, or a fancy title if they were famous for building.