The point is called "Port".
In a PC there are two kinds of ports usually, internal and external. Both are used to as connection points to connect various devices to PC. Internal port can be used to connect devices such as HDD, CD/DVD ROMs etc while external are used for flash drives, printers and such kind of devices.