The reason that legal impossibility will act as a defense to attempt and factual impossibility will not is because it is legally impossible to commit a crime that does not exist.
Factual impossibility only refers to the defense which convey that illegal act can't be done because it's physically impossible to do so ( for example, we deny the accusation of a robbery because we stated that we're on another city during the time when the robbery happen). But these kind of defense tend to be weak and has a lot of loop holes.