One of the worries, in a society without free speech, is that those in power will jail those who speak out against them. When power shifts, those that were jailed return the favor.
So, free speech enables all parties to know that they will have people say things about them but that they also are allowed to say things about their opponents in return.