I live in a place with more free speach than some places but some things like free speech zones as you described is the situation. The point is none of these are justified, and maximum freedom should be the default. I should be free to stand on a public sidewalk and say whatever I want about an abortion provider. Now if I start harassing and assulting those around me, I should be arrested. None of this is complicated. The state simply loves to abuse its authority when no crime has actually occured and dress it up in the terms of public welfare, safety, anti-terrorism, "think of the kids", etc.
You're right that individuals with unpopular ideas can also be persecuted by citizens or companies instead of the government. This is unfortunate, and I would like to see everyone be more tolerant, but I'm clearly focusing this conversation on government's prosecution of free speech because that is the most clear area of moral abuse.
Where do you live? Some states and provinces have buffer zone laws where you cannot protest against abortion providers. People have been fired for saying the wrong things or voting for the wrong candidates. People who live in "free" countries get persecuted for their speech and beliefs all the time. Maybe not by the governments but by the people.