That raises an interesting question. What counts as speech? Does porn count? How about slander? Or threats? Or political donations/bribes (e.g. Citizens United)? Dishonest statements in advertising--who decides which "facts" are correct or dishonest? Should speech be allowed in all public situations (e.g. can I go up to kids and start saying lewd things)? Recruitment for terrorists groups or for a bank robbery?
I 100% support free political speech, expression of religious and philosophical beliefs, etc and all the "obvious" cases not included above. I don't have good answers myself to many of the above questions. But I think the "obvious" cases of political/religious/philosophical should gain universal support, and many (most?) people can't even here move beyond supporting it only for their "side".
I'm not going to lie and say I have a true free speech belief, but I do try to push myself to stay as close to it as possible. For example, It is hella hard to defend animated loli porn, but it doesn't technically hurt anyone and should be a part of protected speech.
I agree with you completely, it seems that a lot people take free speech for granted. They take a view point that if it doesn't affect them, then it isn't a big deal. Or if they disagree with it then it should be silenced. But if we don't take a stance to defend some forms of offensive speech now, later down the line other types will be silenced as well.