Free speech isn't for good ideas. Nobody tried to shout down the guy who first said, "let's make everyone stop when a schoolbus has its lights flashing".
Actually, that is one of the reasons to have it. It's not to give people the right to shout garbage about the Jews because they should. It serves the 'public good' to have ideas out there. Otherwise, the Church would still have control over all of our lives.