Using @here in a channel is the equivalent of standing on your desk and shouting at everyone around you. So what happens is, like you, everyone starts muting it.
And then you just have people shouting into the void.
I think you should rules about it at the workplace. At mine, I can probably count on one hand the amount of times I've seen an @here. And one of them, the person was told he shouldn't have done it since it wasn't important. Really makes everything think before they use it.
We use separate channels for different projects and people join/leave as needed. @here is incredibly useful when you have a general question and not sure who might have insight.
It all boils down to how people use the communications channel. I see @here multiple times a week but do not feel like it is a big distraction. At worst it can create a few minutes of distraction from my current workflow and if I'm truly in the middle of a thought process I ignore the message for a little bit or unless a 2nd notification hits.