I agree with you in that context and use matter. Just as I disagree with blanket banning words, I also disagree with the "words don't matter" reaction. Neither approach make sense to me.