My first thought when I read the name was that it only needs the 1st letter changed to take on a whole new meaning as a heterosexist slur. Then I saw you were in the UK too, and wondered if you've also noticed this or it's just me?!
It's a tricky one that - you don't want to lose the identity you've built up, but recognise the name may have issues.
I'm actually in a similar situation; I chose a company name 15 years ago that I now regret because it's difficult to spell and pronounce (and I couldn't get the '.com', which I mistakenly thought wouldn't matter at the time). I've always been too scared to change it!