Culture definitely hurts it a lot. It seems like any question involving Java and anything performance related is "you don't, you're stupid, the JIT fixes everything, never measure, never benchmark, premature optimization is the root of all evil which means never, ever attempt to optimize anything ever"
There's pockets of people doing Java + performance, but they are far & away the exception and they are frustratingly insular about what they do & why they do it. And yeah it tends to often go against every bit of guidance from things like Effective Java & similar.