ML researchers I've spoken to strongly dislike the way people suggest similarities between biologically-inspired systems and neural networks. The earliest papers did use our understanding of the neuron as inspiration for some of the ideas, but these approaches have far more similarities with statistical analysis than they do to neuroscience. At the end of the day, NNs are just nonlinear function approximation, and if I had my way we would have called it that from the start.
Choosing positioning like "NNs are similar to the brain" or even calling the field "machine learning" makes it harder to speak objectively about the research we perform, makes it hard to understand the sort of explanatory power and limitations of the models we fit, and makes it harder for users to understand how the system works. It's like how quantum mechanics researchers have to deal with readers who misunderstand what linear observable operators are and conflate it with their own ideas about conscious observers or use it as evidence of God's presence or whatever.
This! I was listening to a radio interview with a famous NN researcher and everything was going great until the interviewer asked him a question that only a neurologist could answer, and he kept talking in exactly the same way, answering as though he was a neurologist! I was embarrassed for him. It seemed to me that the correct and useful response would have been something like what you just said.
Choosing positioning like "NNs are similar to the brain" or even calling the field "machine learning" makes it harder to speak objectively about the research we perform, makes it hard to understand the sort of explanatory power and limitations of the models we fit, and makes it harder for users to understand how the system works. It's like how quantum mechanics researchers have to deal with readers who misunderstand what linear observable operators are and conflate it with their own ideas about conscious observers or use it as evidence of God's presence or whatever.