Is there anything you can tell me that will help me drop the nagging feeling that gradient descent trained models will just never be good?
I understand all of what you said, but I can't get over that fact that the term AI is being used for these architectures. It seems like the industry is just trying to do a cool parlor trick in convincing the masses this is somehow AI from science fiction.
Maybe I'm being overly cynical, but a lot of this stinks.
The thing is AI is already "good" for a lot of things. It all depends on your definition of "good" and what you require of an AI model.
It can do a lot of things that are generally very effective. High reliability semantic parsing from images is just one thing that modern LLM's are very reliable at.
Wouldn’t you say that now, finally, what people call AI combines subsymbolic systems („gradient descent“) with search and with symbolic systems (tool calls)?
I had a professor in AI who was only working on symbolic systems such as SAT-solvers, Prolog etc. and the combination of things seems really promising.
Oh, and what would be really nice is another level of memory or fast learning ability that goes beyond burning in knowledge through training alone.
I had such a professor as well, but those people used to use the more accurate term "machine learning".
There was also wide understanding that those architectures were trying to imitate small bits of what we understood was happening in the brain (see marvin minsky's perceptron etc). The hope was, as I understood it that there would be some breakthrough in neuroscience that would let the computer scientists pick up the torch and simulate what we find in nature.
None of that seems to be happening anymore and we're just interested in training enough to fool people.
"AI" companies investing in brain science would convince me otherwise. At this point they're just trying to come up with the next money printing machine.
You asked earlier if you were being overly cynical, and I think the answer to that is "yes"
We are indeed simulating what we find in nature when we create neural networks and transformers, and AI companies are indeed investing heavily in BCI research. ChatGPT can write an original essay better than most of my students. Its also artificial. Is that not artificial intelligence?
Hiding the training data behind gradient descent and then making attributions to the program that responds using this model is certainly artificial though.
Can't you judge on the results though rather than saying AI isn't intelligent because it uses gradient descent and biology is intelligent because it uses wet neurons?
> It seems like the industry is just trying to do a cool parlor trick in convincing the masses this is somehow AI from science fiction.
If you gave a random sci-fi writer from 1960s access to Claude, I'm fairly sure they wouldn't have any doubts over whether it is AI or not. They might argue about philosophical matters like whether it has a "soul" etc (there's plenty of that in sci-fi), but that is a separate debate.
I understand all of what you said, but I can't get over that fact that the term AI is being used for these architectures. It seems like the industry is just trying to do a cool parlor trick in convincing the masses this is somehow AI from science fiction.
Maybe I'm being overly cynical, but a lot of this stinks.