> I think a lot of people haven't picked up on the fact that in both cases chatGPT actually understands what you are telling it.

This is something that I haven't seen mentioned yet.

The stunning aspect of ChatGPT is that it seems to understand* the nuances of what I'm asking. Yes, sometimes it spews bullshit, but the bullshit is still trying to address my questions, no matter how odd.

* I suspect that "understand" may not be the correct word here, depending on your definition. But at the very least, it can parse the nuances of my questions.

The subtlety of its communication is fairly astonishing. "Digital assistants" from the world's biggest tech companies can respond to a few basic template questions and commands and tend to fall back on "here's a link to Wikipedia, chief".

In the same day and age, ChatGPT can respond to a statement like "actually it's a cat" with "yes, my apologies for the error, [repeats a lot of stuff with corrections]". In the process it's recognizing that your response is a correction, what "it" refers to, some of the implications of what that change means, and that you are expecting it to issue a response that amends its previous statements. It's several generations ahead of the state of the art.

> I suspect that "understand" may not be the correct word here,

There's no other word for what's going on. The inputs and resulting outputs show something indistinguishable from understanding.

If we choose to define "understanding" as some deeper internal process well that's a deadend because we don't even know the meaning of the term "understanding" from the context of the human brain.

So more or less from the inputs and the outputs there's only one word that describes what's going on. It "understands" you.

I suspect the sensation of understanding is a biological response to a level of confidence in the evaluation of some neural process. To that extent it’s perception is illusory, and often occurs in the absence of what anyone who knows the subject would term a true understanding.