Why would the model be self aware? There is no mechanism for the llm to know the answer to “what are you” other than training data it was fed. So it’s going to spit out whatever it was trained with, regardless of the “truth”

I agree there's no reason to believe it's self-aware (or indeed aware at all) but capabilities and origins is probably among the questions they get most, especially as the format is so inviting for anthropomorphizing and those questions are popular starters in real human conversation. It's simply due diligence in interface design to add that task to the optimization. It would be easy to mislead about if the maker wished to do that of course, but it seems plausible that it would usually be have been put truthfully as a service to the user.

> There is no mechanism for the llm to know the answer to “what are you” other than training data it was fed.

The training data and the system prompt but yes.

This doesn't necessarily confirm that it's 4, though. For example, when I write a new version of a package on some package management system, the code may be updated by 1 major version but it stays the exact same version until I enter the new version into the manifest. Perhaps that's the same here; the training and architecture are improved, but the version number hasn't been ticked up (and perhaps intentionally; they haven't announced this as a new version openly, and calling it GPT-2 doesn't explain anything either).

It means its training data set has GPT4-generated text in it.

Yes, that's it.

I think using chat gpt output to train other models is against the TOS and something they crack down on hard.

I would love to see that legal argument given their view of “fair use” of all the copyrighted material that went into OpenAI models.

Yeah that isn't reliable, you can ask mistral 7b instruct the same thing and it will often claim to be created by OpenAI, even if you prompt it otherwise.