let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.
Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken).
I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree.
We have no idea how evolution "readied" a deeply complex organ like the brain over many thousands of years, then almost instantly repurposed it for language acquisition and generation. To further hypothesise that what it was "readying" was something that trains from data in a way similar to how chatgpt is trained from data makes it even more astonishing and until this is demonstrated it is more scientific to not accept this hypothesis.
More like 1000+ considering the Chauvet painters certainly had speech.