Sleep has something to do with batch-processing long-term memories. We don't know exactly what.
But that means (speculation warning) long-term memory circuits are switched into a different mode than during the day, and from an outside perspective they recall all sorts of random nonsense. That's probably what causes dreams. If the short term memory is unaffected and works normally, this explains why you can sometimes remember the nonsense when you wake up.
Sleep has been used as a form of regularization in neural networks before, although it's not in the transformer LLMs we're all using this year. This form of regularization makes the network generate outputs from no inputs, and then gradient-ascends (opposite direction from normal) away from those samples. Hey, that sounds a bit like human sleep, doesn't it?
In sleep my fundamental thingy, "that which observes", is catapulted out of the matrix. To the shores of the outer world probably, further maybe. The stuff is real but hallucinating happens too. And I carry my universal translator with me too. Converting the strange experiencestuff into common time-space-meat format.
But yes, it is deeply strange. A profound loss of familiar context. And with it the associated anchors of memory.
But that means (speculation warning) long-term memory circuits are switched into a different mode than during the day, and from an outside perspective they recall all sorts of random nonsense. That's probably what causes dreams. If the short term memory is unaffected and works normally, this explains why you can sometimes remember the nonsense when you wake up.
Sleep has been used as a form of regularization in neural networks before, although it's not in the transformer LLMs we're all using this year. This form of regularization makes the network generate outputs from no inputs, and then gradient-ascends (opposite direction from normal) away from those samples. Hey, that sounds a bit like human sleep, doesn't it?