If 1 million tokens is a lot, how about 2 million?
Today we’re expanding the context window for Gemini 1.5 Pro to 2 million tokens and making it available for developers in private preview. It’s the next step towards the ultimate goal of infinite context. #GoogleIO
In theory, if you have an infinite context, you could build an AI assistant which accompanies you throughout your whole life and would have all the info on what you've done at your disposal for immediate query or discussion. Also, in theory, if you'd then die, the world would have a digital replica of you up to the point of your death.
A replica of what? Of your words? Your thoughts? How would it access those thoughts? What about your feelings? What about the things that actually go on in your physical body outside the brain?
Encoding the human experience will probably get much better over time. They're already experimenting with audio and video interfaces. I saw some other products that combine fitbit like sensors with an LLM.
I don't know what the results of combining all that data with an LLM will look like, but I think recording our experience to be used by an AI will only get better.
If it hears what you hear, hears what you say, reads what you write, sees what you see, notes what you do, it will end up knowing more about you than you yourself. The way you can interact with it will be the same way others can interact with it.
Also, sensors like HRMs or activity monitors could give it insights into your emotional state.
[0] https://x.com/Google/status/1790430189916225799