most machine learning research does not require millions of dollars...
if you’re thinking about GPT-3, it uses the transformer architecture.
But the paper that introduced the transformer architecture trained for a few days on 8 GPUs — surely this was still serious research?
OpenAI sure has a good PR team, but all their spectacular results were built on previous papers (sometimes from their own researchers) that introduced new techniques without spending millions of dollars.
if you’re thinking about GPT-3, it uses the transformer architecture.
But the paper that introduced the transformer architecture trained for a few days on 8 GPUs — surely this was still serious research?
OpenAI sure has a good PR team, but all their spectacular results were built on previous papers (sometimes from their own researchers) that introduced new techniques without spending millions of dollars.