2017 SOTA on Penn Treebank was 47.69 perplexity. GPT-3 is at 20.5. AI has already been productized on consumer devices through Siri, Google Assistant, speech detection, speech generation, textual photo library search, similar data augmentations for web search, Google Translate, recommendation algorithms, phone cameras, server cooling optimization, phone touch screens' touch detection, video game upscaling, noise reduction in web calls, file prefetching, Google Maps, OCR, and more. DLSS alone justified continued investment by NVIDIA. NVIDIA Ampere will be ~6x as fast at running consumer-targeted models as Turing, given raw throughput increases compounded with sparsity and int8 hardware. A huge number of research threads around AI have direct applicability to large tech companies.
I'm not arguing that current machine learning technologies are not useful. I'm just arguing that progress is based on increasing some metric, usually depending on a trade-off of computation. This can even make ML-techniques applicable to some new fields, but it's not what is holding back autonomous driving, the often touted parade example which also brings in a lot of employment for machine learning.
This article clearly sits on the peak of inflated expectations in the hype cycle.
It's not just that you're not arguing it isn't useful; as far as I can tell, neither of your comments contain an argument against ML at all. I have nothing to meaningfully argue against.
ML is undergoing a Cambrian explosion of use-cases (see my prior comment), almost all of this over an incredibly small time period, progress is accelerating, and many of these use-cases are incredibly high value. Scale is not proving a major stopper; Google's MoE experiments show that huge models are productizable, and small models work plenty fine too in restricted places, to the point where they're literally used to parse touch screen sense data in phones.
If you want to claim we're in for another AI winter, you need a vastly stronger argument than ‘something something hype cycle’.
