This is an interesting thought. GPT-3 used 45TB of raw CommonCrawl data (which w... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		greyface- on July 19, 2020 \| parent \| context \| favorite \| on: OpenAI's GPT-3 may be the biggest thing since Bitc... This is an interesting thought. GPT-3 used 45TB of raw CommonCrawl data (which was filtered down to 570GB prior to training). The Internet Archive has 48PB of raw data.

GreenHeuristics on July 19, 2020 [–]

That 48PB is mostly just old video game roms and isos though

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact