The link mentions "our internal job scheduler" and how they had to optimize it f... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		pinko on March 12, 2024 \| parent \| context \| favorite \| on: Building Meta's GenAI infrastructure The link mentions "our internal job scheduler" and how they had to optimize it for this work -- does anyone know what this job scheduler is called, or how it works?

KaiserPro on March 12, 2024 [–]

it might be twine: https://www.usenix.org/system/files/osdi20-tang.pdf

but I suspect its not that, because Twine is optimised for services rather than batch processing, and doesn't really have the concept of priorities.

radicality on March 12, 2024 | [–]

I would think it’s probably that. Also, has this been renamed to Twine from Tupperware?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact