If the model weights are stored as int8, does this mean that the floating point ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		nextaccountic on March 5, 2023 \| parent \| context \| favorite \| on: Show HN: Llama-dl – high-speed download of LLaMA, ... If the model weights are stored as int8, does this mean that the floating point capacity of the GPU is wasted? Or the int8 is converted to float in the GPU?

woodson on March 5, 2023 [–]

Well, tensor cores support int8 instructions (at least from Turing onwards), so the hardware is being used, if that’s your concern.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact