The article and linked GitHub project both say this runs on the CPU. The GitHub ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

zamadatix on March 11, 2023 | parent | context | favorite | on: Running LLaMA 7B on a 64GB M2 MacBook Pro with Lla...

The article and linked GitHub project both say this runs on the CPU. The GitHub project notes the quantization doesn’t work right with other systems for some reason but if you have enough RAM to quantize it you have enough RAM to run it without quantizing it anyways.

pletnes on March 11, 2023 [–]

Aha. Then I guess performance is the only issue. I guess it should run fine on Intel.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact