Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You also need space in VRAM for what is required to support the context window; you might be able to do a model that is 14GB in parameters with a small (~8k maybe?) context window on a 16GB card.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: