I would imagine the importance of weights depends on the prompt. How do you deci... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		dhruvdh on April 17, 2024 \| parent \| context \| favorite \| on: Show HN: Speeding up LLM inference 2x times (possi... I would imagine the importance of weights depends on the prompt. How do you decide which weights are important?

kolinko on April 17, 2024 [–]

Yeah, that is the point more or less - it dynamically chise the weights layer per layer depending on the internal state.

A bit technical explaination here. https://kolinko.github.io/effort/equations.html

Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact