Could you elaborate what you think the problems are? I guess they should be usin... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		zacmps 43 days ago \| parent \| context \| favorite \| on: Claude Code daily benchmarks for degradation track... Could you elaborate what you think the problems are? I guess they should be using some form of multiple comparison correction?

goldenarm 43 days ago [–]

The daily scale is not statistically significant and is meaningless. You should lower the confidence interval by either increasing the scale or the evaluations.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact