Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Cache Misses are pretty bad, i have a locally running deepseek v4 flash, i have tuned it now to have 1100-1300 prefill. Its not great but properly useable. Imagine having a session with already 100k and half of it has to be prefilled it would be waiting minutes with worse numbers. And if you are paying by the token, for hosted models, you are wasting money.


Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: