Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Yes, that makes sense. Some of my models currently run in ollama while others require their own inference servers. I’m curious about custom inference servers in bifrost and the ability to orchestrate keeping some models warm in memory while evicting and cycling other models. All of which span different providers in bifrost (I think). Obviously I can get the mechanics from an agent, more wondering about any experience with something similar.


Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: