How much are you saving once the more expensive model already has all the contex... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

blfr 27 days ago | parent | context | favorite | on: Fable and the end of the free lunch

How much are you saving once the more expensive model already has all the context loaded and ready to go?

csullivannet 27 days ago [–]

API calls get more expensive, not less, as you've loaded more context. This is exactly when you want to switch to cheaper models.

camdenreslink 27 days ago | [–]

There is caching to consider. Switching models throws away the cached tokens.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact