Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The prompt is essentially fine-tuning with extra steps


Not really. Fine-tuning fundamentally changes the model weights to be more amendable to a particular use case/domain; the few-shot prompts for GPT-3 is just a guide (and it's very easy for the model to ignore said guides and go on a tangent).

If you could finetune the 175B model, you'd likely get even better results for these Q&A prompts. (unclear how the OpenAI API is implementing its finetuning demo but I believe it's not on the 175B model).


I think they only fine tune the top layers.




Consider applying for YC's Winter 2027 batch! Applications are open till November 2.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: