I’m looking forward to this! Are you using an adapter (I don’t see it mentioned ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

syntaxing on July 24, 2023 | parent | context | favorite | on: A simple guide to fine-tuning Llama 2

I’m looking forward to this! Are you using an adapter (I don’t see it mentioned in your article)? I was under the impression you cannot fit 7B at 4 bit since it’ll take 25GB of VRAM or so.

samlhuillier on July 24, 2023 [–]

Yes using the qlora adapter that hugging face provides with peft

syntaxing on July 25, 2023 | [–]

ahh, I was on my phone before so I must of glimpsed over it, I see it on the last section. Thanks!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact