Is a single H100 enough? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		nextaccountic on Sept 17, 2023 \| parent \| context \| favorite \| on: Run LLMs at home, BitTorrent‑style Is a single H100 enough?

brucethemoose2 on Sept 18, 2023 [–]

80GB is enough, yeah.

I'm not sure what exact LORA/quantization settings would be ideal, but check out https://github.com/OpenAccess-AI-Collective/axolotl#config

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact