Llama-3 | search
Click here to view Llama-3 news from 60+ newspapers.
Bookmark or Share- How to Install and Deploy LLaMA 3 Into Production on AWS EC2Deploying LLaMA 3 8B is fairly easy but LLaMA 3 70B is another beast. Given the amount of VRAM needed you might want to provision more than one GPU and use a dedicated inference server like vLLM in order to split your model on several GPUs. LLaMA 3 8B requires around 16GB of disk space and 20GB of VRAM (GPU memory) in FP16. As for LLaMA 3 70B ...
- Llama 3 hardware recommendation help : r/LocalLLaMA - RedditAnd Llama-3-70B is, being monolithic, computationally and not just memory expensive. Plus, as a commercial user, you'll probably want the full bf16 version. While quantization down to around q_5 currently preserves most English skills, coding in particular suffers from any quantization at all.
- Llama-3 8b finetuning 2x faster + fixed endless generationsLlama-3 70b can fit 6x longer context lengths!! Llama-3 70b also fits nicely on a 48GB card, while HF+FA2 OOMs or can do short sequence lengths. Unsloth can do 7,600!! 80GB cards can fit 48K context lengths. Also made 3 notebooks (free GPUs for finetuning) due to requests: Llama-3 Instruct with Llama-3's new chat template.
- Llama-3 8b on Intel "AIPC" : r/LocalLLaMA - RedditOn "AI PC" you can try doing the same with llama.cpp cpu finetuning, but it's unbelievably slow and will take you 50 days. It's like saying that a e-bike is more power efficient than a car, but you're forgetting that you're not gonna ever be able to even do simple things like go to a shop and buy yourself a new TV/fridge and drive home with it ...
- Llama 3 vs. llama 3 instruct : r/LocalLLaMA - RedditSubreddit to discuss about Llama, the large language model created by Meta AI.
More
