#70B model on consumer GPU

70B LLMs on 4GB VRAM — AirLLM's Layer-by-Layer Trick That Rewrites the Rules

AirLLM is an open-source Python library that runs 70B large language models on a single 4GB consumer GPU — with no quantization, distillation, or pruning — by loading just one transformer layer onto the GPU at a time.