#llama.cpp MoE

Tencent Hy3: Run a 295B Flagship MoE Model on a Single GPU with GGUF and llama.cpp

Tencent's Hy3 is a 295B Mixture-of-Experts model that activates only 21B parameters per token — and with new 1-bit and 4-bit GGUF quantizations, it can now run on a single GPU via llama.cpp, with MTP speculative decoding for extra speed.