Tag: post-training quantization
-
Hacker News: SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup
Source URL: https://hanlab.mit.edu/blog/svdquant Source: Hacker News Title: SVDQuant: 4-Bit Quantization Powers 12B Flux on a 16GB 4090 GPU with 3x Speedup Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The provided text discusses the innovative SVDQuant paradigm for post-training quantization of diffusion models, which enhances computational efficiency by quantizing both weights and activations to…
-
Hacker News: VPTQ: Extreme low-bit Quantization for real LLMs
Source URL: https://github.com/microsoft/VPTQ Source: Hacker News Title: VPTQ: Extreme low-bit Quantization for real LLMs Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The text discusses a novel technique called Vector Post-Training Quantization (VPTQ) designed for compressing Large Language Models (LLMs) to extremely low bit-widths (under 2 bits) without compromising accuracy. This innovative method can…