MACaQ offers model compression tools for text, image, video and audio inference

The MACaQ repository provides tools to compress text, image, video and audio models for inference through llama.cpp, vLLM, Diffusers and PyTorch. The project claims it can quantize a 122B-parameter model in 64 minutes using 32.5 GB of device memory. Quantization reduces the numerical precision used to represent model weights or calculations. MACaQ supports 2-, 3- and 4-bit GGUF text models, and its llama.cpp workflow supports CPU execution. The project reports faster vLLM inference for Qwen3.8-27B at lower precision, with a higher perplexity score than the original model. Full-model execution on Ascend requires integration with an inference runtime.