# MACaQ offers model compression tools for text, image, video and audio inference

_Published Wednesday, November 11, 2026 at 4:53 PM EST · AI · Latest · Tier 2 — Notable_

![SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch. - huawei-csl/MACaQ — Primary](https://repository-images.githubusercontent.com/1452417816/74bc9106-306c-4e93-96da-77dcff2e0d77)

The MACaQ repository provides tools to compress text, image, video and audio models for inference through llama.cpp, vLLM, Diffusers and PyTorch. The project claims it can quantize a 122B-parameter model in 64 minutes using 32.5 GB of device memory.

Quantization reduces the numerical precision used to represent model weights or calculations. MACaQ supports 2-, 3- and 4-bit GGUF text models, and its llama.cpp workflow supports CPU execution. The project reports faster vLLM inference for Qwen3.8-27B at lower precision, with a higher perplexity score than the original model. Full-model execution on Ascend requires integration with an inference runtime.

## Sources

- [GitHub](https://github.com/huawei-csl/MACaQ)

---
Canonical: https://techandbusiness.org/newswire/SKabinwu5Fq1lSElXUP7hZ
Published: 2026-11-11T21:53:29.118Z
Story chronology: 2026-11-11T20:23:57.000Z
Retrieved: 2026-11-12T01:07:57.310Z
Publisher: Tech & Business (techandbusiness.org)