DwarfStar 4 offers local inference through selective model compression

DwarfStar 4 offers a C inference engine for running supported DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next models on high-memory Mac, CUDA and ROCm machines, its project site says. The MIT-licensed software combines text and vision models, local APIs, a command-line interface and a native agent. The engine compresses routed model experts to two bits while preserving precision in critical shared paths. It also saves long prompt prefixes to SSD for reuse after restarts. The project lists V4 Flash Q2 as its baseline and says V4.1 Q2 streams from SSD; its published performance figures are estimates from its benchmark table.