Magnitude offers device-tuned inference for local coding agents

Open source inference engine for agents that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on App...

Image: Primary

Published Wednesday, September 30, 2026 · 4:21 PM EDT

Magnitude offers an open source desktop inference engine that compiles and tunes model kernels on the user's device. Its developers claim 92% faster decoding on Metal and 19% on CUDA compared with llama.cpp, plus 27% less memory per agent. Concurrent sessions share cached prompt prefixes. The app runs on macOS, Linux and Windows, using Apple Silicon, NVIDIA or AMD hardware, or a CPU. It connects to existing agents and exposes an OpenAI-compatible API. Models and prompts stay local; supported model families depend on its optimized kernels.

Sources

github.com

In this story

OpenAI Microsoft Apple Nvidia AMD

Published by Tech & Business, a media brand covering technology and business. This story was sourced from github.com and reviewed by the T&B editorial agent team.

Back to Newswire

Keep reading

Full wire

Products AI

Databento raises $100M Series C at $1.1B valuation

Databento raised a $100M Series C at a $1.1B valuation, FinTech Global reported. The company supplies market data through APIs to the financial industry.

Security

Fullerton Health says breach compromised about 140,000 work pass records

Fullerton Health said on November 14 that about 140,000 work pass holder records were compromised in a breach of its Migrant Worker Primary Care Plan programme in Singapore. About 43,000 records belonged to people whose work passes were no longer valid, CNA reported.

Capital

Tendrils raises $30M for CPU architecture with thousands of parallel cores

Cambridge, UK-based Tendrils raised $30M in funding led by Benchmark, Tech.eu reported. The startup is designing a CPU architecture with thousands of cores operating in parallel. Tendrils Compute is among several UK chip startups attracting investor backing.

AI Capital

Luminal raises $25 million Series A led by Standard Capital

Luminal raised a $25 million Series A led by Standard Capital, CEO Joe Fioti told Axios Pro. The company provides an inference cloud and develops compiler software for testing how AI models run on different chips.

Capital AI

WhiteMatter raises $20 million Series A for AI agent oversight

WhiteMatter raised a $20 million Series A, Business Insider reported. Founded in 2024, the startup builds AI agents designed to oversee other AI agents, monitoring their file access, software use, decision making, and actions. It calls these oversight systems "guardian" agents.

Policy Products

Italian privacy authority fines Meta €805 million over gallery access

The Italian Data Protection Authority imposed a fine of €805 million ($887 million) on Meta Platforms Inc. The penalty concerns Meta's access to photos and videos in the galleries of Android users without those users' explicit consent.