Mesh LLM: distributed AI computing on iroh

Mesh LLM announced a distributed AI computing system that pools GPUs and memory across multiple machines using the iroh networking protocol. The system exposes a single OpenAI-compatible API at localhost:9337/v1, allowing users to run larger models without purchasing larger GPUs by splitting models across several machines in a pipeline mode called Skippy. The architecture uses iroh endpoints for authenticated QUIC connections with NAT traversal and relay fallback, eliminating the need for a central server. A gossip layer controls mesh admission, version compatibility, and peer trust. The lightweight software is about 18 MB and includes a catalog of over 40 models ranging from half-a-billion-parameter models to 235B mixture-of-experts models. Users can join a public mesh or configure private deployments. A mobile app built on iroh's Swift SDK is planned to support the emerging ACP agent standard. The project aims to provide more peer-to-peer computing, fewer closed servers, and no vendor lock-in.