Mesh LLM Distributed GPU Cluster API

Mesh LLM turns your GPU cluster into a single OpenAI-compatible API endpoint that routes or splits models across nodes, with support for dozens of GGUF families and experimental MoA arbitration. It’s still experimental, but it ships binaries for Windows, macOS, and Linux (CPU/GPU) and lets you join the public mesh to share compute.

Source

GitHub - Mesh-LLM/mesh-llm: Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat. · GitHub

Our indexed copy