Researchers introduced Minions, a system that uses idle Apple Silicon devices to power AI workloads. The paper notes, “The platform is particularly valuable in environments where centralised GPU resources are constrained.” This setup splits tasks across many Macs, helping teams save money by using existing hardware for local AI inference.
In a paper titled ‘Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices,’ the researchers explain that when GPU resources are constrained, teams in enterprises experience long queue times for inference tasks and increased operational cost. At the same time, a large population of macOS devices with Apple Silicon have high-performance GPUs capable of performing those tasks.
The key components of this system architecture are a Minion Server, a Minion Executor and a Minion Watchdog. The Minion Server splits submitted tasks into subtasks and queues them, which the Minion Executor fetches as the primary GPU execution agent on each connected Mac. The Minion Watchdog monitors active subtask executions and checks for failures.
“Minions currently supports two categories of GPU inference workloads on macOS devices. For embedding generation, Minions leverages open-source transformer models from HuggingFace. Executors use the HuggingFace transformers Python API to load supported embedding models and generate vector representations locally on Apple Silicon GPUs. For large language model (LLM) completion workloads, Minions integrates with Ollama, an open-source framework for running LLMs locally on macOS devices. Ollama is installed and executed on participating devices, enabling local inference without remote API (Application Programming Interface; this defines how apps communicate with each other) calls,” the paper explains.
If by chance an executing Mac becomes unresponsive due to a number of reasons including network connectivity loss, device sleep or power-off, or intentional termination of a task by the device user for running other tasks, the Watchdog releases the affected subtask back to the queue, allowing other healthy executors to re-acquire and execute it.
This paper comes at a time when AI compute infrastructure faces not just a scaling concern but also spiralling costs for adding new hardware. “The platform is particularly valuable in environments where centralised GPU resources are constrained, enabling teams to unlock substantial compute capacity at zero cost using existing hardware,” the paper notes. The Minions system may be relevant for organisations to run AI on-premise and researchers say that Minions support will be extended for distribution model training in addition to inference, which would allow greater community inclusion that would benefit the broader ecosystem.