EXO
Network AI clusters to model and move on
Few things can rival the computing demand of AI models. Tam Hanna helps you spread the load with a cluster of affordable systems.
Credit: https://github.com/exo-explore/exo
OUR EXPERT
Tam Hanna hates all things Raspberry Pi and will mention the Orange Pi at every chance! Follow him at www.instagram. com/tam.hanna/
QUICK TIP
When originally submitted,Exoran on the Raspberry Pi and Orange Pi. Post-submission a regression seems to have broken this. Typically we’d boot this along an issue or two until it was fixed, but here we are not wanting to waste a good project…
Mainframes and similar large systems command a premium price. Since time immemorial, PC enthusiasts have been looking for a way to combine a large number of inexpensive computers. The arrival of high-performance SoCs such as the RK3588 made single-board computers a viable alternative for cluster applications.
Exo is a cluster system with a wide array of promises. The developers pledge to support both Mac OS and Linux, and promise good performance even when running on extremely slow individual systems. What’s more, the system promises a comfortable web interface, thereby ensuring that model management does not become a chore.
We are taking the developer team at its word. To find out more, we are putting together a cluster consisting of various machines, from process computers to a machine with an Nvidia GPU.
Start the control terminal
We are starting with the most powerful machine we’ll be using: our trusty MSI mobile workstation, whose RTX2060 GPU with 6GB of VRAM might just be enough to host a decent fragment of a model. Ubuntu 22.04 LTS shall be used as the operating system. The documentation at https://github.com/exo-explore/exo reveals that the developer team recommends compilation from source.