At Supervity, we're looking for a Lead AI Infrastructure & Distributed Systems Engineer to architect a high-performance local GPU training cluster capable of training 26B-parameter AI models—without relying on unlimited cloud compute.
If solving hardware bottlenecks, distributed systems, and large-scale AI training excites you, keep reading...
What You'll Build
You'll design, scale, and optimize high-speed local GPU training rings using consumer GPUs (RTX 4090s/5090s and PCIe server nodes) to efficiently train large language models.
Your work will include:
🔹 Designing distributed training architectures using PyTorch FSDP, DeepSpeed (ZeRO-3), and Megatron-LM
🔹 Implementing Pipeline Parallelism (PP) and Tensor Parallelism (TP) to distribute 48–64 transformer layers across multi-node GPU clusters
🔹 Optimizing gradient checkpointing, CPU RAM offloading, NCCL communication, Linux performance tuning, and 10GbE/100GbE networking to eliminate training bottlenecks
🔹 Maximizing PCIe Gen4/Gen5 bandwidth, thermal efficiency, and cluster health across mixed consumer-grade GPU environments
🎯 What We're Looking For
✔ 3+ years of experience building distributed ML training infrastructure using FSDP, DeepSpeed, Horovod, or similar frameworks
✔ Strong proficiency in Python and familiarity with low-level C/C++ configurations
✔ Expertise in Linux systems tuning, NCCL optimization, PCIe routing, and multi-node GPU communication
⭐ Bonus Points If You've Worked On
• University AI research labs
• Decentralized AI projects like Exo or Petals
• HPC clusters
• GPU rendering or mining farms
• Custom-built AI infrastructure
ML training infrastructure
FSDP
DeepSpeed
Horovod
Python
c/C++
Linux system tuning
NCCL optimization
PCle routing
multi-node GPU communication