Software Engineer, Inference Infrastructure
At a glance
Mid-level ML Platform & Ops role at Rhoda AI. Mountain View · full-time · $175,000–$280,000 base.
aimlroles summary, based on the employer's posting.
What you'll do
- Run large model workloads across cloud and on-prem systems, keeping deployment infrastructure scalable and reliable
- Build Kubernetes pipelines that deploy distributed machine-learning workloads
- Schedule work across GPU clusters, balancing utilization and workload demands against cost and performance
- Connect machine-learning frameworks and serving systems for research and production deployments
- Create deployment, versioning, and monitoring tools that help teams iterate on models
What you bring
- At least three years working in machine-learning infrastructure, MLOps, or distributed systems
- Strong Kubernetes skills and experience deploying workloads in containers
- Experience coordinating GPU resources and scheduling large distributed jobs
- Hands-on work with cloud providers and hybrid cloud or on-prem setups
- Familiarity with machine-learning frameworks and model-serving tools, plus the ability to debug issues across the stack
Who this fits
This mid-level engineering role suits someone who wants to operate model deployment infrastructure across cloud and on-prem environments. The position is based in Mountain View.
From the employer
At Rhoda AI, we’re building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.
We're looking for a Software Engineer to help build and operate the systems that power our model deployment stack. You'll be responsible for running large foundation models efficiently and reliably across cloud and on-prem environments, with a focus on resource management, scheduling, and infrastructure scalability.
What You'll Do
Design and operate large-scale infrastructure to run model workloads across cloud and on-prem environments
Build and maintain Kubernetes-based deployment pipelines for managing distributed ML workloads
Own resource scheduling and orchestration across GPU clusters — optimizing utilization, workload balancing, and cost-performance tradeoffs
Integrate and manage ML frameworks and model serving systems (e.g., Triton, Ray Serve, TorchServe) across research and production use cases
Build tooling for model deployment, versioning, and observability to support fast iteration cycles
Contribute to the reliability and scalability of the infrastructure stack as model complexity and deployment footprint grow
What We're Looking For
3+ years of experience in ML infrastructure, MLOps, or distributed systems
Strong proficiency with Kubernetes and containerized deployment pipelines
Experience with GPU orchestration and resource scheduling across large distributed jobs
Experience with cloud providers (e.g., AWS, GCP) and hybrid cloud/on-prem infrastructure
Familiarity with ML frameworks (e.g., PyTorch, JAX) and model serving tools (e.g., Triton, Ray Serve, TorchServe)
Strong debugging instincts and ownership mentality — comfortable driving issues to resolution across the stack
Nice to Have (But Not Required)
Experience with streaming systems or high-throughput data transport (e.g., Kafka, gRPC, NATS)
Background in networking, low-latency systems, or network-aware scheduling
Experience with edge/cloud hybrid deployment patterns and the latency constraints that come with them
Familiarity with on-robot or embedded inference environments
Experience with large-scale cluster topology and scheduling systems (e.g., SLURM, Ray, Volcano)
Why This Role
Own the infrastructure layer that connects our foundation models to real robot behavior — a direct line between your work and what the robot does in the world
Be part of building the infrastructure stack for one of the most technically ambitious robotics companies in the world