Senior Inference Optimization ML Engineer
At a glance
Senior Machine Learning Engineering role at Rhoda AI. Mountain View · full-time · $175,000–$250,000 base.
aimlroles summary, based on the employer's posting.
What you'll do
- Measure production model latency, throughput and efficiency, then find and address the causes of slowdowns
- Break performance down across compute, memory bandwidth and I/O to prioritize bottlenecks across model families
- Use quantization, pruning, distillation, fusion and compilation to make large models run more efficiently
- Tune attention, KV caching and memory layouts for vision, video, language and proprioception models
- Build benchmarks that track latency and throughput and flag regressions across model versions
What you bring
- At least three years in inference optimization, ML systems or a closely related area
- Hands-on PyTorch experience, plus a strong grasp of compute, memory and I/O limits during large-model inference
- Experience applying model optimization methods such as quantization, distillation, pruning and compilation
- You can diagnose slow inference, design experiments and verify whether changes improve performance
Who this fits
This senior individual-contributor role suits someone who can take ownership of inference performance and work with research engineers to prepare models for deployment. The work spans cloud and on-robot targets, with a Mountain View location.
From the employer
At Rhoda AI, we’re building the next generation of generalist intelligent robots. We own the full robotics stack from high-performance hardware and robot systems to the infrastructure and state-of-the-art foundation world models that control our robots. Our robots are designed to be generalists capable of operating in complex, real-world environments and handling long-tail edge cases, made possible by our cutting edge research and end-to-end system design. We've raised over $450M and are investing aggressively in model research, infrastructure, hardware development, and manufacturing scale-up to make generalist robotics a reality.
We're looking for an Inference Optimization MLE to help build and operate the systems that make our foundation models run fast and efficiently in production. You'll be responsible for squeezing maximum performance out of large multimodal models, across cloud and on-robot deployment targets. You will working closely with research and robotics teams to close the gap between training and real-world deployment.
What You'll Do
Own inference performance end-to-end — diagnose and improve latency, throughput, and efficiency of large foundation models in production
Build systematic performance attribution: latency decomposition (compute vs. memory bandwidth vs. I/O), bottleneck identification, and prioritization across model families
Apply and develop optimization techniques including quantization, pruning, distillation, operator fusion, and model compilation (e.g., TensorRT, torch.compile, XLA)
Optimize attention mechanisms, KV caching, and memory layouts for large multimodal models (vision, video, language, proprioception)
Work with kernel-level tooling (e.g., CUDA, Triton) to identify hotspots and implement or tune custom kernels where needed
Build benchmarking and regression detection infrastructure: latency baselines, throughput curves, and automated detection of performance regressions across model versions
Collaborate closely with research engineers to translate model innovations into optimized, deployment-ready implementations
What We're Looking For
3+ years of experience in inference optimization, ML systems, or a closely related field
Deep hands-on experience with modern ML stacks (PyTorch required; JAX a plus)
Strong understanding of compute, memory bandwidth, and I/O bottlenecks in large model inference
Experience with model optimization techniques: quantization (INT8/FP8/AWQ), distillation, pruning, and compilation
Familiarity with inference serving frameworks (e.g., Triton, TensorRT, vLLM, TorchServe)
Exceptional debugging and measurement ability: turn "inference is slow" into clear bottlenecks, experiments, and validated improvements
High ownership mindset and comfort in a fast-moving environment
Nice to Have (But Not Required)
GPU kernel or compiler-level experience (CUDA, Triton, graph capture, operator fusion)
Experience with multimodal or video model inference (variable-length sequences, packing/bucketing)
Familiarity with edge/cloud hybrid deployment patterns and on-robot inference constraints
Experience with speculative decoding, continuous batching, or other LLM serving optimizations
Background in streaming or low-latency systems relevant to real-time robot control
Why This Role
Direct leverage on research velocity and real-world robot performance — every efficiency gain you make accelerates model iteration and tightens the loop between model and robot behavior
Own the optimization layer that determines how quickly and efficiently our foundation models run in the real world — high ownership, high impact, small elite team