Machine Learning Engineer — Training Optimization
At a glance
Mid-level Machine Learning Engineering role at Featherless AI. Remote · full-time.
Pay not stated Machine Learning Engineer salary
aimlroles summary, based on the employer's posting.
What you'll do
- Speed up large-model training by improving throughput, convergence, stability and compute use
- Refine data, model and pipeline parallelism, then tune optimizers, schedules, batch sizes and numerical precision
- Profile training runs to find bottlenecks and reduce runtime and compute costs
- Work with researchers on training approaches that account for model architecture
- Build reliable training systems for checkpointing, fault recovery and reproducible runs; track performance and test new techniques
What you bring
- You have substantial hands-on experience training large neural networks, such as LLMs, and improving how they train
- You understand backpropagation, optimization methods, training behavior and distributed systems for machine learning
- PyTorch experience is required, along with comfort working around GPU, memory and networking limits
- You can turn research concepts into production-ready code
Who this fits
This remote role suits a mid-level engineer who wants to move between research collaboration and production training systems. It is an individual technical role with ownership of training performance.
From the employer
About the Role
We’re looking for an ML Engineer focused on training optimization to help us scale and improve large-scale model training. You’ll work at the intersection of research and production, optimizing training pipelines for speed, stability, and cost—while collaborating closely with researchers pushing model architecture and capability forward.
This is a high-impact role with real ownership: your work directly affects how fast we can iterate, how large we can scale, and how efficiently we deploy new models.
What You’ll Do
Optimize large-scale model training pipelines (throughput, convergence, stability, and cost)
Improve distributed training strategies (data, model, and pipeline parallelism)
Tune optimizers, schedulers, batch sizing, and precision (bf16 / fp16 / fp8)
Reduce training time and compute cost via profiling, bottleneck analysis, and systems-level improvements
Collaborate with researchers on architecture-aware training strategies
Build and maintain robust training infrastructure (checkpointing, fault tolerance, reproducibility)
Evaluate and integrate new training techniques (e.g. gradient checkpointing, ZeRO, FSDP, custom kernels)
Own training performance metrics and continuously push them forward
What We’re Looking For
Strong experience training large neural networks (LLMs or similarly large models)
Hands-on experience with training optimization (not just model usage)
Solid understanding of:
Backpropagation, optimization algorithms, and training dynamics
Distributed systems for ML training
Experience with PyTorch (required)
Comfort working close to hardware (GPUs, memory, networking constraints)
Ability to move fluidly between research ideas and production-ready code
Nice to Have
Experience with large-scale distributed training (multi-node, multi-GPU)
Familiarity with DeepSpeed, FSDP, Megatron, or custom training stacks
Experience optimizing training on AMD or NVIDIA GPUs
Contributions to open-source ML infrastructure or research codebases
Exposure to non-Transformer architectures (RNNs, hybrid models, etc.)
Why Join Us
Real ownership at Series-A stage — your work shapes the company’s trajectory
Work on cutting-edge models and training systems at scale
Small, highly technical team with fast feedback loops
Strong emphasis on engineering quality and research rigor
Competitive compensation + meaningful equity