Senior LLM Inference Performance & Evaluation Engineer
At a glance
Senior AI Engineering role at Bitdeer Technologies Group. Singapore · full-time.
Pay not stated AI Engineering salary
aimlroles summary, based on the employer's posting.
What you'll do
- Build benchmark workflows for TTFT, inter-token delay, output volume, request latency, concurrency, errors and GPU utilization
- Create launch checks for API compatibility, streaming, tool calls, reasoning, multimodal output and long-context handling
- Maintain generated, replayed and customer-like traffic tests for both steady workloads and bursts
- Assess model, runtime and provider options, then advise on routing, fallback, pricing and capacity
- Catch regressions through CI/CD and staging automation; work with runtime engineers and SRE on bottlenecks, SLOs and alerts
What you bring
- At least five years in production ML infrastructure, performance work, model evaluation, QA automation or backend testing
- You understand LLM serving measures, including TTFT, TPOT or ITL, request latency, token throughput, concurrency and GPU utilization
- Strong Python ability for automating benchmark and evaluation work
- You can design statistically meaningful tests and explain tradeoffs among quality, latency, reliability, cost and customer experience
Who this fits
This senior individual contributor role suits someone focused on evaluating and improving LLM serving. You’ll collaborate with runtime engineers and SRE, and share findings with engineering, product and business stakeholders. The role is listed in Singapore.
From the employer
About Bitdeer:
Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud.
Bitdeer is committed to providing comprehensive Bitcoin mining solutions for its customers. Apart from designing industry-leading ASIC chips and manufacturing mining rigs, the Group handles complex processes involved in computing across the value chain. This includes equipment procurement, transport logistics, datacenter design and construction, equipment management, and network and facility operations. Bitdeer also offers advanced cloud capabilities to customers with a high demand for artificial intelligence.
Headquartered in Singapore, Bitdeer operates globally with a diversified 3 GW energy portfolio, and deploys Bitcoin mining and HPC datacenters in the United States, Bhutan, Norway, Canada, Malaysia, and Ethiopia.
What you will be responsible for:
- Build benchmark pipelines for TTFT, inter-token latency, output throughput, request latency, concurrency curves, error rate, and GPU utilization.
- Create model launch gates covering OpenAI/Anthropic compatibility, streaming behavior, tool calling, reasoning output, multimodal behavior, and long-context cases.
- Maintain representative workloads using synthetic, replayed, and customer-like traffic patterns for both steady-state and burst scenarios.
- Compare model/runtime/provider options and publish clear recommendations for routing, fallback, pricing, and capacity decisions.
- Automate regression detection in CI/CD and staging so model, runtime, or configuration changes do not silently degrade quality, latency, or cost.
- Work with runtime engineers to identify bottlenecks and verify improvements; work with SRE to convert benchmark results into SLOs and alert thresholds.
How you will stand out:
- 5+ years in ML infrastructure, performance engineering, model evaluation, QA automation, or backend testing for production systems.
- Experience with LLM serving metrics such as TTFT, TPOT/ITL, request latency, token throughput, concurrency, and GPU utilization.
- Strong Python skills for benchmark/evaluation automation; Go experience is a plus for integrating with platform services.
- Familiarity with OpenAI/Anthropic APIs, vLLM/Dynamo/SGLang/Triton-style servers, and Kubernetes-based test environments.
- Able to design statistically meaningful tests and communicate tradeoffs between model quality, latency, reliability, cost, and customer experience.
- Experience building dashboards, reports, and release gates for engineering, product, and business stakeholders.
What you will experience working with us:
- A culture that values authenticity and diversity of thoughts and backgrounds;
- An inclusive and respectable environment with open workspaces and exciting start-up spirit;
- Fast-growing company with the chance to network with industrial pioneers and enthusiasts;
- Ability to contribute directly and make an impact on the future of the digital asset industry;
- Involvement in new projects, developing processes/systems;
- Personal accountability, autonomy, fast growth, and learning opportunities;
- Attractive welfare benefits and developmental opportunities such as training and mentoring.
--------------------------------------------------------------------
Bitdeer is committed to providing equal employment opportunities in accordance with country, state, and local laws. Bitdeer does not discriminate against employees or applicants based on conditions such as race, colour, gender identity and/or expression, sexual orientation, marital and/or parental status, religion, political opinion, nationality, ethnic background or social origin, social status, disability, age, indigenous status, and union.