AVP/VP, AI/ML Model Validation Engineer, Data Management Office
At a glance
VP-level AI Engineering role at Sumitomo Mitsui Banking Corporation. Singapore.
Pay not stated AI Engineering salary
aimlroles summary, based on the employer's posting.
What you'll do
- Build test plans for statistical, machine-learning, language and agentic AI models, then run the checks
- Test model behavior and workflows through functional, regression and scenario-based exercises
- Measure accuracy, fairness, resilience and drift, and assess data, features, tool use and multi-step tasks
- Create automated evaluation and regression pipelines, and check API and tool integrations in production-like settings
- Trace failures, document findings, maintain dataset metadata and support proof-of-concept and analytics work
What you bring
- At least four years in model testing, QA, AI/ML evaluation, CI/CD, MLOps, data engineering or a related technical role
- You can design AI/ML test plans and automated regression systems, then assess performance, fairness, robustness and drift
- You are proficient in Python, including PySpark, MLlib and pytest, as well as R and SQL
- You can investigate faults with logs and traces, assess data and feature quality, and explain findings and fixes
Who this fits
This suits a senior technical contributor who works with data scientists and teams across engineering, IT and governance. The work spans model evaluation, data management and analytics enablement.
From the employer
Responsibilities
• Define and execute comprehensive test strategies covering statistical, ML, LLM and agentic AI models.
• Perform functional, regression and scenario‑based testing of model behaviours and workflows.
• Conduct AI/ML evaluations including accuracy checks, bias/fairness assessment, robustness analysis and drift detection.
• Assess end‑to‑end model workflows including data inputs, feature transformations, task completion, tool‑use accuracy and multi‑step reasoning.
• Design and maintain automated test and evaluation pipelines, including benchmarking and regression frameworks.
• Validate API and tool‑integration behaviour in production‑like environments, identifying dependency or orchestration issues.
• Diagnose issues using observability, logging, tracing and debugging tooling, and document findings clearly.
• Collaborate with data scientists across departments to understand modelling intent, feature logic and expected behaviours.
• Perform data‑management tasks to support AI/ML model testing, including maintaining metadata, documenting key datasets and ensuring clarity of data inputs.
• Contribute to AI/ML proof‑of‑concept (POC) initiatives to strengthen evaluation methodologies and support innovation.
• Support data‑management/analytics initiatives such as the Analytics Workbench and contribute to AI/ML/data analytics enablement.
Requirements
• Minimum 4 years of relevant experience in model testing, QA/QC, AI/ML evaluation, CI/CD, MLOps, data engineering, or related technical roles.
• Proficiency in Python (especially PySpark, MLlib, pytest), R and SQL; knowledge of Scala, Rust, Java, JS or C++ is a plus.
• Experience designing and executing test strategies for ML/AI models, including automated pipelines and regression frameworks.
• Ability to evaluate statistical, ML and LLM models using performance, bias, robustness and drift metrics.
• Strong ability to assess feature engineering logic, dataset integrity, workflow reliability and tool‑integration behaviours.
• Experience troubleshooting using logs, traces and debugging tools to identify root‑cause issues.
• Strong documentation and communication skills to articulate findings, risks and remediation requirements.
• Ability to collaborate effectively with data science, engineering, IT and governance functions.
• Understanding of Responsible AI concepts and quality expectations for production‑ready AI/ML systems.