About this role
- Develop and execute testing strategies for AI, Machine Learning (ML), and Generative AI solutions to ensure accuracy, reliability, performance, and business value.
- Define and implement robust AI test frameworks, methodologies, and automation solutions covering functional, integration, API, system, end-to-end, model validation, data validation, prompt validation, and adversarial testing scenarios.
- Establish comprehensive validation processes for AI model outputs against business requirements, acceptance criteria, Responsible AI principles, regulatory standards, and organizational governance policies.
- Drive quality assurance across the complete AI lifecycle, including data ingestion, data preparation, feature engineering, model training, evaluation, deployment, inference, monitoring, retraining, and reporting.
- Assess AI solution performance across dimensions such as accuracy, relevance, consistency, explainability, fairness, robustness, and reliability.
- Integrate AI testing activities and test automation framework & scripts with continuous integration, CI/CD pipelines and tools to support continuous testing and deployment of AI solutions.
- Collaborate with developers, business analysts, product owners, and project managers to define testing requirements, success criteria, and quality measures.
- Manage the execution of automated test suites, monitor the test results and collaborate with teams to troubleshoot and resolve the issues promptly.
- Lead defect triage, root cause analysis, and quality risk management activities, ensuring timely resolution of issues impacting AI model performance and business outcomes.
- Champion the adoption of AI-enabled testing tools, intelligent automation techniques, synthetic test data generation, and predictive quality engineering practices to improve testing efficiency and coverage.
- Drive research and evaluation of emerging AI testing methodologies, LLM evaluation frameworks, model benchmarking techniques, automation technologies, and industry best practices, providing strategic recommendations for continuous improvement.
- Establish and govern enterprise standards, processes, controls, and best practices for AI testing, validation, Responsible AI compliance, and model quality assurance across multiple AI initiatives.
- Design, develop, execute, and maintain test cases and test scripts for AI, ML, and Generative AI applications.
- Validate data quality, data integrity, and data lineage across AI systems and supporting platforms.
- Lead functional, integration, API, regression, user acceptance, and end-to-end testing of AI-enabled solutions.
- Assess AI model outputs for accuracy, consistency, bias, hallucinations, explainability, fairness, and alignment with business requirements and responsible AI standards.
- Execute prompt testing, response validation, and scenario-based testing for Generative AI solutions.
- Execute model validation and inference testing activities, ensuring AI solutions perform consistently across diverse datasets, use cases, and operating conditions.
- Monitor and analyse test execution results, quality metrics, and model performance indicators, producing actionable insights and comprehensive test reports for stakeholders.
- Identify, document, track, and drive resolution of defects, model performance issues, data quality concerns, and AI-specific risks throughout the testing lifecycle.
- Collaborate with business analysts, developers, and other stakeholders to understand requirements and deliver high-quality AI solutions.
- Support test data preparation, environment validation, and model release verification activities.
- Contribute to continuous improvement initiatives by identifying opportunities to enhance AI testing frameworks, processes, and automation capabilities.
- Provide guidance and knowledge sharing to project teams on AI testing methodologies, Responsible AI validation, test automation approaches, and emerging industry practices.
- Mentor junior testers and contribute to building AI testing capabilities across teams through coaching, standards adoption, and continuous learning initiatives.
- Identify quality, model, data, security, privacy, ethical, and compliance risks associated with AI solutions and proactively support mitigation activities.
- Validate adherence to Responsible AI principles, regulatory requirements, organizational policies, and governance standards.
- Conduct and support the assessment and reporting of AI-related risks, including model bias, explainability, fairness, and data quality concerns.
- Maintain appropriate test evidence, validation reports, defect records, and audit documentation to support governance and compliance requirements.
- Escalate risks, issues, and control gaps that may impact AI solution quality, regulatory compliance, or business outcomes.
- 5+ years of experience in Quality Engineering, Test Automation, and AI-driven testing, with preference for candidates having 4+ years of experience in Retirement Solutions, Financial Services, or BFSI domains.
- Strong experience in designing, developing, and executing AI-powered test automation frameworks and quality assurance solutions using tools such as Selenium, Appium, UFT/OpenText (at least one mandatory), Python, Maven, Postman, REST APIs, SoapUI, Cortex, and Snowflake.
- Hands-on experience leveraging AI-assisted testing tools and technologies for test case generation, test optimization, defect prediction, intelligent test execution, and quality analytics.
- Strong understanding of software testing principles, methodologies, and best practices, including test strategy, test planning, test design, test execution, defect management, reporting, and test governance.
- Experience validating AI/ML-enabled applications, including model testing, data quality validation, prompt testing and optimization, response evaluation, bias and fairness assessment, hallucination detection, adversarial testing, explainability validation, and performance assessment of AI-powered systems.
- Solid understanding of Software Development Life Cycle (SDLC) and Software Quality Assurance (SQA) processes across Agile, Waterfall, and DevOps delivery models.
- Experience working with CI/CD pipelines and DevOps practices using tools such as Jenkins, Azure DevOps, Azure Repos, Bitbucket, Git, Docker, and Kubernetes.
- Strong analytical and problem-solving skills with the ability to define quality standards, establish AI testing governance, track AI model evaluation metrics and quality KPIs, automate validation processes, and ensure reliable, scalable, and secure AI-driven applications.
- Experience with cloud platforms, data validation, API testing, integration testing, and complex enterprise environments is highly desirable.
- Lead AI testing strategy and provide technical leadership for AI testing initiatives.
- Collaborate with Developers, Testers, and Business Analysts to design test cases and develop them using AI and automation testing tools.
- Perform functional, integration, system, regression, and API testing using tools such as Postman and REST Assured.
- Manage defects through JIRA and Azure DevOps and produce comprehensive test evidence documentation.
- Develop and execute end-to-end test automation solutions covering the entire data lifecycle, from raw data through to the reporting layer.
- Mentor team members and support the onboarding, training, and upskilling of testers in AI and automation testing tools, frameworks, and best practices.
- Strong analytical, problem-solving, and troubleshooting skills.
- Strong communication, collaboration.
- Strong attention to detail and quality orientation.
- Demonstrate ability to solve technical problems and implement innovative solutions.
- Ability to work independently and as part of a team.
- Ability to learn new AT testing tools, test automation tools, technologies, and trends quickly and effectively.
- Ability to handle multiple tasks and projects simultaneously and prioritize them
Through our two businesses MUFG Retirement Solutions and MUFG Corporate Markets, we partner with a diversified portfolio of global clients to provide robust, efficient and scalable services, purpose-built solutions and modern technology platforms that deliver world class outcomes and experiences.
A member of MUFG, a global financial group, we help manage regulatory complexity, improve data management and connect people with their assets, through exceptional user experience that leverages the expertise of our people combined with scalable technology, digital connectivity and data insights.
Our MUFG Retirement Solutions division combines its proprietary technology, process and people to deliver a comprehensive financial data solution to its superannuation clients. We support clients across all superannuation fund sectors including government, industry, retail and corporate.
By accompanying our core member and employer administration with a full range of value-added services, we offer the most comprehensive superannuation administration solution on the market. The scale, adaptability and ease of use of our proprietary systems, in conjunction with our integrated analytics offering, allow us to innovate and grow with the needs of our clients.
MUFG Pension & Market Services is building a dynamic, client focused, caring and inclusive culture that is built on the foundations of an entrepreneurial spirit, effective risk management, empathy and trust, and underpinned by its core values.
We are an inclusive employer whose people work collaboratively. We encourage, support and value the various talents and perspectives of our people and promote a flexible and blended work environment where our people can thrive and their wellbeing is supported. We know that diversity drives better client outcomes, continuous improvement, and growth. Be part of the MUFG Pension & Market Services journey and together we will achieve our full potential.
We treat all individuals fairly and equitably and do not discriminate on the basis of diverse characteristics including, but not limited to gender, gender identity, sexual orientation, age, ethnicity, cultural background, physical abilities/disabilities, religious or political belief, marital or family status or carers responsibilities.
Candidates must have the relevant work rights to be considered for an opportunity at MUFG Pension & Market Services. Successful applicants will be required to complete background screening prior to commencement of employment.
Tired of cold applications?
Sign up with Clera and we'll reach out the moment a role actually fits you — no more spraying applications into the void.
Know someone who'd be great for this?