SDET Automation Engineer AI

We are looking for a highly skilled Senior QA Engineer/Lead with strong experience in testing Generative AI and Agentic AI solutions across enterprise platforms. In this role, you will ensure the quality, reliability, and performance of AI-powered capabilities such as autonomous agents, conversational experiences, RAG-based intelligence, and workflow automation. You will work closely with cross-functional teams to define testing strategies, build AI-aware automation frameworks, and validate implementations at scale across OCI, ServiceNow, Oracle Fusion, and custom enterprise applications
Responsibilities:

• Design and execute comprehensive test strategies to ensure the quality, reliability, and effectiveness of Generative AI and Agentic AI capabilities across enterprise use cases.
• Validate AI-driven workflows across ServiceNow ITSM modules, Oracle Fusion processes, and other enterprise applications, leveraging automation and conversational interactions.
• Test RAG-based intelligence for reliable context grounding and consistent retrieval performance
• Test Agentic AI behaviors to ensure appropriate task execution, autonomous decision quality, and safe workflow orchestration, and evaluate performance, scalability, and reliability of AI agents integrated across cloud and enterprise environments
• Build and maintain automation frameworks for API, functional, web, and backend testing. Integrate automation into CI/CD pipelines (Azure DevOps / GitHub / Jenkins)
• Create and maintain QA documentation, including test plans, test cases, traceability matrices, and execution reports
• Conduct performance and reliability testing for AI endpoints and automations across workloads
• Diagnose failures, analyze patterns, and drive issue resolution by collaborating with platform and engineering teams
• Drive continuous improvement by adopting advanced AI testing practices and metrics suitable for nondeterministic systems
• Stay current with advancements in AI testing tools, automation accelerators, and emerging testing standards

Requirements:

• 8+ years of experience in QA with 2+ years of testing Generative AI / Agentic AI / ML-based systems
• Hands-on experience validating AI workflows in one or more enterprise ecosystems such as ServiceNow, Oracle Fusion, or cloud-native platforms
• Strong scripting and automation skills (Python preferred) with experience in frameworks like Selenium, Playwright, REST Assured, or PyTest
• Knowledge of LLM architectures, NLP behavior, RAG workflows, and vector retrieval validation
• Proficiency with Git, CI/CD pipelines, Docker/Kubernetes for enterprise automation support
• Expert in QA including test design, defect lifecycle, and quality metrics
• Knowledge of Responsible AI, bias/fairness testing, and governance practices
• Strong communication skills to explain testing results and quality improvements clearly
• Ability to guide and support team members on testing best practices