The Engineering Manager - SDET will oversee AI application quality assurance, focusing on test automation, model validation, and AI product testing, while mentoring the SDET team.
About HighLevel:
HighLevel is an AI powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth. We are proud to support a global and growing community of over 2 million businesses, comprised of agencies, consultants, and businesses of all sizes and industries. HighLevel empowers users with all the tools needed to capture, nurture, and close new leads into repeat customers. As of mid 2025, HighLevel processes over 4 billion API hits and handles more than 2.5 billion message events every day. Our platform manages over 470 terabytes of data distributed across five databases, operates with a network of over 250 microservices, and supports over 1 million hostnames.
Our People
With over 1,500 team members across 15+ countries, we operate in a global, remote-first environment. We are building more than software; we are building a global community rooted in creativity, collaboration, and impact. We take pride in cultivating a culture where innovation thrives, ideas are celebrated, and people come first, no matter where they call home.
Our Impact
As of mid 2025, our platform powers over 1.5 billion messages, helps generate over 200 million leads, and facilitates over 20 million conversations for the more than 2 million businesses we serve each month. Behind those numbers are real people growing their companies, connecting with customers, and making their mark - and we get to help make that happen.
About the Role:
We are looking for a highly skilled SDET Manager to spearhead the quality of AI application development at HighLevel. This role is very critical in ensuring the relevancy, consistency, reliability, contextual recall and fairness of our AI platform.
Ideal candidates will have a combination of technical depth in AI/ML, Agentic AI along with deep expertise in test automation covering model validation, agentic workflows, and quality leadership.
Requirements:
- Overall 8-12 years of experience in which minimum of 1 year in testing AI products
- Good experience in Prompt Engineering, testing AI applications like conversations AI, voice AI, workflow AI
- Architect and scale test automation frameworks for APIs, micro-services, ML pipelines, and agentic AI workflows
- Experience in testing RAG based applications using frameworks like TruLens, RAGAs
- Leverage AI to improve coverage and efficiency across testing lifecycle
- Hands-on experience testing agentic AI frameworks (e.g., Agentic Studio, LangChain, AutoGen, crewAI)
- Strong programming skills in Python and Java or TypeScript/JavaScript
- Proven expertise in test automation frameworks (Selenium, Playwright, Cypress etc.)
- Experience with cloud platforms (AWS, GCP) and CI/CD pipelines
Responsibilities:
- Review testing strategies for AI application and fine tune strategy to minimize issues in production
- Establish evaluation pipelines for detecting hallucination, context retention, latency in multi model environments
- Generate right data set to simulate customer usage scenarios
- Test for data, model and code issues
- Define relevant KPIs for AI application testing
- Non Functional testing - Performance, Scalability, Reliability, Security
- Implement A/B testing at appropriate levels to identify customer impact
- Run down defect triaging for all customer issues to identify root cause and expand test coverage to mitigate gaps.
- Ensure quality deliverables from AI teams
- Hire top talent and mentor SDET team on AI testing skills
EEO Statement:
The company is an Equal Opportunity Employer. As an employer subject to affirmative action regulations, we invite you to voluntarily provide the following demographic information. This information is used solely for compliance with government record-keeping, reporting, and other legal requirements. Providing this information is voluntary and refusal to do so will not affect your application status. This data will be kept separate from your application and will not be used in the hiring decision.
#LI-Remote #LI-NJ1
Top Skills
AWS
Cypress
GCP
Java
JavaScript
Playwright
Python
Selenium
Typescript
Similar Jobs
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
The Software Engineer will assist with user requirements, software development, testing, and production support for applications and platforms, utilizing Agile and DevSecOps methodologies.
Top Skills:
.Net FrameworkAppdynamicsAzure AppinsightsAzure DevopsAzure SqlData LakeElasticJavaPowershellPythonRest ApiSplunkSQLVeracode
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Assist internal and external customers with technology issues in the DnA eReporting Production Support team, manage ticket resolutions, promote Agile best practices, and ensure timely issue mitigation.
Top Skills:
CognosCore JavaMs SqlRest ApiService NowSQLUdb
Fintech • Information Technology • Insurance • Financial Services • Big Data Analytics
Join MetLife as an Associate Software Platform Engineer II, participating in hackathons and contributing to financial services technology.
What you need to know about the Kolkata Tech Scene
When considering the industries shaping India's tech scene, gaming might not immediately come to mind. However, in the last decade, increased internet usage and greater access to mobile devices have catapulted the industry to new heights, with Kolkata-based companies like Virtualinfocom, Red Apple Technologies and Digitoonz, at the forefront, driving the design and animation of new gaming titles for players.
