GenAI Test Automation Lead

About the Position

We are seeking a GenAI Test Automation Lead with over 10 years of QA experience to design scalable, Python-driven automation frameworks for GenAI applications in Toronto, ON.

Responsibilities

• Lead the design, implementation, and maintenance of scalable end-to-end test automation frameworks tailored for GenAI applications and Large Language Models (LLMs).
• Build automation solutions that integrate multiple testing tools, API layers, and backend systems into a cohesive automation suite.
• Actively test and evaluate LLMs, prompt engineering frameworks, and Retrieval-Augmented Generation (RAG) systems using specialized AI evaluation tools like LangChain.
• Embed automated GenAI evaluation and regression suites into modern CI/CD pipelines (e.g., GitHub Actions, GitLab CI/CD, or Jenkins) to achieve continuous testing and rapid feedback loops.
• Perform deep data validation and backend testing by writing complex SQL queries and managing test data across cloud data platforms like Snowflake.
• Guide and upskill traditional QA automation engineers in Python, AI evaluation techniques, and the nuances of testing cognitive applications.

Requirements

• 10+ years QA experience
• End-to-end test architecture / test integration for GenAI development projects.
• GenAI Experience: Strong experience in testing and evaluating GenAI models, prompt engineering frameworks, and RAG systems.
• Technical Proficiency: Proficiency in Python and AI evaluation tools like LangChain.
• Hands-on Automation Experience: Experience with LLMs and integration of multiple tools with automation (not just Selenium).
• Hands-on DB experience: SQL/Snowflake.
• CI/CD with test automation.

Nice to Have

• Ability to design scalable testing frameworks within cloud environments (AWS, Azure, GCP).
• Deep knowledge of Software Testing Life Cycle (STLC), CI/CD pipelines, and API testing.

We Offer

• Competitive compensation: Base salary ranges between CAD 90,000 – 110,000 per year.
• Hybrid work model: 4 days a week in a Cognizant or client office in Toronto.
• Benefits: Comprehensive benefits that support your physical, mental, and financial wellbeing.

About the Company

Cognizant (Nasdaq: CTSH) is an AI Builder and technology services provider, bridging the gap between AI investment and enterprise value by building full-stack AI solutions for our clients. Our deep industry, process, and engineering expertise enables us to build an organization’s unique context into technology systems that amplify human potential, drive tangible outcomes, and keep global enterprises ahead in a fast-changing world. See how at cognizant.ai or @cognizant.

Application Process

1. Apply to the right opportunity: Find an open role that matches your skills and career goals and show us why you’re the person for the job.
2. Connect with your recruiter: If one of our recruiters sees a fit, they’ll set up a short intro call to learn more about you and how your experiences and skills match the role.
3. Show off your skills: If you and our team would like to continue the process, you’ll interview with our hiring team. Some roles may also require technical assessments and/or client interviews.
4. Decision time: Our team will then review each candidate’s potential to succeed in the role. This step may take some time because we want to find the right fit for both you and us-but you can count on us to keep you updated.

Note: Cognizant will only consider applicants for this position who are legally authorized to work in Canada without requiring employer sponsorship, now or at any time in the future.

Applications for this position are reviewed by our recruitment team without the use of artificial intelligence screening tools.