Job Description, Responsibilities & Requirements
About the Position
As a GenAI Test Automation Lead, you will design scalable, Python-driven automation frameworks to validate LLMs, prompt frameworks, and RAG systems across UI and API layers. You will leverage modern CI/CD pipelines and write complex SQL queries to perform deep backend and database data validation.
Responsibilities
- Lead the design, implementation, and maintenance of scalable end-to-end test automation frameworks tailored for GenAI applications and Large Language Models (LLMs).
- Build automation solutions that go beyond standard UI tools (like Selenium), integrating multiple testing tools, API layers, and backend systems into a cohesive automation suite.
- Actively test and evaluate LLMs, prompt engineering frameworks, and Retrieval-Augmented Generation (RAG) systems using specialized AI evaluation tools like LangChain.
- Embed automated GenAI evaluation and regression suites into modern CI/CD pipelines (e.g., GitHub Actions, GitLab CI/CD, or Jenkins) to achieve continuous testing and rapid feedback loops.
- Perform deep data validation and backend testing by writing complex SQL queries and managing test data across cloud data platforms like Snowflake.
- Guide and upskill traditional QA automation engineers in Python, AI evaluation techniques, and the nuances of testing cognitive applications.
Requirements
- 10+ years QA experience
- End-to-end test architecture / test integration for GenAI development projects.
- Must have GenAI Experience: Strong experience in testing and evaluating GenAI models, prompt engineering frameworks, and RAG systems.
- Technical Proficiency: Proficiency in Python and AI evaluation tools like LangChain.
- Hands-on Automation Experience in Python (Must Have).
- Hands-on experience with LLMs (Must Have).
- Experience with integration of multiple tools with automation (not just Selenium).
- Hands-on DB experience - SQL/Snowflake (Must have).
- CI/CD with test automation (Must have).
Nice to Have
- Ability to design scalable testing frameworks within cloud environments (AWS, Azure, GCP).
- Deep knowledge of Software Testing Life Cycle (STLC), CI/CD pipelines, and API testing.
We Offer
- Total compensation: The base salary for this position ranges between CAD 90,000 – 110,000 per year. The successful candidate may fall within the range depending on relevant education, work and/or management experience and other business-related and job-necessary qualifications. This position is also eligible for Cognizant’s discretionary annual performance-based bonus, as well as benefits that support your physical, mental and financial wellbeing.
- Working arrangements: Hybrid work model requiring 4 days a week in a Cognizant or client office in Toronto.
About the Company
Cognizant (Nasdaq: CTSH) is an AI Builder and technology services provider, bridging the gap between AI investment and enterprise value by building full-stack AI solutions for our clients. Our deep industry, process and engineering expertise enables us to build an organization’s unique context into technology systems that amplify human potential, drive tangible outcomes and keep global enterprises ahead in a fast-changing world. See how at cognizant.ai or @cognizant.
Note: The working arrangements for this role are accurate as of the date of posting. This may change based on the project you’re engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.
Cognizant will only consider applicants for this position who are legally authorized to work in Canada without requiring employer sponsorship, now or at any time in the future.
Applications for this position are reviewed by our recruitment team without the use of artificial intelligence screening tools.
Benefits that help you thrive and grow
Our benefits program is built with you in mind-so you can enjoy a fulfilling, balanced and healthy life.
Financial wellbeing
We regularly review market data to ensure compensation reflects the value you bring. Your benefits extend beyond pay and may include retirement plans, financial education, etc.
Physical and mental health
We empower you to prioritize your wellbeing through paid time off, flexible working where possible, healthcare plans, counselling, our Mental Health Allyship program and more.
Your career, your way
With 350,000+ roles at Cognizant, you’ll have opportunities explore new technologies, industries and locations-and build the skills you need to grow your career.
Real-world impact
Think about the biggest brands you rely on. Chances are, they rely on us to help strengthen their business. Here, you’ll turn bold ideas into solutions that improve lives everywhere.
Your path to Cognizant
Here’s what you can expect from our hiring process. Please keep in mind that it may vary by role and location. Explore our FAQs to learn more.
Step 1: Apply to the right opportunity
Find an open role that matches your skills and career goals and show us why you’re the person for the job. Don’t see the right opportunity? Consider joining our Talent Community.
Step 2: Connect with your recruiter
If one of our recruiters sees a fit, they’ll set up a short intro call to learn more about you and how your experiences and skills match the role.
Step 3: Show off your skills
If you and our team would like to continue the process, you’ll interview with our hiring team. Some roles may also require technical assessments and/or client interviews.
Step 4: Decision time
Our team will then review each candidate’s potential to succeed in the role. This step may take some time because we want to find the right fit for both you and us-but you can count on us to keep you updated.
Haven't yet found the right opportunity?
Get the latest updates on job opportunities, recruitment events and company news-tailored just for you!
Be in the know
We care about your privacy
Thank you, your preferences have been updated.
By clicking "Accept All", you agree to the storing of cookies on your device to give you the most optimal experience using our website. We may also use cookies to enhance performance, analyse site usage and to personalise your experience.
Manage Preferences