Module Lead, Data Science (LLM-as-a-judge)
Primary stack
Job description
About the Position
Module Lead, Data Science (LLM-as-a-judge)
Location: Noida, Uttar Pradesh, India
Department: Quality Assurance
BOLD is seeking a senior professional to implement and run Testing and Quality Assurance specifically designed for AI-driven products and Data Science workflows. In this role, you will bridge the gap between traditional software testing and modern machine learning evaluation. You have great team management and organizational skills. You’ll have a broad mastery of capabilities, techniques, technologies, and tools in the Testing and QA domains. You’ll be handling QA teams of various sizes and expertise.
Responsibilities
- Develop and own the QA strategy for AI-driven features, focusing on model accuracy, data drift, and performance metrics (Precision, Recall, F1-score).
- Implement state-of-the-art AI tools (e.g., GitHub Copilot, or custom LLM agents) to automate test case generation, code analysis, and synthetic data creation.
- Oversee the testing of Generative AI outputs, implementing "LLM-as-a-judge" frameworks to evaluate hallucination rates, toxicity, and prompt adherence.
- Develop and own BOLD’s quality framework from product development through to production.
- Drive initiatives (both Business and QA) in your area along with needed smooth coordination across the board to ensure a timely achievement of goals.
- Take part in Requirement Analysis, Sprint planning, Bug triage meetings.
- Liaise with product, development, and infrastructure teams to collaborate on the delivery of high-quality, high-performance, and high-availability applications.
- Develop quality scripts with needed coverage to ensure defect-free releases.
- Manage and mentor team members to maximize output and efficiency and promote a team culture of quality and fun.
- Motivate, Encourage, Guide Teams maintaining equilibrium and satisfaction level.
- Give technical/logical solutions to problems faced by the teams and remove hurdles blocking releases.
- Evaluate and suggest new tools and technologies.
Requirements
- Solid understanding of the Machine Learning lifecycle (EDA, Training, Evaluation, Deployment).
- Experience using LLMs (GPT, Claude, Llama) and AI-coding assistants to accelerate the testing lifecycle.
- Experience in a high-volume, high-throughput web/ecommerce environment.
- Hands-on experience with UI/API automation (Selenium, Playwright, or Cypress) integrated with AI-driven healing capabilities.
- Experience with web service technology like XML, REST, SOAP.
- Knowledge of any query language like SQL, JQL, etc.
- Experience in testing complex web and mobile apps.
- Strong application architecture and technical knowledge.
- Strong knowledge of web concepts, including web performance infrastructure.
- Able to play the role of a scrum master, with strong coordination skills.
- Know-how of basic UX standards, SEO/SEM concepts, and key tracking matrices.
- Strong communication skills.
Experience
- Module Lead, QA (AI/ML & Automation) - 5 years+
We Offer
Outstanding Compensation
- Competitive salary
- Tax-friendly compensation structure
- Bi-annual bonus
- Annual Appraisal
- Equity in company
100% Full Health Benefits
- Group Mediclaim, personal accident, & term life insurance
- Group Mediclaim benefit (including parents' coverage)
- Online Health (OPD) Consultation Benefit
- Personal accident and term life insurance coverage
Flexible Time Away
- 24 days paid leaves
- Declared fixed holidays
- Paternity and maternity leave
- Compassionate and marriage leave
- Covid leave (up to 7 days)
Additional Benefits
- Internet and home office reimbursement
- In-office catered lunch, meals, and snacks
- Certification policy
- Cab pick-up and drop-off facility
About the Company
We Transform Work Lives
As an established global organization, BOLD helps people find jobs. Our story is one of growth, success, and professional fulfillment. We create digital products that have empowered millions of people in 180 countries to build stronger resumes, cover letters, and CVs. The result of our work helps people interview confidently, finding the right job in less time. Our employees are experts, learners, contributors, and creatives.
We Celebrate And Promote Diversity And Inclusion
We value our position as an Equal Opportunity Employer. We hire based on qualifications, merit, and our business needs. We don't discriminate regarding race, color, religion, gender, pregnancy, national origin or citizenship, ancestry, age, physical or mental disability, veteran status, sexual orientation, gender identity or expression, marital status, genetic information, or any other applicable characteristic protected by law.
© BOLD PL. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
Recruitment & HR Tech
About the Company BOLD PL is a globally recognized leader in career development tools, empowering over three million people in 180 countries. The company specializes in creating products, tools, and resources to help job seekers maximize their career potential. BOLD's offerings include AI-powered resume and cover letter builders, job search sites, networking platforms, and career productivity tools. The company's technology platform is designed for individuals at various stages of their careers, from job seekers to professionals looking to advance. Products, Services & Tech Stack Core Solutions: AI-powered resume and cover letter builders Job search sites Networking platforms Career productivity tools Project Scope & Target Clients Client Base: Job seekers in every industry and at every career stage Engineering Scale: The company has a global presence with offices in Puerto Rico, Poland, India, and the United States, employing over 1,500 people worldwide.