Senior AI Engineer with Spark, AWS Services

Senior · Remote · Georgia, Armenia, Kazakhstan, Kyrgyzstan, Uzbekistan

About the Role

We are seeking a Senior AI Engineer with Spark and AWS Services expertise to join the RBQM Production Pod within our program. You will build and maintain data pipelines that power AI/GenAI applications for Risk-Based Quality Management in clinical trials. This role focuses on RAG document ingestion, vector indexing, and building data APIs for AI applications.

Responsibilities

• Design and build RAG document ingestion pipelines (chunking, embedding, vector indexing) for clinical trial quality data
• Build and manage vector databases (AWS OpenSearch) for RAG-powered AI workflows
• Develop batch and streaming ETL/ELT pipelines from scratch for unstructured clinical data (PDF, DOCX, clinical reports)
• Build and expose data APIs for AI application consumption
• Optimize chunking strategies, embedding generation, and retrieval performance for RAG architectures
• Manage data quality, lineage, and governance for AI/ML data pipelines
• Deploy and maintain AWS data infrastructure (S3, Lambda, Glue, Athena, Step Functions, DynamoDB)
• Collaborate with Data Scientists and Backend Developers in an integrated pod team

Requirements

• 5+ years of hands-on data engineering experience at scale
• Expertise in RAG document ingestion pipelines (chunking, embedding, vector indexing)
• Proficiency in AWS OpenSearch as a vector database for RAG workflows
• Advanced proficiency in Python, including SQL and Spark SQL
• Skills in unstructured data transformation (PDF, DOCX) for RAG/LLM applications
• Familiarity with AWS Services: S3, Lambda, Glue, Athena, Bedrock, Step Functions, API Gateway, CloudWatch, DynamoDB
• Knowledge of containerization with Docker
• Capability to build custom pipelines from scratch, beyond configuring out-of-the-box services
• Proficiency in English at a B2+ level

Nice to Have

• Background in pharmaceutical or life sciences domain
• Familiarity with Snowflake, Pinecone (vector DB alternative)
• Knowledge of SageMaker processing jobs
• Skills in CI/CD tools (Jenkins, Git/Bitbucket) and infrastructure tools (CDK or Terraform)
• Understanding of clinical data standards (CDISC, ADaM, SDTM)

What We Offer

• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events
• We invest in your growth: Leadership development, career advising, soft skills and well-being programs, certifications, including GCP, Azure and AWS, unlimited access to EPAM's internal learning database, free English classes with certified teachers
• We cover it all: Participation in the Employee Stock Purchase Plan, monetary bonuses for engaging in the referral program, comprehensive medical & family care package, trust days per year (sick leave without a medical certificate), discounts for fitness clubs and sports programs, benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership

About the Company

We connect like-minded people: Delivering innovative solutions to industry leaders, making a global impact.