Senior AI Platform Engineer
Primary stack
Nice to have's
Job description
About the Position
We are looking for a hands-on AI Platform Engineer to design, build, and maintain the Enterprise Agent Development Platform, a production-grade, cloud-native ecosystem that enables engineering teams to define, orchestrate, deploy, and observe AI agents at scale. The role includes standardizing agent development across the organization, implementing advanced agentic evaluation frameworks (using AWS AgentCore), and building robust LLM-as-a-judge and CI/CD deployment gates to reduce development from months to days while enforcing strict security and quality standards. Strong Python skills and deep expertise in LLM evaluation are required, along with experience in cloud infrastructure.
Responsibilities
- Develop and launch scalable conversational AI systems that operate across multiple channels, including chat, voice, email, and APIs
- Architect microservices and asynchronous systems for AI applications, ensuring high availability and performance
- Use Python to implement orchestration and decision logic - the control layer that manages tools, workflows, and system interactions
- Agentic Evaluation & Quality Gates: Design evaluation frameworks for autonomous agents tracking multi-step workflows, tool-calling accuracy, and session performance, integrating automated gates into CI/CD pipelines
- LLM-as-a-Judge & Testing Pipelines: Establish automated evaluation protocols utilizing LLM-as-a-judge patterns
- Optimize AI models through fine-tuning and LoRA integration, making data-driven decisions between different approaches (FT vs RAG)
- Work with data specialists to design pipelines for collecting, cleaning, and structuring information for AI agents
- Design CI/CD pipelines, container standards, and cloud strategies for infrastructure migrations, cost optimization, and high availability
Requirements
- Strong background in Python, mainly for hooking up AI models, automating stuff, and handling orchestration logic
- 1–2 years of hands-on experience with Generative AI and large language models (LLMs)
- Practical experience in machine learning engineering: data preparation, working with vector databases in the cloud, and training/optimizing models
- Expertise in Agentic Evaluation: Hands-on experience designing evaluation frameworks for complex, multi-agent systems and evaluating reasoning traces, tool selection, and agentic workflows
- LLMOps & CI/CD Gate Design: Proven experience in building automated testing and evaluation gates within CI/CD pipelines for AI models
- Strong backend and frontend architecture experience
- Experience working with cloud platforms such as AWS, GCP, or Azure, including container orchestration and CI/CD pipelines
- Experience with agentic frameworks (LangChain, LangGraph) and multi-agent system design
- Comfort with ML frameworks such as TensorFlow, PyTorch, or Scikit-learn is a plus
We Offer
- Opportunity to work remotely in Poland
- Engaging in cutting-edge AI projects
- Collaborative and innovative work environment
About the Company
EPAM Systems is a next-generation global technology services company with a passion for innovation and a commitment to excellence. We empower businesses to transform and thrive in the digital age by delivering end-to-end software development and digital platform engineering services.
© EPAM. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
EPAM helps organizations innovate their business processes and rethink the way they manage their businesses so they can remain competitive in this new digital age.