Architect / Principal Software Engineer

About the Position

We are seeking an Architect / Principal Software Engineer to lead the technical direction of our AI Engineering pods in the USA. This role is fully remote.

Responsibilities

• Set architectural direction across multiple AI pods, making clear trade-offs between model capabilities, cost, latency, and operational complexity.
• Spend roughly a third of your time in code: writing backend AI services, building orchestration patterns, and defining integration contracts with frontend teams.
• Design agent workflows with streaming responses, predictable error handling, and low latency for chat and automated tasks.
• Establish shared standards for Model Context Protocol (MCP) servers, tool schemas, and context management.
• Work with Okta Security and Infrastructure to enforce least-privilege tool execution, tenant isolation, and secret handling within agent loops.
• Define observability requirements and service-level objectives, including tracing with tools like LangSmith or Arize Phoenix.
• Mentor Staff and Senior engineers through direct design reviews and technical guidance.

Requirements

• 10+ years of software engineering experience, with a background in designing, scaling, and operating distributed backend systems.
• 6+ years of Python experience, especially with FastAPI, AsyncIO, and modern typing practices. Python is our primary language for backend and AI services.
• 3+ years building production systems with LLMs, including LangChain, LangGraph, AWS Bedrock, Anthropic Claude, or OpenAI APIs. Candidates with classical ML backgrounds (PyTorch, scikit-learn) who moved into generative AI architectures are welcome.
• Hands-on experience with LangGraph or similar stateful agent frameworks, particularly managing state graphs, loops, checkpointing, and human-in-the-loop steps.
• 3 to 4 years of React and TypeScript experience, enough to review UI code, understand frontend state, and ensure streaming APIs work cleanly with the web client.
• Production AWS experience, including EKS or ECS, Lambda, Bedrock, S3, Docker, and Terraform or OpenTofu.
• Authentication and security fundamentals, specifically OAuth 2.0, OIDC, and secure credential handling.

Nice to Have

• Practical experience building custom MCP servers and defining tool discovery patterns.
• Experience migrating from older retrieval systems (such as Amazon Kendra) to vector platforms like OpenSearch, Pinecone, or Qdrant.
• Experience handling multimodal data (images, documents) within LangGraph state.
• Background in Identity and Access Management (IAM) or Customer Identity (CIAM).

We Offer

• Medical, Dental and Vision Insurance (Subsidized)
• Health Savings Account
• Flexible Spending Accounts (Healthcare, Dependent Care, Commuter)
• Short-Term and Long-Term Disability (Company Provided)
• Life and AD&D Insurance (Company Provided)
• Employee Assistance Program
• Unlimited access to LinkedIn learning solutions
• Matched 401(k) Retirement Savings Plan
• Paid Time Off – the employee will be eligible to accrue 15-25 paid days, depending on specific level and tenure with EPAM (accrual eligibility may change over time)
• Paid Holidays - nine (9) total per year
• Legal Plan and Identity Theft Protection
• Accident Insurance
• Employee Discounts
• Pet Insurance
• Employee Stock Purchase Program
• If otherwise eligible, participation in the discretionary annual bonus program
• If otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program