Senior Data Engineer/ Databricks, PySpark, Python

About the Position

We are seeking a Senior Data Engineer with expertise in Databricks, PySpark, and Python to join our team and work on a large-scale vehicle telemetry data project.

Responsibilities

• Monitor the health, stability, and performance of the production data platform
• Investigate and resolve performance issues and production incidents
• Analyze Databricks workloads and identify performance bottlenecks
• Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency
• Support and improve approximately 200 existing Databricks jobs and associated data pipelines
• Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded
• Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs
• Track and optimize Databricks and Azure resource consumption
• Improve platform observability, monitoring, alerting, and operational processes
• Ensure consistency across multiple regional deployments in Europe, North America, and China
• Identify opportunities for technical improvements, automation, and reduction of operational overhead

Requirements

• 3+ years of hands-on experience in data engineering
• Expertise in Azure Databricks
• Proficiency in PySpark and distributed data processing
• Advanced skills in Python development
• Understanding of performance optimization for large-scale Spark workloads
• Background in operating and troubleshooting production data platforms
• Knowledge of monitoring, observability, and capacity planning combined with cloud cost optimization
• Ability to analyze existing systems, identify bottlenecks, and propose pragmatic improvements
• English proficiency at an Upper-Intermediate level (B2) or higher

Nice to Have

• Familiarity with Microsoft Azure services and cloud infrastructure
• Experience building and maintaining CI/CD pipelines using GitHub Actions
• Exposure to large-scale telemetry, IoT, or automotive data
• Background in managing Databricks environments with a large number of scheduled jobs and pipelines
• Showcase of multi-region or geographically distributed cloud deployments

We Offer

• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events
• We invest in your growth: Leadership development, career advising, soft skills and well-being programs, Certifications, including GCP, Azure and AWS, Unlimited access to EPAM's internal learning database, Free English classes with certified teachers
• We cover it all: Participation in the Employee Stock Purchase Plan, Monetary bonuses for engaging in the referral program, Comprehensive medical & family care package, Trust days per year (sick leave without a medical certificate), Coverage of psychology sessions of your choice, Discounts for fitness clubs and sports programs, Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership

About the Company

We connect like-minded people: Delivering innovative solutions to industry leaders, making a global impact.