Senior Data Engineer

About the Position

We are seeking an experienced Senior Data Engineer to join our team in Ukraine. The project entails the development and optimization of a new Data Platform to adjust our current data processing system. If you have a passion for Data Software Engineering to deliver impactful web applications, we would love to hear from you.

Responsibilities

• Develop data processing pipelines using Azure Data Factory and Azure Functions
• Implement data ingestion and transformation processes using Databricks and PySpark
• Write efficient code in Python and SQL to extract, transform, and load data
• Collaborate with stakeholders to gather requirements and ensure the platform meets business needs
• Monitor and troubleshoot the platform's performance and address any issues
• Implement data quality assurance measures and process improvements
• Optimize data storage and retrieval operations
• Conduct data analysis and generate insights to guide decision-making
• Train and mentor junior Data Engineers
• Participate in peer code reviews and collaborative team activities

Requirements

• 3+ years of hands-on experience in big-data field and expertise in Data Software Engineering
• Strong knowledge and experience with Azure Data Factory, Azure Functions, and Databricks
• Proficiency in PySpark, Python, and SQL
• Understanding of batch processing and ETL principles in Data Warehouses
• Experience with data completeness signals and orchestration
• Knowledge of schema migrations and datasets evolution
• Familiarity with approaches for historical reprocessing and data correction
• Good problem-solving and communication skills
• Proficient in English with a B2+ level, strong speaking English

Nice to Have

• Familiarity with Microsoft Power BI
• Competency in Azure Blob Storage, Azure File Storage

We Offer

• Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
• Receive the necessary equipment to perform your work tasks
• Change projects and technology stacks within EPAM
• Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
• Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
• Participate in volunteer, charity programs and communities (both technical and interest-based)
• Focus on your professional growth:
  • Plan your individual career path together with your manager
  • Receive regular feedback from colleagues
  • Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
  • Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
  • Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
  • Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
  • Study at EPAM Solution Architecture School with the instructors who are practicing architects
  • Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
  • Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)
• Vacation and sick leave (including a sick leave without a medical certificate)
• A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
• Medical insurance for family members at corporate rates
• Company support during significant life events (childbirth or adoption, marriage, etc.)
• Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
• E-kids program - a free programming language training program for EPAMers' children