Senior Data Engineer
Primary stack
Job description
About the Position
We are seeking an experienced Senior Data Engineer to join our team.
The project entails the development and optimization of a new Data Platform to adjust our current data processing system. The Senior Data Engineer will be responsible for designing, implementing, and maintaining the platform to meet the business's data needs. If you have a passion for Data Software Engineering to deliver impactful web applications, we would love to hear from you.
Responsibilities
- Develop data processing pipelines using Azure Data Factory and Azure Functions
- Implement data ingestion and transformation processes using Databricks and PySpark
- Write efficient code in Python and SQL to extract, transform, and load data
- Collaborate with stakeholders to gather requirements and ensure the platform meets business needs
- Monitor and troubleshoot the platform's performance and address any issues
- Implement data quality assurance measures and process improvements
- Optimize data storage and retrieval operations
- Conduct data analysis and generate insights to guide decision-making
- Train and mentor junior Data Engineers
- Participate in peer code reviews and collaborative team activities
Requirements
- 3+ years of hands-on experience in big-data field and expertise in Data Software Engineering
- Strong knowledge and experience with Azure Data Factory, Azure Functions, and Databricks
- Proficiency in PySpark, Python, and SQL
- Understanding of batch processing and ETL principles in Data Warehouses
- Experience with data completeness signals and orchestration
- Knowledge of schema migrations and datasets evolution
- Familiarity with approaches for historical reprocessing and data correction
- Good problem-solving and communication skills
- Proficient in English with a B2+ level, strong speaking English
Nice to Have
- Familiarity with Microsoft Power BI
- Competency in Azure Blob Storage, Azure File Storage
We Offer
- Remote work in Ukraine
About the Company
[Company description if present]
© EPAM. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
EPAM helps organizations innovate their business processes and rethink the way they manage their businesses so they can remain competitive in this new digital age.
More at EPAM
All 2384 rolesLead Full-Stack Developer
EPAM · Argentina +1
Python Engineering Manager
EPAM · Mexico
Product Manager
EPAM · Argentina +2
Automation Tester (JavaScript)
EPAM · Brazil