Senior Data Integration Engineer
Primary stack
Nice to have's
Job description
About the Position
We are seeking a motivated Senior Data Integration Engineer to join our team and play a key role in building, optimizing, and migrating data pipelines across diverse systems. This role focuses on utilizing SQL, Azure, PySpark, HDFS, and Snowflake to improve data workflows, ensure seamless integrations, and deliver solutions to complex data challenges.
Responsibilities
- Design, develop, and optimize SQL queries and stored procedures for efficient data processing and integration
- Lead data migration projects from on-premises systems or legacy environments to Azure Synapse Analytics
- Work with advanced Snowflake concepts such as window functions, common table expressions (CTEs), and JavaScript-based stored procedures
- Implement data pipelines using PySpark and manage file processing in Hadoop HDFS environments
- Build and troubleshoot Azure Data Factory (ADF) pipelines to integrate data from ADLS and SFTP into target systems
- Collaborate with cross-functional teams to implement and automate end-to-end data workflows
- Monitor and maintain the performance of data pipelines, ensuring scalability and reliability
- Develop and maintain technical documentation for data architecture, transformations, and pipeline processes
- Provide technical leadership and mentoring to junior engineers and team members as needed
Requirements
- 3+ years of experience working with data integration, migration, and pipeline optimization across SQL and cloud platforms
- Expertise in Snowflake, including Snowflake Analytics
- Advanced SQL skills with knowledge of window functions, CTEs, and JavaScript-based stored procedures
- Background in PySpark and Hadoop HDFS, including pipeline migration from Cloudera Data Lake to Azure Synapse Analytics
- Proficiency in creating ADF pipelines for data transfer involving sources such as ADLS and SFTP
- Competency in Python, PySpark, and SQL for data engineering tasks and automation
- Understanding of Azure Cloud services and expertise in tools like Azure Synapse Analytics and Power Automate
- English level of minimum B2 (Upper-Intermediate) for effective communication
Nice to Have
- Familiarity with legacy systems and experience handling data integration from on-prem environments to cloud platforms
- Showcase of performance tuning techniques for Snowflake stored procedures and SQL pipelines
- Qualifications in additional Azure services or certifications
We Offer
- Opportunity to work remotely in Ukraine
- Engaging role in a dynamic and innovative team
- Competitive compensation package
About the Company
[Company description if present]
© EPAM. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
EPAM helps organizations innovate their business processes and rethink the way they manage their businesses so they can remain competitive in this new digital age.