Senior Data Integration Engineer
Primary stack
Nice to have's
Job description
About the Position
We are actively seeking an experienced Senior Data Integration Engineer to join our team. In this role, you will play a crucial part in designing, developing, testing, deploying, maintaining, and enhancing data integration pipelines for our cutting-edge projects. If you possess a wealth of experience in data integration and pipeline development and thrive in a dynamic, collaborative environment, we welcome you to be a key player in shaping the future of our data-driven initiatives.
Responsibilities
- Design, develop, test, deploy, maintain, and improve data integration pipelines
- Utilize Python and common libraries for efficient data integration and pipeline development
- Employ advanced SQL for analytical purposes and to optimize complex queries
- Collaborate with cross-functional teams to ensure the success of data integration projects
- Implement and maintain data integration and pipeline developments in an AWS environment
- Optimize and debug user-defined functions, views, and indexes for efficient data integration
- Utilize GitLab for source control and Jenkins for build and continuous integration
- Contribute to the enhancement and maintenance of data integration pipelines
- Provide insights and recommendations for improving data integration processes
- Collaborate with the wider engineering team to drive standards of excellence across all data integration projects
Requirements
- At least 3+ years of hands-on experience in data integration and pipeline development
- Strong expertise in Amazon Web Services (AWS) Cloud for data integration with Apache Spark, Glue, Kinesis, and S3
- Proficiency in Python for data integration, with real-life experience in Python development
- Solid understanding of advanced SQL for analytical purposes and complex query optimization
- Experience with Databricks in an AWS environment for effective data integration
- Knowledge of source control systems such as GitLab and continuous integration tools like Jenkins
- Analytical experience with databases, including writing complex queries, query optimization, and debugging
- Strong communication and collaboration skills, essential for effective teamwork
- Fluent English language skills at an Upper-Intermediate level
Nice to Have
- Familiarity with other data integration tools and technologies
About the Company
[Company description if present]
© EPAM. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
EPAM helps organizations innovate their business processes and rethink the way they manage their businesses so they can remain competitive in this new digital age.