Senior Data Quality Engineer
Primary stack
Job description
About the Position
We are seeking a Senior Data Quality Engineer to support three key engineering streams (Data Product Engineering Framework, Polarmesh Portal, Platform Engineering Team). You will collaborate closely with Data Engineers, Platform Engineers, and DevOps teams to design, build, and maintain automated testing solutions across data pipelines, APIs, and platform components in a modern Azure-based ecosystem.
Responsibilities
- Design and build automated testing solutions across data pipelines, APIs, and platform components
- Maintain and enhance test automation frameworks within an Azure-based ecosystem
- Validate data quality, data transformations, and pipeline outputs using SQL and/or Python
- Collaborate closely with Data Engineers, Platform Engineers, and DevOps teams on testing strategy
- Integrate automated tests into CI/CD pipelines
- Monitor, log, and troubleshoot pipeline failures
- Support the maintenance and optimization of ETL/ELT processes and data pipelines
- Contribute to testing efforts across three engineering streams, including the Data Product Engineering Framework, Polarmesh Portal, and Platform Engineering Team
Requirements
- 5+ years of experience as a QA Engineer / Test Automation Engineer in Data Engineering or Data Platform environments
- Strong hands-on experience with Azure cloud services
- Experience working with Azure Databricks, Spark/PySpark, or similar big data platforms
- Understanding of data pipelines, ETL/ELT processes, and data pipeline maintenance
- Skills in testing data quality, data transformations, and pipeline outputs using SQL and/or Python
- Experience with CI/CD pipelines, preferably using Azure DevOps, GitHub Actions, or similar tools
- Knowledge of automated testing approaches for data pipelines, APIs, and platform components
- Familiarity with version control systems such as GitHub
- Understanding of monitoring, logging, and troubleshooting pipeline failures
- Ability to work closely with Data Engineers, Platform Engineers, and DevOps teams
Nice to Have
- Experience with Delta Lake, Azure Data Factory, ADLS, Synapse, or Event Hub
- Knowledge of infrastructure as code tools such as Terraform or ARM/Bicep
- Experience with data quality tools/frameworks
- Basic understanding of Docker, Kubernetes, or containerized environments
- Experience in Agile/Scrum teams
© EPAM. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
EPAM helps organizations innovate their business processes and rethink the way they manage their businesses so they can remain competitive in this new digital age.
More at EPAM
All 2384 rolesLead Full-Stack Developer
EPAM · Argentina +1
Python Engineering Manager
EPAM · Mexico
Product Manager
EPAM · Argentina +2
Automation Tester (JavaScript)
EPAM · Brazil