Data Engineer
Primary stack
DatabricksDelta LakeApache SparkPythonSQLGitCI/CD
Nice to have's
Delta Lake
Job description
About the Position
We are seeking a skilled Data Engineer to develop and optimize a Lakehouse platform on Databricks for an international client. Experience with Delta Lake, Apache Spark, and Python is required.
Responsibilities
- Development and optimization of a production Lakehouse platform on Databricks at enterprise scale for an international client
- Working in an international team focused on large-scale data processing, process automation, and solving advanced analytical challenges using Delta Lake, Apache Spark, and Python
- Design and development of data integrations, CI/CD pipelines, and performance and cost optimization for end-to-end data solutions
- Option to work fully remote (for candidates based in the Czech Republic, a hybrid model with 2 days onsite in Prague is available, allowing involvement in other local projects)
Requirements
- Advanced experience with:
- Databricks and Delta Lake
- Apache Spark (development of data transformations and integrations)
- Experience with:
- Python and SQL at an advanced level
- Cloud platforms (Azure, AWS, or GCP)
- Git version control and CI/CD processes
- Advanced knowledge of:
- English (C1/B2) for day-to-day communication in an international team
Nice to Have
- Optimization of parallel data processing and performance tuning on Delta Lake
- Data warehousing (DWH), SCD2, CDC, Data Quality, and BI
We Offer
- Fully remote work option (hybrid model available for candidates based in the Czech Republic)
- Involvement in other local projects in Prague
About the Company
The Workly s.r.o.
4D CENTER, Kodaňská 1441/46,
101 00, Praha 10.
Contact:
[email protected]
- 420 775 371 991
©2023, All right reserved.
Apply for this position
Our team will review your application within the next 5 days.
© The Workly. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.