Senior Data Engineer (Investment Data)

About the Position

We are seeking a Senior Data Engineer with expertise in financial markets to design and operate data pipelines for investment teams. You will be responsible for designing, building, and operating data pipelines and models that power analytics and applications across investment teams and Middle/Back office.

Responsibilities

• Participate in requirements clarification and sprint planning sessions.
• Design technical solutions and implement them, including ETL Pipelines.
• Build robust data pipelines in PySpark to extract, transform.
• Optimize ETL Processes to enhance performance, scalability, and reliability.
• Write unit and integration tests.
• Support QA teammates in the acceptance process.
• Resolve PROD incidents as a 3rd line engineer.

Requirements

• Must have

  • Bachelor’s degree (Computer Science, Engineering, Information Systems, or related discipline).
  • 5+ years experience in data engineering roles.
  • Strong hands-on experience with Databricks in production environments.
  • Strong programming experience with PySpark and strong SQL.
  • Proven experience with Declarative Pipelines / pipeline orchestration on Databricks.
  • Strong understanding of data engineering fundamentals: ingestion patterns, transformation design, incremental processing, testing, performance tuning.
  • Experience delivering production-ready datasets with appropriate operational controls.
  • Experience with modern Lakehouse concepts (Delta tables, optimization strategies, file skipping, metadata/statistics awareness).
  • Exposure to data governance practices: cataloguing, documentation, business glossary/terms, lineage.
  • Experience working in enterprise environments with CI/CD pipelines and structured release processes.
  • Familiarity with vendor market data feeds (e.g., Bloomberg, Refinitiv, MSCI, FactSet) or similar multi-source mastering patterns.

• Nice to have

  • Strong Hands-on Expertise in Palantir Foundry. Proven experience with Foundry pipelines, ontologies, data lineage, transformations, and platform governance.
  • Proven Migration Experience from Palantir / to Databricks. Demonstrated experience leading or executing platform migrations, including pipeline conversion, data model redesign, and production cutover.
  • Familiarity with Dynatrace or Datadog for system observability and monitoring.
  • Databricks certification, cloud certifications (Azure/AWS), or enterprise data architecture certifications.

We Offer

• Competitive salary
• Relocation friendly
• Opportunity to work in a dynamic and innovative environment

About the Company

Join our team in Abu Dhabi, United Arab Emirates, and be part of a leading consulting firm in the BCM Industry.

Application

Apply for Senior Data Engineer (Investment Data) in Abu Dhabi.