Data Engineer

About the Position

We are seeking a highly skilled Data Engineer with 5-8 years of experience to join our large-scale Banking and Financial Services transformation program. The role involves designing, developing, and supporting high-performance data pipelines that process and integrate large volumes of structured and unstructured data across multiple banking systems.

Responsibilities

• Design, develop, and maintain scalable ETL processes and large-scale data pipelines using Spark and Scala.
• Build and support batch data processing solutions across enterprise data platforms.
• Develop and optimize data solutions using Snowflake for warehousing, integration, and performance tuning.
• Work with the Hadoop ecosystem (HDFS, Hive) for data storage and processing.
• Design and optimize SQL queries for high-volume datasets and ensure efficient data retrieval.
• Develop and manage data solutions on AWS using services such as S3, Glue, Athena, and Redshift.
• Integrate data pipelines with AI/ML and analytics workloads on AWS.
• Collaborate with business and technology teams to deliver end-to-end data engineering solutions within the Banking domain.
• Implement and support CI/CD pipelines using GitHub, GitHub Actions, TeamCity, or similar tools.
• Develop automation scripts using Shell Scripting.
• Provide production support, including incident resolution, troubleshooting, and root cause analysis.
• Follow clean coding standards, secure coding practices, and code review processes.
• Participate in performance optimization, release deployments, and ongoing platform improvements.
• Communicate effectively with stakeholders and contribute to problem-solving across the data engineering lifecycle.

Requirements

• Overall 5-8 years of total experience and 3-5 years of hands-on experience in Data Engineering.
• Strong hands-on experience with Spark and Scala for ETL development and data pipeline implementation.
• Experience working with the Hadoop ecosystem, including HDFS and Hive.
• Strong expertise in SQL, including query optimization, performance tuning, and handling large-volume datasets.
• Proven experience in building and maintaining large-scale data pipelines and batch processing solutions.
• Hands-on experience with Snowflake for data warehousing, data integration, and performance optimization.
• Exposure to AI/ML-driven data solutions on AWS cloud platforms and integration of data pipelines with analytics and machine learning workloads.
• Strong end-to-end Banking domain knowledge combined with extensive Data Engineering experience in large-scale enterprise environments.
• Hands-on experience with AWS data services such as S3, Glue, Athena, and Redshift.
• Experience with GitHub, GitHub Actions, and TeamCity (or similar CI/CD tools).
• Proficiency in Shell scripting.
• Experience in production support, including incident management and root cause analysis.
• Strong understanding of clean coding standards, secure coding practices, and code review processes.
• Strong problem-solving and communication skills.

Nice to Have

• Experience in Banking, Financial Crime, AML (Anti-Money Laundering), or Fraud domains.
• AWS Certification (e.g., AWS Certified Data Engineer).
• Experience with Oracle Database and Oracle SQL Developer.
• Exposure to AutoSys or other scheduling/orchestration tools such as Apache Airflow.
• Experience in release management and production deployments.
• Exposure to AI/ML use cases and integration of data pipelines with machine learning workloads.
• Experience working in large-scale enterprise cloud transformation programs.
• Knowledge of DevOps and CI/CD best practices beyond GitHub Actions and TeamCity.

We Offer

• Opportunity to work on cutting-edge cloud-based data engineering initiatives within a complex enterprise banking ecosystem.
• Contribute to data-driven business outcomes and innovation.

About the Company

Luxoft is a global provider of digital transformation and technology services, specializing in the banking and financial services sector. We are committed to delivering innovative solutions that drive business success and efficiency.

Location

Bengaluru, India

Employment Type

FULL_TIME