Senior Data Engineer

About the Position

We're seeking a Senior Data Engineer with 10+ years of experience to join our team in Pune, India, focusing on designing and building optimized data pipelines.

Responsibilities

• Designing and building optimized data pipelines using cutting-edge technologies in a cloud environment to drive analytical insights.
• Constructing infrastructure for efficient ETL processes from various sources and storage systems.
• Collaborating closely with Product Managers and Business Managers to design technical solutions aligned with business requirements.
• Leading the implementation of algorithms and prototypes to transform raw data into useful information.
• Architecting, designing, and maintaining database pipeline architectures, ensuring readiness for AI/ML transformations.
• Creating innovative data validation methods and data analysis tools.
• Ensuring compliance with data governance and security policies.
• Interpreting data trends and patterns to establish operational alerts.
• Developing analytical tools, programs, and reporting mechanisms.
• Conducting complex data analysis and presenting results effectively.
• Preparing data for prescriptive and predictive modeling.
• Continuously exploring opportunities to enhance data quality and reliability.
• Applying strong programming and problem-solving skills to develop scalable solutions.

Requirements

• Must have

  • 10+ years of hands-on experience designing, building, deploying, testing, maintaining, monitoring, and owning scalable, resilient, and distributed data pipelines.
  • High proficiency in Scala/Java and Spark for applied large-scale data processing.
  • Expertise with big data technologies, including Spark, Data Lake, Delta Lake, and Hive.
  • Solid understanding of batch and streaming data processing techniques.
  • Proficient knowledge of the Data Lifecycle Management process, including data collection, access, use, storage, transfer, and deletion.
  • Expert-level ability to write complex, optimized SQL queries across extensive data volumes.
  • Experience with RDBMS and OLAP databases like MySQL, Redshift.
  • Familiarity with Agile methodologies.
  • Obsession for service observability, instrumentation, monitoring, and alerting.
  • Knowledge or experience in architectural best practices for building data lakes.

• Nice to have

  • Passion for testing strategy, problem-solving, and continuous learning.
  • Willingness to acquire new skills and knowledge.
  • Possess a product/engineering mindset to drive impactful data solutions.
  • Experience working in distributed environments with global teams.

We Offer

• Competitive salary
• Opportunity to work with cutting-edge technologies
• Collaborative and innovative work environment
• Professional growth and development opportunities

About the Company

Luxoft is a global leader in digital transformation, providing innovative technology solutions to clients across various industries. We are committed to fostering a culture of continuous learning and professional development.

Apply Now

Interested candidates are encouraged to apply with their updated resume and portfolio showcasing relevant experience.