Lead Big Data Engineer

About the Position

Lead Big Data Engineer

In this role, you will lead complex data migration and modernization programs, helping organizations transform legacy platforms into scalable cloud-native environments. Leveraging AWS technologies, distributed data processing frameworks, and AI-assisted development tools, you will drive large-scale transformation initiatives that accelerate delivery, improve data quality, and unlock business value from modern data platforms.

Responsibilities

• Design and implement data migration workflows and cloud-based data solutions on AWS
• Build and maintain scalable batch and streaming data pipelines using Python and PySpark
• Analyze legacy applications, databases, and data processing workflows to define migration approaches and implementation strategies
• Leverage AI-assisted development tools and LLM-powered solutions to accelerate migration, validation, and automation activities
• Design and implement data quality, reconciliation, and validation frameworks for migration projects
• Develop automation solutions using AWS services such as Step Functions, Lambda, and CloudWatch
• Support modernization initiatives, including migration from legacy technologies to cloud-native platforms
• Collaborate with stakeholders to define requirements, identify risks, and support delivery planning and execution
• Provide technical guidance to engineering teams and promote best practices in data engineering and cloud migration
• Contribute to architecture discussions, technical reviews, and continuous improvement initiatives

Requirements

• 7+ years of experience in Data Engineering, Big Data, or Data Migration projects
• Strong proficiency in Python and SQL
• Advanced experience with PySpark and distributed data processing frameworks
• Hands-on experience with AWS services such as Step Functions, S3, Lambda, and CloudWatch
• Strong knowledge of ETL processes and scalable data pipeline development
• Experience leveraging AI-assisted workflows and Large Language Models (LLMs) in engineering activities
• Understanding of migration strategies, legacy system modernization, and cloud transformation initiatives
• Experience with data validation, quality assurance, and migration governance processes
• Proficiency in guiding technical implementation activities and collaborating effectively across engineering teams
• Upper-intermediate or higher level of English

Nice to Have

• Experience with video, image, or audio processing pipelines for AI applications
• Experience with IoT data platforms and data processing solutions
• Familiarity with NVIDIA technologies, GPU-accelerated computing, or AI infrastructure
• Knowledge of Infrastructure as Code: Terraform, CloudFormation, or similar

We Offer

Flexible Work Model

Work from home, from the office, or in a hybrid format that supports focus and collaboration.

Compensation & Benefits

Competitive, market-based pay, benchmarked by role and location - plus health coverage, paid time off, wellness support, and learning opportunities.

People-first Leadership

Approachable leaders who communicate openly, keep teams close to the strategy, and support long-term planning.

Advanced tech communities

Stay close to AI/ML, Cloud, Quantum Computing, IoT, and Robotics communities, with projects built on modern frameworks.

About the Company

We are a digital engineering and technology consulting company where expertise grows alongside people. For more than 30 years, we have been elevating technology: helping organizations navigate complex business challenges by combining deep engineering knowledge with thoughtful, research-backed innovation. Our teams work across key areas: digital engineering, data and analytics, Cloud, and AI/ML. In each, we deliver practical, scalable solutions rooted in real business needs and measurable human impact.

You bring your perspective and ambition. We create an environment where your work meets clarity, confidence, and purpose.