Senior Data Engineer

About the Position

Senior Data Engineer

We are seeking a Senior Data Engineer with deep experience in distributed data systems, Spark-based data processing, and production-grade data platform engineering. These engineers will be expected to operate independently, own technical solutions end-to-end, and contribute to both system design and implementation in a highly automated development environment.

Responsibilities

• Build and maintain scalable batch and streaming data pipelines.
• Optimize Spark and Scala workloads for performance, reliability, and cost efficiency.
• Monitor production pipelines, troubleshoot issues, and resolve data quality defects.
• Partner with stakeholders to define requirements, mappings, and dependable data solutions.
• Orchestrate workflows using Airflow or similar tools.
• Process real-time data using Kafka, Flink, Kafka Streams, or similar technologies.
• Support data modeling, schema design, validation, and quality controls.
• Apply strong engineering practices across testing, versioning, deployment, and environments.
• Review and validate AI/LLM-generated code for production readiness.
• Support observability, SLAs, SLOs, and ongoing platform reliability.

Requirements

• 5+ years of Data Engineering experience.
• Strong Python, Scala, and Spark experience.
• Batch data processing and Spark job optimization.
• Streaming experience with Kafka Streams, Flink, Kafka, or similar.
• Airflow workflow orchestration.
• Data modeling, schema design, and source-to-target mapping.
• Data quality, validation, and production troubleshooting.
• Strong software engineering fundamentals: testing, deployment discipline, versioning, and environment management.
• Ability to review and validate AI/LLM-generated code.
• Awareness of frameworks and tools like spec-led development, spec-kit, agent-skills, etc. preferred.
• Flink experience preferred.
• Large-scale NoSQL experience, especially Scylla, Cassandra, DynamoDB, or similar preferred.
• Clickstream, customer identity, loyalty, booking, commerce, or transactional data experience preferred.
• Experience with SLAs, SLOs, SLIs, data quality thresholds, or observability preferred.
• Experience with GitHub Copilot, Claude, Cursor, or other AI-assisted engineering tools preferred.

Nice to Have

• Flink expertise.
• ScyllaDB, Cassandra, DynamoDB, or other NoSQL platforms.
• Customer Data Platform (CDP), identity resolution, loyalty, clickstream, or booking data experience.
• Experience with SLAs, SLOs, observability, and monitoring.
• GitHub Copilot, Claude, Cursor, or similar AI-assisted development tools.

We Offer

• Opportunity to work in a remote role.
• Collaborative and inclusive work environment.
• Range of benefits that support your well-being and professional growth.

About the Company

At Intellias, where technology takes center stage, people always come before processes. By creating a comfortable atmosphere in our team, we empower individuals to unlock their true potential and achieve extraordinary results. We are committed to fostering equity, diversity, and inclusion as an equal opportunity employer. All applicants will be considered for employment without discrimination based on race, color, religion, age, gender, nationality, disability, sexual orientation, gender identity or expression, veteran status, or any other characteristic protected by applicable law.

Join Intellias for a career where your perspectives and contributions are vital to our shared success.