Senior Data Engineer

About the Position

We are seeking a Senior Data Engineer with deep experience in distributed data systems, Spark-based data processing, and production-grade data platform engineering. These engineers will be expected to operate independently, own technical solutions end-to-end, and contribute to both system design and implementation in a highly automated development environment.

Responsibilities

• Own end-to-end delivery of data platform and customer data initiatives, including migrations, pipeline repair, streaming updates, and production troubleshooting.
• Support Kafka / Confluent / streaming-related work, including migration, debugging, producer/consumer flow updates, and downstream impact validation.
• Work with batch and streaming data pipelines, including Spark-based jobs, ETL/ELT flows, API ingestion, and data product/source migrations.
• Analyze existing Java, Scala, Python, SQL, and configuration-heavy codebases, using AI-assisted tools where appropriate.
• Lead investigation and resolution of production data pipeline issues, including broken jobs, schema changes, data quality problems, and downstream failures.
• Validate migrated or updated pipelines to ensure data correctness, performance, reliability, and production readiness. Identify migration risks, hidden dependencies, edge cases, and potential downstream breakages.
• Collaborate with engineering managers, product teams, and stakeholders to clarify requirements, provide updates, and align on delivery priorities.
• Work with internal documentation, proprietary tools, cloud-native environments, and AI-agentic engineering workspaces. Use AI-assisted development tools to accelerate delivery while maintaining strong human review, validation, and quality control.

Requirements

• 5+ years of Data Engineering, Data Platform Engineering, or similar production data roles.
• Strong hands-on experience with production ETL/ELT pipelines, batch processing, streaming data flows, and pipeline troubleshooting.
• Strong understanding of Kafka / Confluent / event-driven architecture, including topics, producers, consumers, and streaming pipeline behavior.
• Good Java experience or strong ability to understand Java-based data/platform systems.
• Ability to read and work with Scala-based pipelines or frameworks; deep Scala expertise is a plus, but not mandatory.
• Ability to read and understand Python, especially for ETL, API ingestion, scripting, or pipeline support.
• Strong SQL skills and practical understanding of data modeling, schemas, source-to-target mapping, and downstream impact analysis.
• Experience with Spark, PySpark, or similar distributed data processing frameworks.
• Experience with cloud-native environments, preferably AWS, and basic understanding of Docker, Kubernetes, CLI tools, and deployment/runtime concepts.
• Experience with migration projects, production support, pipeline repair, or platform modernization.
• Practical experience using AI-assisted engineering tools such as GitHub Copilot, Claude Code, Cursor, or similar.
• Understanding of agentic workflows, prompting, context management, AI output validation, and AI-assisted debugging is a strong plus.
• Ability to work independently, manage ambiguity, communicate clearly, and support U.S. working hours overlap.

Skills

• Kafka
• Scala
• Spark

We Offer

• Opportunity to work in a modern engineering environment that leverages agentic development and AI-assisted software delivery.
• Competitive salary and benefits package.
• Opportunity to work remotely from Brazil, Colombia, Mexico, or Peru.

About the Company

At Intellias, where technology takes center stage, people always come before processes. By creating a comfortable atmosphere in our team, we empower individuals to unlock their true potential and achieve extraordinary results. That’s why we offer a range of benefits that support your well-being and charge your professional growth. We are committed to fostering equity, diversity, and inclusion as an equal opportunity employer. All applicants will be considered for employment without discrimination based on race, color, religion, age, gender, nationality, disability, sexual orientation, gender identity or expression, veteran status, or any other characteristic protected by applicable law. We welcome and celebrate the uniqueness of every individual. Join Intellias for a career where your perspectives and contributions are vital to our shared success.