AgileEngine logo

Senior Data Engineer

AgileEngine·Salary not specified

Primary stack

dbtPythonAirflow

Job description

About the Position

We are looking for a Senior Data Engineer with deep Airflow and dbt expertise to support Medallion Architecture implementation, harden pipeline reliability through automated testing and failure alerting, and migrate scheduled ETL jobs into orchestrated Airflow DAGs within a modern Snowflake data stack. You will build complex DAG architectures using Python, audit dbt codebases for performance and cost optimization, maintain integration points between Paradime and Airflow, and ensure zero downtime for downstream BI tools during ongoing infrastructure changes. Eastern Time Zone overlap preferred for daily team standups.

Responsibilities

  • Support implementation of bronze/silver/gold layering across the Snowflake warehouse, working with the Lead Data Engineer on architecture and modeling decisions.
  • Build automated testing and failure alerting across Paradime (ETL) and Airflow (orchestration), including reworking select existing jobs so failures are clearly attributable.
  • Migrate select schedule-based Paradime jobs into orchestrated Airflow DAGs.
  • Work within established data governance and access controls when building or modifying pipelines touching PII data.
  • Translate dbt transform schedules and dependencies into scalable, dynamic Airflow DAGs using modern Python design patterns (@task decorators, dynamic task mapping).
  • Maintain and evolve integration points between Paradime and Airflow as both tools continue to be used in production.
  • Audit dbt codebases and pipeline execution paths to identify bottlenecks, reducing compute runtime and cost.
  • Partner with the Lead Data Engineer to adjust the modern data stack architecture to support future AI semantic layers and downstream consumption.
  • Establish and maintain automated error handling, alerting, and failure recovery routines across the pipeline stack.
  • Ensure no downtime or data loss for downstream BI (Sigma) and analytics tools during ongoing infrastructure changes.
  • Take on additional data engineering priorities as assigned, based on evolving business needs.

Requirements

  • 4+ years of experience as a Data Engineer.
  • Apache Airflow / Astronomer: Substantial hands-on production experience building, debugging, and managing complex DAG architectures using Python.
  • dbt (Data Build Tool) Expertise: Strong proficiency with dbt Core architecture, macros, packages, custom materializations, CI/CD patterns (state deferral), and environment configurations.
  • Advanced Git & Data CI/CD: Track record building deployment pipelines and automated regression testing for data teams using GitHub Actions or similar.
  • Production Data Engineering: Solid general software/data engineering background writing clean, maintainable, modular Python and SQL.
  • Upper-intermediate English level.

Nice to Have

  • Snowflake expertise: tuning compute (warehouse sizing, query profiling, clustering), RBAC/security architecture.
  • Infrastructure as Code: Terraform, Docker, or CloudFormation.
  • AWS S3 experience as part of a data pipeline (e.g., staging/raw ingestion layers).
  • Enterprise retail, supply chain, or inventory systems data background.

We Offer

  • Growth without limits: build your skills through mentorship, internal TechTalks, challenging projects, and a dedicated annual learning budget.
  • Competitive compensation: get recognition that reflects your skills and impact, with regular performance and compensation reviews.
  • Flexibility: work 100% remotely with flexible hours that support focus, autonomy, and a healthy work rhythm.
  • Meaningful, modern projects: build impactful products using modern technologies alongside global teams and leading brands.
  • Collaborative culture: join a supportive environment with zero micromanagement where ideas are welcomed and contributions are recognized.
  • Well-being & support: access local well-being programs and people-focused support tailored to your location.

About the Company

We are a remote-first company, so all our positions are remote. Schedules are flexible, but the main requirement is to have an overlap with your client’s schedule to ensure smooth collaboration. Specific details depend on the project and are discussed during the interview process.

Hiring Process

  1. Submit a short application form.
  2. Pass a quick test (30’- 60’).
  3. Record a short video to fast-track your application process.
  4. Meet our team for a technical interview.
  5. Get an offer and welcome to AgileEngine!

Work Format and Schedule

We are a remote-first company, so all our positions are remote. Schedules are flexible, but the main requirement is to have an overlap with your client’s schedule to ensure smooth collaboration. Specific details depend on the project and are discussed during the interview process.

English Proficiency

We look for Upper-Intermediate (B2) proficiency or higher. Since you will be collaborating with international teams and global clients (including Fortune 500 companies), English is a part of your daily work.

Work Equipment

It depends on your location. We provide equipment in Ukraine, Poland, Argentina, Colombia, Mexico, Brazil, Guatemala, Portugal, Spain, and India. In the USA, equipment is usually provided by the client; otherwise, you will be expected to use your own setup.

Professional Growth Opportunities

We support continuous growth through personalized development paths tailored to each expert. Also, you can expect:

  • An annual learning and development budget.
  • Internal workshops, tech talks, and mentorship programs.
  • Opportunities to switch projects or grow into new roles over time.

Team and Tools

Teams vary by project, but you’ll usually work with a mix of experts across different fields, along with a delivery or project manager who supports collaboration and onboarding.

For day-to-day work, teams commonly use tools like Jira and Google Workspace, along with communication platforms such as Google Chat. Depending on the client, you may also work with Slack, Confluence, Notion, or similar tools.

Need Help?

Want more details? See full FAQ

If something doesn’t work or you have questions regarding your application process, drop us a line.

© AgileEngine. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.

AgileEngine is a privately held company established in 2010 and HQed in the Washington DC area. We rank among the fastest-growing US companies on the Inc 5000 list and the top-3 software developers in DC on Clutch.

More at AgileEngine

All 81 roles

Similar jobs

Popular searches