Data Engineer

About the Position

We are seeking a Data Engineer to join our team in Colombia and Mexico, developing an MLOps platform for machine learning model development and deployment.

Skills

• Data Science
• Apache Spark
• Machine Learning Model Development
• Azure Data Factory
• Azure Data Lake Storage

Responsibilities

• Migrate existing Spark feature engineering pipelines to Azure
• Collaborate with the team to identify and develop a forecasting model
• Support the deployment of the developed machine learning model to production
• Contribute to the design and development of the CI/CD pipeline feeding the Databricks Feature Store
• Work closely with Data Scientists and engineers to align feature engineering with ML model requirements
• Ensure the reliability and scalability of data pipelines across the platform
• Troubleshoot and optimize existing Spark-based workflows during migration

Requirements

• 2+ years of experience working with PySpark
• Strong background in Data Science core skills
• Proficiency in feature engineering and machine learning pipeline development
• Experience migrating data pipelines to cloud environments
• English proficiency at B2 level or higher

Nice to Have

• Familiarity with MLflow and Docker
• Knowledge of Azure Kubernetes Services, Azure Data Factory, and Azure Databricks
• Understanding of ML model deployment practices

We Offer

• International projects with top brands
• Work with global teams of highly skilled, diverse peers
• Healthcare benefits
• Employee financial programs
• Paid time off and sick leave
• Upskilling, reskilling and certification courses
• Unlimited access to the LinkedIn Learning library and 22,000+ courses
• Global career opportunities
• Volunteer and community involvement opportunities
• EPAM Employee Groups
• Award-winning culture recognized by Glassdoor, Newsweek, and LinkedIn