About the Position

Hi there! If you’re looking for a high-impact position in an ambitious software house, we’ve got a match for you!

Currently, we are searching for a full-time Lead Data Engineer (Databricks) to cooperate with our US client - BioPharma organization. The mission is to unlock the potential of AI/ML to improve the lives of patients. Our client delivers powerful cloud software and services specifically designed to meet the evolving needs of the BioPharma sector.

You will be an integral part of the hands-on delivery of reliable, scalable data pipelines and datasets that power AI discoverability, analytics, reporting, and workflow automation across scientific, clinical, and enterprise data.

⚠️ Due to the client's location, the team is working till 6:00 PM CEST.

Responsibilities

Build, evolve, and operate data ingestion and processing capabilities for structured, semi-structured, and unstructured data, supporting the transition from early prototypes through early adoption and general release.
Implement and maintain rich metadata and data quality practices that enable cross-record querying, traceability, and AI-ready data access across experiments, files, inventory, and workflows.
Work with architects, AI engineers, workflow engineers, and domain experts to ensure data is usable, performant, and trustworthy for downstream GenAI and analytics use cases.
Lead architectural decisions and work with senior business and technology stakeholders.
Guide and mentor data and engineering teams.

Requirements

Experience as a Lead / Principal Data Architect in multinational environments.
Deep knowledge of data modelling, integration, warehousing, and modern data platforms.
Strong hands-on Python and complex SQL, with production Spark experience on Databricks (Structured Streaming, Auto Loader, Delta Lake, Unity Catalog) to support analytics, AI/ML, and enterprise application use cases.
Practice working with structured and unstructured data across the full lifecycle: ingestion, transformation, enrichment, indexing, and retention, delivered through disciplined engineering practice including infrastructure-as-code, CI/CD, and automated testing of pipelines.
Proven experience ingesting from heterogeneous operational sources including relational (Oracle, PostgreSQL) and document stores (MongoDB), using Full load + CDC or equivalent replication patterns, with practical handling of schema drift, deletes, and late or out-of-order data.
Skills in data modelling across a layered architecture, with clear contracts between raw, curated, and serving layers, supporting cross-entity queries and contextual linking, and performant data access patterns.

Nice to Have

Experience building unstructured document pipelines: parsing and extraction from PDF, Office and scanned formats, chunking, embedding generation, and maintaining searchable indexes at scale.

Salary

170-210 PLN netto per month - B2B

gross per month - Employment Contract

We Offer

Multisport: Access to sport facilities of your choice.
Private healthcare insurance: Coverage for you and your family.
Internal initiatives: Regular internal events such as game or movie nights, bar-hopping, or sports.
Top-class equipment: MacBook Pro 16" with keyboard, mouse, headphones, and external monitor(s) provided.

Recruitment Process

STEP 1: Application

After you apply, we will review your resume and get back to you. If your experience matches the requirements, we will schedule a quick screening call to get to know you.

STEP 2: Interview

After successful screening, you will be invited to interview with one of our delivery managers alongside with a software engineer who will ask you technical questions.

STEP 3: Decision

After the interviews, we will get back to you as soon as possible with feedback and, if all goes well, a job offer. The feedback process usually takes up to 5 business days.

Locations

Remote from Poland
Pune, Maharashtra, India
San Jose, United States
Warsaw, Poland
Gdynia, Poland

About the Company

We are constantly seeking for top talent all over the world. If you did not find the job position that suits your skills and experience, do not hesitate to contact us directly - we will be happy to talk to you about your future with us.

Contact Us

San Jose Office

84 W Santa Clara Street

Suite 700

San Jose CA 95113

United States

Warsaw Office

al. Niepodległości 18

Warsaw 02-653

Poland

Privacy policy / Cookie policy

Got a question?

Join Acaisoft & let's start our adventure together!

Natalia Przybył

Recruiter

Your success is our goal.

Lead Data Engineer (Databricks)

Job description