Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

About the Position

Senior Python Data & AI Engineer (Web Crawling) - Project setup requires service delivery from a nearshore country (m/f/d)

Freelance/Contracting project

Location: Dortmund

Start date: asap

Reference number: 899110/1

---

Responsibilities

• Analysis and Indexing of Appropriate Web Sources: The contractor shall identify and evaluate appropriate sources.
• Design of the crawling and extraction solution: Development of a technical target architecture.
• Development of Crawling, Extraction, and AI Components.
• Data Structuring and Provision: Transformation of the extracted information.
• Quality Assurance and Domain Validation: Implementation of procedures and a Databricks app.
• Pilot Testing.
• Documentation and Handover.

---

Requirements

• Web Crawling & Web Scraping: Scrapy, Beautiful Soup, or comparable frameworks, HTML, DOM, and table analysis, working with dynamic web pages and download structures.
• Information Extraction & Generative AI: LLM & NLP, prompt engineering and structured output formats, classification, mapping, and duplicate detection.
• Interface and Document Processing: Rest APIs, PDF and datasheet extraction, Table extraction and schema mapping.
• Data Engineering & Databricks: Python, Pandas and PySpark, Delta Lake, and development of Databricks apps.

---

We Offer

• A very renowned company.
• Possibilities to work remote.

---

About the Company

Hays is the biggest privately owned IT recruitment agency in Germany. We offer the best jobs for every career level – whether you are interested in vacancies in agile SMEs or international DAX groups. Hays masters the entire IT job spectrum, from support to software architecture or digitalisation. Our highly specialized consultants can cater to your every wish and expectation and will prepare you for interviews and contract negotiations. Give it a try and learn what the market has to offer – our services are free of charge, non-binding and discreet! We look forward to hearing from you.

---

Contact at Hays

Michelle Winkels

[email protected]

Reference number: 899110/1

© Copyright Hays plc,. The HAYS word, the H devices, 'Hays Working for your tomorrow' and 'Powering the world of work' and associated logos and artwork are trademarks of Hays PLC. The H devices are original designs protected by registration in many countries. All rights are reserved.