AI Operations Engineer

About the Position

We are seeking an AI Operations Engineer to design and operate the end-to-end infrastructure that supports production AI systems in Lisbon, Portugal.

Responsibilities

• Design and implement scalable, source-agnostic ingestion pipelines for production ML and LLM data.
• Build and maintain data processing workflows including ingestion, redaction, storage, classification, and slicing of production signals.
• Define and implement data storage strategies, including tiering, retention policies, and privacy-aware data handling.
• Develop observability and debugging tools, including dashboards and query systems for production AI systems.
• Implement evaluation and monitoring frameworks, including offline evaluation sets, online autoreters, and regression detection systems.
• Build automated triage systems to identify, classify, and surface production failures.
• Develop and maintain PII redaction mechanisms and enforce data governance and compliance policies at ingestion level.
• Design and operate LLM evaluation mining workflows to continuously improve model and prompt performance.
• Implement alerting systems to detect regressions across model and prompt deployments.
• Collaborate with AI, engineering, and product teams to ensure robustness and reliability of production AI systems.
• Evaluate and select tooling, infrastructure, and hosting strategies for ML/LLM platforms.
• Own the operational reliability of the entire ML/LLM data and evaluation pipeline.

Requirements

• Proven experience building and operating production-grade data platforms or ML systems, including ingestion, storage, access control, monitoring, and on-call responsibilities.
• Hands-on experience developing ML/LLM evaluation systems (e.g., regression test sets, autoreters, LLM-as-a-judge frameworks, or golden datasets).
• Strong understanding of LLM observability, tracing, and debugging tools.
• Experience implementing data privacy controls such as PII redaction in production environments.
• Deep understanding of failure modes in ML and LLM systems (hallucinations, retrieval failures, agent loops, ASR/TTS degradation, prompt/model regressions).
• Strong production-level Python engineering skills, with a hands-on mindset.
• Solid understanding of data pipelines, system reliability, and distributed data processing.

Nice to Have

• Experience in multi-tenant or SaaS architectures with strict data isolation requirements.
• Familiarity with Azure and/or AWS cloud ecosystems.
• Experience making infrastructure trade-off decisions between managed services and self-hosted solutions.
• Knowledge of vector databases, embedding techniques, and clustering or unsupervised failure detection methods.
• Experience with data versioning tools such as LakeFS, DVC, or Delta Lake.
• Familiarity with GDPR, data deletion workflows, and compliance-driven data systems.
• Exposure to embedded, automotive, or constrained environments.
• Experience with non-English language model evaluation or multilingual datasets.
• Experience working with LLM APIs such as Anthropic Claude, OpenAI models, or open-source alternatives.
• Familiarity with CI/CD workflows, GitHub-based development, and modern DevOps practices.
• Experience with dashboard development using TypeScript or similar frontend technologies.

We Offer

• Competitive salary
• Full-time employment
• Hybrid work location type

About the Company

Tieto Iberia is part of the Tieto Group and combines local expertise with the global strength of Tieto, a leading software and digital engineering services company with worldwide presence and capabilities. We provide customers across different industries with mission-critical solutions through our specialized software businesses Tieto Caretech, Tieto Banktech, and Tieto Indtech as well as Tieto Tech Consulting business.

Our around 14,000 talented vertical software, design, cloud, and AI experts are dedicated to empowering our customers to succeed and innovate with the latest technology.

Tieto’s annual revenue is approximately EUR 2 billion. The company’s shares are listed on the NASDAQ exchange in Helsinki and Stockholm, as well as on Oslo Børs.

At Tieto, we believe in the power of diversity, equity, and inclusion. We encourage applicants of all backgrounds, genders (m/f/d), and walks of life to join our team, as we believe that this fosters an inspiring workplace and fuels innovation. Our commitment to openness, trust, and diversity is at the heart of our mission to create digital futures that benefit businesses, societies, and humanity.

Diversity, equity and inclusion | Sustainability | Tieto