About the Position

Turn agent quality into reproducible evidence across repeated runs, adversarial inputs, and changing models.

Responsibilities

• Create task suites, scoring rules, and reproducible execution environments.
• Investigate incorrect tool calls, data leakage, brittle recovery, and inconsistent outcomes.
• Build regression checks that connect model behavior to release decisions.

Requirements

• Experience testing complex software or evaluating ML systems.
• Ability to analyze noisy results and distinguish model, data, and harness failures.
• Strong Python skills and clear written communication of experimental findings.

Useful Evidence

• An evaluation or test suite you designed and a failure it exposed.
• Experience with security testing, statistics, or agent benchmarks is useful.

Working Terms

• Startup culture, a goal-oriented team, and a research mindset.
• The opportunity to apply your engineering skills to tools and systems for fellow engineers and help shape the future of AI.
• Latest-generation MacBook Pro.
• An in-house GPU cluster for training and experimentation.
• 20 working days of annual leave.
• English courses, educational events, and conferences.
• Medical insurance.

Tools & Systems

• Python
• Inspect
• AIBraintrust
• pytest
• Docker

Relevant experience matters more than knowing every tool listed.

Start a Conversation

Send a CV or profile and one piece of work that helps us understand your contribution.

Location: Kyiv / remote

Experience Level: Mid / Senior

Working Arrangement: Arrangement to be discussed

Apply by email. Please share only work you have permission to disclose.

How we use application data

More roles in Agentic Lab. Explore the laboratory.

Connect with us at Agentic Lab.

Agent Evaluation Engineer