Job Description, Responsibilities & Requirements
About the Position
AI Evaluation & Benchmarking Engineer
We are looking for an AI Evaluation & Benchmarking Engineer with experience in reinforcement learning, LLM-based agents, experiment design, benchmarking, and performance evaluation. This role will support the productionization of an AI evaluation platform used to execute and evaluate algorithms within video game environments.
The engineer will develop and integrate baseline algorithms, reinforcement learning approaches, LLM-based agents, and externally developed algorithms into the platform. This person will also design experiments, define evaluation metrics, run benchmarks, analyze performance, and serve as a primary power user of the platform to provide feedback to the engineering team.
Responsibilities
- Develop and integrate baseline algorithms, reinforcement learning approaches, LLM-based agents, and externally developed algorithms into the platform.
- Design experiments and define evaluation metrics.
- Run benchmarks and analyze performance.
- Serve as a primary power user of the platform to provide feedback to the engineering team.
Requirements
- Hands-on reinforcement learning experience.
- Experience using LLMs for agents, evaluation, reasoning, automation, or benchmark workflows.
- Strong Python experience for ML, data workflows, experimentation, and analysis.
- Experience designing and running experiments with statistical and analytical rigor.
- Strong understanding of evaluation metrics, scoring frameworks, performance comparison, and benchmark design.
- Experience analyzing structured logs, run outputs, model/agent performance, and experiment results.
- Ability to work across APIs, logs, CLI/tools, data structures, and platform workflows.
- Strong communication skills to translate experiment findings into platform improvement requirements.
- Ability to work inside client-owned repositories, infrastructure, workflows, and security controls.
Preferred Skills
- Experience with game environments, simulation environments, Gym-like interfaces, RL environments, or agentic AI test harnesses.
- Experience benchmarking LLM agents, RL policies, autonomous agents, or hybrid AI systems.
- Experience with experiment tracking, run comparison tools, metrics dashboards, or evaluation pipelines.
- Experience with prompt engineering, agent orchestration, tool use, and LLM evaluation frameworks.
We Offer
- Competitive salary range of $150K to $180K (base salary only).
- On-site work model in Minneapolis, MN.
- Opportunity to work in a cutting-edge AI evaluation platform.
- Collaborative environment with strong engineering and ML expertise.
About the Company
GlobalLogic is a leading company in digital engineering and technology solutions, providing innovative services to clients across various industries. We are committed to driving digital transformation and delivering exceptional value to our clients.
Apply now to join our team and contribute to the advancement of AI evaluation and benchmarking.