AI Evaluation Specialist

About the Position

We're seeking an AI Evaluation Specialist to assess and improve enterprise AI assistants. Remote work available in the US, Canada, UK, Ireland, Australia, or New Zealand.

Responsibilities

• Assess AI-generated responses using detailed rubrics and quality standards for accuracy, relevance, and guideline adherence.
• Apply impartial, consistent judgment across a high volume of examples.
• Identify reasoning gaps, logic errors, and tool-use failures, then provide actionable feedback for improvement.
• Write concise feedback that highlights both strengths and opportunities to improve.
• Participate in discussions about rubric interpretation, evolving quality standards, process improvements, and best practices.
• Maintain thorough evaluation records and recommendations to support transparent, traceable assessment workflows.

Requirements

• Experience in grading, quality assurance, editorial review, assessment, annotation, or a comparable field requiring detailed analysis and feedback.
• Advanced daily use of AI assistants, such as ChatGPT or Claude, as a work or productivity tool.
• Ability to synthesize complex information and communicate findings clearly in writing.
• Experience with process improvement, rubric development, or operational quality assessment in an enterprise or educational setting.
• Strong critical thinking, consistency, integrity, fairness, and attention to detail.
• Comfort working independently through large volumes of similar examples.
• A collaborative approach to sharing insights, resolving ambiguous cases, and refining evaluation criteria.

We Offer

• Independent contractor engagement.
• Remote work is available to candidates located in the United States, Canada, United Kingdom, Ireland, Australia, or New Zealand.
• Compensation: $30 to $90 per hour.

About the Company

SaidGig is a company dedicated to improving enterprise AI assistants by leveraging expert evaluation and feedback. We value detailed analysis, critical thinking, and collaborative problem-solving.