Entry-Level AI Evaluation Specialist

About the Position

Entry-Level AI Evaluation Specialist Job at AuraOne Human Data, Remote

AuraOne Human Data is seeking an Entry-Level AI Evaluation Specialist to join our team. This remote role involves evaluating AI outputs across AI training workflows, ensuring workflow correctness, policy adherence, and stakeholder fit. You'll flag operational risks, document the right next steps, and maintain reviewer-quality scores.

Responsibilities

• Review AI outputs against current AI training workflows, playbooks, and firm policy.
• Grade tone, escalation logic, and stakeholder fit on a structured rubric.
• Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
• Capture the right next step so the modeling team can train on it.
• Adjudicate disputed treatments against published playbooks or firm guidance.
• Maintain reviewer-quality scores in inter-rater calibration cycles.

Requirements

• Direct working experience in AI training on real teams.
• Comfort applying multi-page rubrics consistently across long batches.
• Clear written reasoning that names the policy or workflow being applied.
• Strong attention to detail and the ability to flag when a prompt itself is the problem.
• Reliable async availability for at least 10 hours per week.

Nice to Have

• Prior experience training, calibrating, or QA-ing operations teams.
• Familiarity with AI-assisted workflow tooling and its failure modes.
• Bilingual experience for cross-region operations.

Skills

• Operational review
• Policy adherence
• Workflow judgment
• Stakeholder communication
• AI training
• AI model evaluation
• Analytical thinking
• Data annotation
• Literacy
• Presentation development

We Offer

• Remote work from anywhere in the US.
• Independent specialist contractor role.
• Compensation: $30 per hour.

About the Company

AuraOne Human Data is a leading company in the field of AI evaluation and training. We are committed to ensuring that AI outputs meet the highest standards of workflow correctness, policy adherence, and stakeholder fit.

Application Process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.