Senior Machine Learning Engineer

About the Position

We are looking for a Senior Machine Learning Engineer to join our team in Palo Alto, CA, and work on SAGE, Rubrik's Semantic AI Governance Engine. This role is pivotal in monitoring, governing, and remediating autonomous AI agents in real time.

About the Team & Role

SAGE is a groundbreaking system designed to enforce governance policies in natural language, utilizing custom small language models (SLMs) that act as judges on every agent action. These models are fast and accurate enough to be trusted with critical decisions on production traffic.

As an Applied ML Engineer on the SAGE team, you will be responsible for the entire model lifecycle: curating data, training small models, serving them at production latency, and integrating real customer signals to improve model performance. Your work will directly impact AI safety and governance, ensuring that small, specialized models can outperform larger language models in critical tasks.

Responsibilities

• Training, Fine-Tuning, and Distilling Production Small Language Models and Classifiers (25% of time)

  • Own the full training lifecycle for SLMs and classifiers, including base-model selection, supervised fine-tuning, preference optimization, and distillation.
  • Design adversarial training pipelines to improve model robustness.
  • Optimize accuracy, latency, and cost for governance-specific tasks.

• Engineering High-Performance Model Serving and Inference Infrastructure (25% of time)

  • Design multi-stage inference pipelines for real-time enforcement and high-throughput batch workloads.
  • Optimize live deployments to minimize inference cost while maintaining sub-second P99 SLOs.
  • Build serving-layer infrastructure to block agent prompts, responses, and tool calls in real time.

• Building Synthetic Data Pipelines and Online + Offline Evaluation Frameworks (20% of time)

  • Design automated data curation pipelines to mine live customer environments for high-value training examples.
  • Build automated policy back-testing and online evaluation systems to ensure model quality.

• Insights Mining, Failure Diagnosis, and Adaptive Model Improvement (15% of time)

  • Build memory and context harnesses to fuse data sensitivity, identity, and historical agent behavior into real-time enforcement decisions.
  • Mine agent insights to surface security gaps and improve policies.
  • Diagnose model failures and distinguish root causes to implement effective fixes.

• Cross-Functional Collaboration and Translating Customer Reality into Modeling Problems (15% of time)

  • Provide technical leadership and mentor engineers.
  • Translate customer requirements into well-scoped modeling problems.
  • Communicate model behavior and limitations to non-ML stakeholders.
  • Collaborate with cross-functional teams to integrate new SLMs into the enforcement path.

Requirements

• Education: A Bachelor's degree (or higher) in Computer Science, Machine Learning, Computer Engineering, Statistics, or a closely related technical field.
• Experience: 2+ years of professional ML experience with demonstrable end-to-end production ownership.
• Technical Skills:
  • Proficiency in Python and PyTorch for production-grade training and evaluation.
  • Hands-on experience training, fine-tuning, or distilling language models or classifiers in a production setting.
  • Production experience with serving frameworks, including optimization involving continuous batching, KV-cache strategy, and inference-time quantization.
  • Experience designing closed-loop ML systems, including the eval, telemetry, data-curation, and synthetic-data infrastructure.
  • Comfort operating at production scale, including debugging models that handle high QPS in safety-critical request paths.

Preferred Qualifications

• Deep background in AI safety and red-teaming.
• Expertise in model evaluation methodology, specifically building "LLM-as-judge" pipelines, calibration monitoring, and adversarial benchmarks.
• Experience with context-fusion and retrieval systems.
• Production experience with low-latency inference for streaming or safety-critical request paths.
• Mastery of label-efficient training and data mining.
• Hands-on knowledge distillation experience.
• Familiarity with the agentic ecosystem.
• Active open-source contributions to mainstream ML training, serving, or evaluation libraries.

We Offer

• Competitive Salary: $188,500-$282,700 USD
• Bonus Potential
• Equity
• Benefits

About the Company

Rubrik is the Security and AI Operations Company, delivering a platform built at the intersection of data, identity, and AI. Our platform enables agentic cyber resilience and trusted AI operations for enterprises worldwide.

Inclusion @ Rubrik

At Rubrik, we are dedicated to fostering a culture where people from all backgrounds are valued, feel they belong, and believe they can succeed. Our commitment to inclusion is at the heart of our mission to secure the world’s data.

Equal Opportunity Employer/Veterans/Disabled

Rubrik is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, genetic information, protected veteran status, or any other status protected by applicable law.

Apply For This Job

Apply Here