Role Overview
As a Staff Machine Learning Engineer on the Evaluation team at Waymo, you will lead the design and implementation of novel evaluation frameworks for Vision-Language Models (VLMs) and Large Language Models (LLMs). Your work will directly impact the safety and reliability of Waymo's autonomous driving technology by ensuring our models perform robustly across diverse real-world scenarios.
Key Responsibilities
- Architect and develop scalable evaluation pipelines for state-of-the-art VLM/LLM models
- Design automated metrics and benchmarks to assess model behavior, fairness, and robustness
- Collaborate with research and engineering teams to identify evaluation gaps and propose novel methodologies
- Analyze large-scale model performance data to uncover failure modes and drive model improvements
- Mentor junior engineers and establish best practices for model evaluation and validation
Requirements
- Education: MS/PhD in Computer Science, Machine Learning, AI, or a related technical field
- Experience: 7+ years of industry experience in machine learning or AI engineering, with a focus on NLP or computer vision
- Technical Skills:
- Deep expertise in LLM/VLM architectures (e.g., GPT, LLaMA, CLIP, Flamingo)
- Proficiency in Python and ML frameworks (PyTorch, TensorFlow)
- Strong understanding of evaluation metrics and statistical analysis
- Experience with large-scale data processing (Spark, BigQuery, etc.)
- Soft Skills: Excellent problem-solving, communication, and cross-team collaboration abilities
Benefits
- Compensation: Competitive salary and equity package
- Health & Wellness: Comprehensive health, dental, and vision insurance; gym membership reimbursement
- Work-Life Balance: Flexible hours, remote work options, and generous PTO
- Financial Growth: 401(k) with company match, employee stock purchase plan
- Professional Development: Annual learning stipend, conference attendance, and internal mentorship programs
- Perks: Daily catered lunches, snacks, and on-site childcare support
- Culture: Inclusive environment with employee resource groups and team offsites