AI Benchmark & Red-Teaming Specialist
Explore this ai & data opportunity. The information below summarizes the supplied role details; check the application page for the latest requirements and availability.
Role highlights & qualifications
- LLMQuality ManagementQA
- Seeking an analytical researcher and red-teaming specialist to build evidence-based web-research benchmark tasks that reliably expose weaknesses in state-of-the-art (SOTA) AI models (e.g., Claude Opus 5 Max). In this role, you will discover hard-to-surface facts, build precise research tasks around them, test them directly against leading LLMs, and diagnose the exact failure mechanisms in the model's search and reasoning paths.
- Your work will directly shape AI evaluation frameworks, ensuring next-generation frontier models are tested against rigorous, non-trivial real-world research scenarios.
- Core Experience: 1+ years in investigative research, fact-checking, AI benchmark task authoring, red-teaming, or technical QA review.
- Benchmark Familiarity: Proven experience authoring, testing, or reviewing tasks for AI benchmarks such as GAIA, BrowseComp, SimpleQA, SWE-bench, Terminal-Bench, or Humanity’s Last Exam (HLE).
- Analytical & OSINT Skills: Intermediate-to-advanced web search and Open Source Intelligence (OSINT) skills; ability to navigate academic archives, local records, or obscure databases.
- Model Trajectory Debugging: Strong analytical capability to inspect model reasoning chains, search logs, and source selection to diagnose root causes of failure.
- Job Responsibilities:
- Obscure Fact Discovery: Deeply research niche topics to identify factual ground truths that cannot be surfaced via simple, direct search queries.
- Task Construction: Draft clear, answerable benchmark tasks designed specifically to expose structural weaknesses in model research strategies.
Information is summarized from the supplied job posting. Please verify location eligibility, compensation, contractual terms and current availability on the application page.