Job Title: Senior Product Manager
Job Responsibilities:
- Strong Legal Research: The Senior Product Manager will deconstruct complex legal research and related litigation workflows to identify opportunities for generative AI to enhance specific tasks such as legal research, brief analysis, and motion improvement. They will define the evaluation criteria necessary to ensure AI outputs meet the high standards of legal practice.
- Evaluation Strategy & Framework Development: The role involves defining, designing, and driving the implementation of robust, scalable automated evaluation frameworks and metrics for generative AI in the legal domain. This includes assessing critical factors like factual accuracy, legal soundness, citation validity, hallucination detection, bias mitigation, and relevance.
- Data Management, Testing & Validation: The Senior Product Manager will oversee the creation and curation of high-quality datasets for auto-evaluation, including ground truth data and adversarial examples. They will lead the execution of automated testing and validation pipelines, identifying and addressing performance bottlenecks and areas for improvement.
- Cross-Functional Leadership: Partnering effectively with research scientists, engineers, attorney editors, and product managers to translate complex requirements into actionable plans, align on evaluation priorities, and ensure seamless execution of generative AI initiatives is a key responsibility.
- Technical Insight & Innovation: The role requires staying informed about the latest advancements in LLMs, generative AI evaluation techniques, and prompt engineering. The Senior Product Manager will apply this knowledge to innovate and continuously improve evaluation methodologies and tools.
- Stakeholder Communication: Articulating evaluation strategies, results, risks, and insights to diverse audiences, including senior leadership, technical teams, and legal experts, is essential for driving informed decision-making.
- Team Leadership: The Senior Product Manager will mentor and guide junior members of the auto evaluation team, fostering a culture of excellence and innovation.
- Market & User Insights: Conducting market research, competitive analysis, and user research (interviews, surveys, user testing) to deeply understand the needs, pain points, and expectations of legal professionals regarding AI-generated content and its evaluation.
Education and Experience Information:
- Legal Expertise (Required): The candidate must hold a . from an accredited law school and possess a deep understanding of legal research methods, concepts, and their application within litigation workflows, along with the ability to dissect complex legal issues.
- Product Management Fundamentals (Required): The role requires 5+ years of experience in managing the end-to-end lifecycle (concept to launch) of complex technical products/features. The candidate should have the ability to lead customer research, specifically with experienced litigators, to understand workflows and translate needs into product requirements, especially for evaluation.
- AI & Technical Proficiency (Required): A minimum of 1 year working with generative AI is required, along with fluency in current tools/techniques (Legal AI focus highly preferred). The candidate should have experience developing/implementing auto-evaluation frameworks for GenAI models and a strong grasp of relevant metrics/techniques for NLP. High-level familiarity with Large Language Model architecture and operation, understanding of data management and quality best practices for AI, and experience working directly with data science teams on model processes (training, tuning, evaluation) are essential. Comfort with intermediate programming, specifically Python, is preferred.
- Analytical & Strategic Problem-Solving: The candidate should possess excellent critical thinking, strategic analysis, and creative problem-solving abilities. They should have the ability to clarify unclear problems, identify core, transferable principles, proactively anticipate risks, model scenarios, and provide actionable guidance. A creative approach to leveraging technology and assessing new tech risks/benefits beyond standard procedures is required.
- Communication, Collaboration & Leadership: Exceptional communication, collaboration, presentation, and influencing abilities across diverse teams are necessary. Demonstrable leadership potential/experience is required.
- Mindset & Work Ethic: The candidate should be a bright, tech-savvy self-starter with strong ownership and results orientation. They should have the ability to think strategically about organizational needs and an interest in team building. Staying informed on emerging research and industry trends is essential.
Salary Information:
- The base compensation range varies across locations. Eligible office location(s) for this role include one or more of the following: New York City, San Francisco, Los Angeles, and/or Irvine, CA; McLean, VA; Washington, DC. The base compensation range for the role in any of those locations is $134,680 - $250,120.
- For any eligible US locations, unless otherwise noted, the base compensation range for this role is $116,900 - $217,100.
- This role may also be eligible for an Annual Bonus based on a combination of enterprise and individual performance.
- Base pay is positioned within the range based on several factors including an individual’s knowledge, skills, and experience with consideration given to internal equity. Base pay is one part of a comprehensive Total Reward program which also includes flexible and supportive benefits and other wellbeing programs.
Overall, the Senior Product Manager role is critical in transforming legal research and litigation workflows through the strategic implementation of generative AI, requiring a blend of legal expertise, product management skills, and technical proficiency. The position offers a competitive salary and benefits package, along with opportunities for career development in a forward-thinking company dedicated to innovation and social impact.
