Other Attorney in New York City, NY

In-House

New York City, NY

Other Attorney in New York City, NY

Attorney

3-5 yrs required

No

Job Title: Product Manager

Job Responsibilities:

- Strong Legal Research: The Product Manager is responsible for breaking down complex legal research and litigation workflows to identify opportunities for generative AI to enhance tasks such as legal research, brief analysis, and motion improvement. This involves defining evaluation criteria to ensure AI outputs adhere to high legal standards.

- Evaluation Framework Execution: The role involves contributing to the design and managing the implementation of automated evaluation frameworks and metrics for generative AI in the legal domain. This includes assessing factors such as factual accuracy, legal soundness, citation validity, hallucination detection, bias, and relevance.

- Data Management and Testing Coordination: The Product Manager oversees the creation and curation of high-quality datasets necessary for auto-evaluation, coordinates automated testing and validation pipelines, tracks results, and identifies areas for improvement.

- Cross-Functional Collaboration: Collaboration with research scientists, engineers, attorney editors, and other product managers is essential to translate requirements into user stories and specifications, facilitate development, and ensure the smooth execution of generative AI evaluation features.

- Technical Insight & Application: Staying current with advancements in LLMs, generative AI evaluation techniques, and prompt engineering is crucial. The Product Manager applies this knowledge to support continuous improvement of evaluation methods and tools.

- Stakeholder Communication: The role involves clear communication of feature status, requirements, evaluation results, and potential risks to relevant team members and stakeholders.

- User & Market Understanding: The Product Manager participates in and supports market research, competitive analysis, and user research activities to gain insights into the needs and expectations of legal professionals regarding AI-generated content evaluation.

Education and Experience Information:

- Legal Expertise: A . from an accredited law school is required, along with a deep understanding of legal research methods, concepts, and their application within litigation workflows. The ability to dissect complex legal issues is essential.

- Product Management Fundamentals: The candidate should have 3-5+ years of product management experience, with a track record of contributing to the end-to-end lifecycle of complex technical products/features. Experience in translating user needs into product requirements, especially for evaluation, is important.

- AI & Technical Proficiency: A minimum of 6 months working with generative AI is required. Familiarity with and a strong interest in learning about current AI tools and techniques, particularly with a legal AI focus, is highly preferred. Understanding the importance of evaluation metrics and data quality for AI is crucial. Direct experience contributing to the development or implementation of evaluation processes for AI/ML models, experience working directly with data science teams, and comfort with intermediate programming, specifically Python, is a plus.

- Analytical & Problem-Solving Skills: Strong critical thinking and analytical skills are required. The candidate should have the ability to break down complex problems, identify core issues, and contribute to effective solutions. Comfort with ambiguity is necessary.

- Communication & Collaboration: Excellent communication and collaboration skills are essential. The candidate should be able to work effectively within cross-functional teams and articulate requirements.

- Mindset & Work Ethic: The role requires a bright, tech-savvy, proactive self-starter with a strong sense of ownership and a drive to deliver results. The candidate should be eager to learn and stay informed about emerging research and industry trends.

Salary Information:

- The base compensation range for this role varies by location. For eligible office locations such as New York City, San Francisco, Los Angeles, Irvine, CA, McLean, VA, and Washington, DC, the range is $101,640 - $188,760. For other eligible US locations, unless otherwise noted, the range is $88,200 - $163,800.

- This role may also be eligible for an Annual Bonus based on a combination of enterprise and individual performance.

- Base pay is positioned within the range based on factors such as an individual’s knowledge, skills, and experience, with consideration given to internal equity. Base pay is part of a comprehensive Total Reward program, which includes flexible and supportive benefits and other wellbeing programs.