Note: The job is a remote job and is open to candidates in USA. Weekday AI is seeking experienced legal professionals to author and review high-quality academic assessment content for an AI research initiative. The Applied Legal Benchmark Specialist will create or verify challenging legal questions, assess their difficulty, provide rigorous explanations and references, and help develop reliable benchmarks for evaluating AI systems' legal understanding and reasoning.
Responsibilities
- Develop original legal questions that assess deep conceptual understanding, analytical ability, and legal reasoning rather than surface-level recall
- Ensure every question is clear, unambiguous, self-contained, and sufficiently detailed to support a definitive answer
- Classify questions by difficulty level: Medium (introductory undergraduate), Hard (advanced undergraduate), or Expert (postgraduate level and above)
- Provide one correct answer along with nine plausible but subtly incorrect alternatives designed to challenge advanced legal reasoning
- Develop concise, logically structured solution explanations that demonstrate the appropriate legal reasoning process
- Provide 1-5 credible academic or professional references for each question, drawing from reputable legal publications, academic sources, case law, statutes, and legal repositories
- For verification assignments, identify issues involving legal accuracy, clarity, completeness, precision, or solvability and clearly explain any recommended revisions
- Apply consistent quality standards across questions and contribute to the development of reliable, gold-standard legal evaluation benchmarks
Skills
- Commitment: 10+ hours per week
- Work Arrangement: Fully remote
- Work Style: Asynchronous and flexible
- H-1B and STEM OPT candidates are not eligible for this opportunity at this time
- JD, LLM, SJD, or doctoral-level qualification in Law or a closely related discipline
- Strong command of legal reasoning, statutory and regulatory interpretation, case analysis, and legal research
- Deep expertise in at least one of the covered practice areas
- Excellent written English with the ability to communicate complex legal concepts clearly, precisely, and concisely
- Strong attention to detail and the ability to identify subtle legal inaccuracies or ambiguities
- Master's-level candidates may be considered where they demonstrate exceptional expertise in a specialized legal domain
- Experience with legal education, bar examination content development, academic assessment, judicial clerkships, legal research, or legal writing is highly valued
- Prior experience with AI evaluation, training data, benchmarking, or human-in-the-loop research is a plus
Benefits
- Fully remote
- Asynchronous and flexible work style
- Flexible schedule
- Payments are made weekly based on services rendered through available payment methods
- Project-based opportunities with potential for continued participation based on performance and project requirements
Company Overview