Note: The job is a remote job and is open to candidates in USA. Swoon is seeking an Artifacts SME to support the Frontier Evals team in a high-priority evaluation campaign. The role focuses on monitoring data collection, conducting quality control, supporting contributors and reviewers, tracking campaign health, and using spreadsheets, SQL, or Python for light technical work.
Responsibilities
- Review incoming tasks, deliverables, and rubrics to confirm they meet campaign standards
- Identify common failure modes
- Provide actionable feedback to contributors and reviewers and track campaign progress
- Monitor campaign health and throughput, including submission volume, rework rates, bottlenecks, turnaround time, and recurring quality issues
- Surface systematic issues to Frontier Evals leads and propose solutions
- Maintain lightweight QC logs, issue trackers, or audit trails to support retrospectives, analysis, and process improvement
- Answer contributor questions to improve quality
- Coordinate with reviewers and subject matter experts on edge cases and escalate ambiguous decisions
- Help calibrate contributors on what “good” looks like for tasks and rubrics in the target domain
- Use Codex, spreadsheets, light SQL, or simple Python scripts
- Assist with basic data hygiene and organization of uploaded materials, naming conventions, and record completeness
- Draft or revise campaign documentation, examples, QC checklists, and “good vs. bad” examples for the target domain
- Participate in targeted QA sweeps
Skills
- Fluency in one of Consulting, Finance, Data Science, Software Engineering, or Legal
- A passion for professional artifacts quality (docs, spreadsheets, decks, and clear and compelling writing) is a must
- 3+ years of professional experience in the target domain
- Passion for docs, spreadsheets, decks, and clear and compelling writing
- Strong judgment about what realistic, high-quality work looks like in that field
- Excellent attention to detail and comfort applying standards and rubrics consistently
- Able to write clear, direct feedback that improves quality quickly
- Comfortable working in spreadsheets
- Able to work in ambiguous, fast-moving environments and make pragmatic judgment calls
- Experience in multiple domains is a plus!
- Familiarity with basic SQL or Python
Benefits
- Fully remote
Company Overview
Company H1B Sponsorship