About the Company The company operates across SaaS, hospitality, media, and marketing, with a focus on understanding how businesses and their content appear in AI-generated answers. With a team of approximately 50 employees, the organization works at the intersection of data analytics, AI search, content performance, and product development.
About the Role The company is seeking a Senior Data Analyst to measure and analyze how AI search systems represent clients, which sources they rely on, and how AI-generated recommendations connect to business outcomes. Reporting to the Vice President of Product, you will develop measurement frameworks, maintain prompt-tracking panels, evaluate AI-generated answers, and investigate the factors that influence visibility across AI platforms.
This role combines analytics, research, experimentation, and hands-on work with large language models (LLMs) and agentic tools. You will work closely with product and engineering teams to define meaningful metrics, test hypotheses, and build reliable reporting methods. A key part of the position is ensuring that customer-facing findings can be traced back to documented inputs and that reported changes reflect meaningful patterns rather than statistical noise.
The position is remote within the United States, with working hours aligned to Eastern or Central Time. Occasional travel to New York City or Atlanta is expected.
Key Responsibilities
•Measure how frequently, accurately, and favorably clients appear in AI-generated answers compared with defined competitors, using clear baselines, sampling methods, and documented uncertainty.
•Maintain tracked-prompt panels across AI engines, topics, and markets, and report changes in visibility over time.
•Identify and analyze the sources cited by AI systems for specific topics.
•Track when published content begins appearing as cited evidence, how long it takes to surface, and how long it remains visible.
•Identify gaps in clients' evidence bases and determine what supporting content may be missing.
•Test how factors such as source, framing, format, specificity, recency, and structure influence the likelihood of content being cited by AI systems.
•Connect AI visibility metrics with referral traffic, engagement, bookings, and conversions, identifying what can be measured reliably and where attribution gaps remain.
•Evaluate AI-generated answers for accuracy, relevance, and completeness, while documenting recurring failure patterns across engines.
•Maintain evaluation rubrics, benchmarks, and processes for measuring agreement between human raters.
•Refine prompt construction and apply consistent evaluation methods to internally developed AI features.
•Research how AI search systems retrieve and cite sources, validating findings through research rather than relying solely on industry commentary.
•Work with product and engineering teams to define product metrics and ensure customer-facing figures can be reproduced from documented data and methods.
Required Qualifications
•At least 5 years of experience in analytics, data science, research, and/or search products.
•At least 2 years of experience in AI search, SEO, Generative Engine Optimization (GEO), Answer Engine Optimization (AEO), content performance, or LLM evaluation.
•Hands-on experience with LLMs and agentic tools, including prompt design, evaluation harnesses, and agents that collect and process data at scale.
•Strong SQL and Python skills.
•Experience designing experiments with observational data, including comparison group construction, difference-in-differences or matched comparisons, and appropriate assessment of statistical power.
•Strong analytical judgment when working with unstructured text, including developing evaluation rubrics, converting qualitative signals into structured data, and handling ambiguous cases carefully.
•Startup experience, including working in fast-moving, small teams with significant individual ownership and visibility.
Preferred Qualifications / Requirements
•Experience with Databricks or similar data platforms.
Salary & Benefits The compensation package includes an annual salary of $120,000–$140,000, plus equity and an annual bonus.
Additional benefits include:
•401(k) plan with a 4% employer match.
•Monthly wellness stipend.
•Comprehensive medical and dental insurance plans.
•Long-term disability coverage.
•Additional benefits.
Employment Details
•Job Title: Sr. Data Analyst
•Work Arrangement: Remote within the United States
•Time Zones: Eastern or Central Time
•Travel: Occasional trips to New York City or Atlanta
•Team Size: Approximately 50 employees
•Reports To: Vice President, Product