Note: The job is a remote job and is open to candidates in USA. Peak Performance Human Resources Corp. is conducting a confidential search for a national not-for-profit healthcare certification and assessment body. The Interim Vice-President, Psychometrics and Data Analytics will lead psychometric quality, enterprise data strategy, analytics, business intelligence, data governance and AI/ML model governance while advising executives and the Board on high-stakes assessment decisions. The role also oversees the department's people, priorities, projects and budget.
Responsibilities
- Oversee the psychometric quality of all assessments, validity, reliability, fairness and defensibility, through test design and administration support, standard setting, equating, scaling, scoring and results reporting
- Ensure consistent application of psychometric methodologies across assessments, in line with professional standards and best practices
- Direct statistical and forensic analyses of examination outcomes, candidate performance and testing incidents (including remote proctoring) to support fair, evidence-based decisions
- Lead the initial development and implementation of an enterprise data governance framework, data quality, integrity, security, privacy, retention, definitions and stewardship, alongside metadata management, a data catalogue, and master and reference data management
- Govern analytical and AI/machine-learning models used in or affecting assessments, including model validation, bias and fairness testing, performance monitoring, drift detection and documentation
- Own the reporting, dashboard and data-visualization environment, including self-serve business intelligence for senior leadership, the Board and stakeholders, underpinned by a governed single source of truth and semantic layer
- Champion a data strategy that treats assessment and operational data as a strategic asset, and advance analytics from descriptive and diagnostic reporting toward predictive and prescriptive capability
- Steward the multi-year data strategy and analytics roadmap aligned to the strategic plan, in collaboration with technology leadership, which owns the underlying data platform and engineering
- Chair the data governance operating model, with named data owners and stewards in each function, reporting through the Senior Management Team to the CEO
- Advise on psychometric and standard-setting decisions with high-stakes or policy-level impact, and on statistical and forensic findings related to testing irregularities
- Partner closely with executive peers in Assessments, Technology and Operations, and present psychometric results, validity evidence and recommendations to the Board's examinations committee
- Lead all HR matters for the Psychometrics and Data Analytics department, including recruitment, training, retention and performance management, along with annual business priorities, projects and budget
Skills
- PhD required, ideally in psychometrics or educational measurement, or in a closely related quantitative discipline such as statistics
- Significant progressive senior leadership and management experience, including leading technical teams or functions
- Substantial experience as a psychometrician in educational or psychological testing (typically eight or more years), including high-stakes assessment
- Strong expertise in psychometric methodologies, including standard setting, equating and scaling
- Experience leading or overseeing enterprise data strategy, data governance, data analytics and business intelligence/reporting
- Proven ability to design, conduct, interpret and present psychometric analyses and studies in high-stakes assessment contexts
- Strong data analysis and scientific-computing skills, and familiarity with business-intelligence and data-visualization tools
- Experience leveraging data science, machine learning or emerging AI tools to support measurement and analytics
- You are a strong, seasoned leader with a track record of setting direction, someone who has redefined strategy for a technical function, not just managed one
- You bring high professional standards and a strong commitment to assessment quality, fairness and defensibility, with sound analytical judgment on complex, high-impact psychometric and data issues
- You balance strategic leadership with technical rigour, hands-on where needed and treat data integrity, security and privacy as paramount
- You are a credible, confident and collaborative communicator, able to translate complex psychometric and data concepts for non-technical audiences, inspire and develop a technical team, and operate independently in a high-stakes assessment environment
- This is a fully remote position open to candidates located in Canada or the United States
- Occasional travel may be required
- Assets: exposure to predictive/prescriptive analytics and AI/ML model governance; professional certification and licensure experience; proficiency in French
Benefits
- Fully remote position open to candidates located in Canada or the United States
- Occasional travel may be required
Company Overview