Note: The job is a remote job and is open to candidates in USA. Greystar is a global real estate platform offering property management, investment management, development, and construction services. The DataOps Engineer will own the reliability, scalability, observability, and deployment lifecycle of the Databricks-native enterprise data platform on Microsoft Azure, while developing AI-driven monitoring and self-healing pipeline capabilities. The role also covers Azure infrastructure integration, data quality and governance, CI/CD, platform optimization, documentation, and on-call support.
Responsibilities
- Implement AI-powered observability — using LLMs and ML models to detect pipeline drift, classify anomalies, predict SLA risk, and generate automated incident summaries
- Build agentic monitoring workflows that proactively surface data quality degradation, pipeline dropout, schema drift, and volume anomalies across all DMP layers
- Integrate AI tooling (Databricks Mosaic AI, Genie, OpenAI APIs, or equivalent) into operational DataOps processes — not as experiments, but as production-grade capabilities
- Develop and maintain AI-assisted root cause analysis tooling to reduce MTTR on pipeline failures, with structured learnings fed back into the platform
- Contribute to Greystar’s 18-month agentic AI roadmap, leading near-term delivery of self-healing pipeline capabilities
- Operate the full Azure data services stack supporting DMP: ADLS Gen2, Azure Data Factory (ADF), Azure Monitor, Log Analytics, Key Vault, and Event Hub
- Design and maintain ADF pipelines for source system ingestion, including orchestration patterns for multi-tenant ERP environments (Yardi, Entrata, RealPage)
- Collaborate with Azure infrastructure and cloud engineering teams on networking, identity, security, and resource provisioning
- Drive cost governance through Azure Cost Management, Databricks DBU optimization, and storage lifecycle policies
- Own the design, build, and optimization of data pipelines on Databricks using Delta Live Tables (DLT), PySpark, Workflows, and Jobs across the full DMP medallion stack
- Administer and govern the Databricks workspace: Unity Catalog, cluster policies, access controls, compute configurations, and Delta table lifecycle management
- Tune Spark jobs for performance, reliability, and cost — profiling bottlenecks, optimizing partitioning, managing Z-ordering, and controlling compute spend
- Leverage Databricks Mosaic AI and Genie to build AI-native DataOps capabilities including intelligent pipeline monitoring, anomaly detection, and natural language data access
- Architect and enforce DMP platform standards: naming conventions, schema evolution policies, SLA tiers, and medallion layer contracts
- Own the full deployment pipeline for DMP data workflows — promoting changes from development through staging to production with rigor and minimal disruption
- Build and maintain CI/CD workflows using GitHub Enterprise, including branch strategies, pull request automation, environment-specific configuration management, and release gating
- Use Linear for sprint planning, release tracking, and issue management across deployment cycles; coordinate engineering work items with cross-functional stakeholders
- Enforce deployment standards: automated testing gates, rollback procedures, change documentation, and environment parity controls
- Partner with the analytics engineering and integration teams to align deployment cadences across the DMP stack
- Instrument DQ checks across Bronze, Silver, and Gold layers covering completeness, consistency, accuracy, uniqueness, and referential integrity
- Partner with Brett Finley’s Data Governance team to enforce data contracts, ownership standards, and quality SLAs within Unity Catalog
- Build feedback loops between DQ scoring, pipeline observability, and upstream source owners to drive systemic data reliability improvements
- Partner with analytics engineers, data governance, and product stakeholders to align pipeline and platform design with business requirements
- Produce thorough technical documentation — runbooks, deployment playbooks, incident post-mortems, ADRs, and platform specs
- Participate in on-call rotation and support SLA commitments for business-critical DMP data domains
Skills
- 7+ years of DataOps, data engineering, or platform engineering experience in a production environment
- Expert-level hands-on experience with Databricks: Delta Live Tables, Jobs/Workflows, Unity Catalog, Spark performance tuning, and Delta Lake internals
- Strong command of the Azure data services ecosystem: ADF, ADLS Gen2, Azure Monitor, Log Analytics, Key Vault, and related services
- Demonstrated, production use of AI tools in DataOps or data observability workflows — LLM-assisted diagnostics, intelligent alerting, agentic monitoring, or equivalent
- Proven CI/CD experience using GitHub Enterprise — branch strategies, PR automation, environment promotion, and release management for data pipelines
- Solid Python and/or Scala skills for pipeline development; SQL fluency for Gold layer transformation and DQ validation
- Hands-on experience with ADF pipeline design and orchestration at scale
- Experience with medallion / lakehouse architecture patterns and multi-environment deployment discipline
- Strong collaborative skills across engineering, governance, and business stakeholder teams
- Experience with Linear for engineering sprint management and release tracking
- Familiarity with Databricks Mosaic AI, Genie, or other AI-native Databricks capabilities
- Exposure to agentic AI frameworks or MCP (Model Context Protocol) server integrations
- Background in real estate, property management, or multi-source ERP data environments (Yardi, Entrata, RealPage)
- Experience with Cosmos DB, Azure SQL, or similar operational data stores alongside lakehouse platforms
- Knowledge of data governance frameworks, data lineage tooling, and metadata management within Unity Catalog
- Background in legacy BI migration or platform modernization programs
Benefits
- A high-impact role at the center of Greystar’s enterprise data transformation
- Collaborative, engineering-driven team culture with a strong focus on craft, automation, and continuous improvement
- Access to cutting-edge tooling — Databricks, full Azure stack, GitHub Enterprise, and an active AI innovation agenda
- Flexible work arrangements
- Opportunity to define the DataOps discipline and lead Greystar’s self-healing pipeline and agentic AI roadmap
- In addition to the base salary, this role may be eligible to participate in a quarterly or annual bonus program based on individual and company performance.
- Competitive Medical, Dental, Vision, and Disability & Life insurance benefits. Low (free basic) employee Medical costs for employee-only coverage; costs discounted after 3 and 5 years of service.
- Generous Paid Time off. All new hires start with 15 days of vacation, 4 personal days, 10 sick days, and 11 paid holidays. Plus your birthday off after 1 year of service! Additional vacation accrued with tenure.
- For onsite team members, onsite housing discount at Greystar-managed communities are available subject to discount and unit availability.
- 6-Week Paid Sabbatical after 10 years of service (and every 5 years thereafter).
- 401(k) with Company Match up to 6% of pay after 6 months of service.
- Paid Parental Leave and lifetime Fertility Benefit reimbursement up to $10,000 (includes adoption or surrogacy).
- Employee Assistance Program.
- Critical Illness, Accident, Hospital Indemnity, Pet Insurance and Legal Plans.
- Charitable giving program and benefits.
- Benefits offered for full-time employees.
- This position may be performed remotely anywhere within the United States except the state of Alaska.
Company Overview
Company H1B Sponsorship