Note: The job is a remote job and is open to candidates in USA. SoTalent seeks a Senior Software Engineer – Data Platform to lead the evolution of a shared enterprise data platform supporting multiple products and business solutions. The role combines hands-on data platform engineering, architecture, technical leadership, and cross-functional collaboration to deliver reliable data products and customer-facing insights.
Responsibilities
- Design, build, and maintain scalable data platforms that support multiple products and business functions
- Own the end-to-end delivery of data capabilities, from ingestion and transformation to storage and consumption
- Develop data solutions that align technical execution with product and business objectives
- Drive platform enhancements focused on data coverage, accuracy, freshness, and accessibility
- Ensure data infrastructure supports both current business requirements and future growth
- Architect and optimize large-scale data ingestion pipelines processing high-volume structured and unstructured data
- Build and maintain data collection frameworks utilizing APIs, structured feeds, data transformations, and web-based data acquisition techniques
- Develop robust ETL/ELT workflows that ensure reliable and efficient movement of data across systems
- Implement scalable processing solutions capable of supporting millions of records and complex business use cases
- Continuously improve data quality, throughput, and operational efficiency
- Define and enforce standards for data accuracy, completeness, consistency, and timeliness
- Establish governance practices that ensure datasets align with business and product requirements
- Develop automated validation, monitoring, and quality assurance processes
- Create frameworks for identifying, tracking, and resolving data issues
- Drive continuous improvement of platform reliability and data integrity
- Design and maintain cloud-native data solutions using modern cloud services and distributed architectures
- Optimize cloud-based storage, compute, search, and processing capabilities
- Support scalable and cost-effective infrastructure strategies
- Collaborate with engineering teams to ensure system performance, resilience, and security
- Leverage modern cloud technologies to support long-term platform growth
- Build monitoring, alerting, and observability capabilities that provide visibility into data platform health
- Develop automated processes for anomaly detection, issue identification, and operational reporting
- Ensure platform reliability through proactive monitoring and incident response practices
- Support performance optimization initiatives across ingestion, processing, and delivery systems
- Implement operational best practices that minimize downtime and improve service reliability
- Partner closely with Product Management, Data Teams, and Application Engineering teams to deliver impactful customer-facing features
- Translate business requirements into scalable technical solutions
- Help identify new opportunities to leverage data for product innovation and enhanced customer value
- Communicate technical concepts and project status effectively to stakeholders at all levels
- Align platform investments with strategic product objectives
- Provide technical leadership on architecture decisions, engineering standards, and platform evolution
- Mentor engineers and share best practices in data engineering, software development, and system design
- Promote a culture of ownership, innovation, and continuous improvement
- Lead complex technical initiatives from concept through deployment and operational support
- Drive adoption of emerging technologies and engineering methodologies
- Maintain comprehensive documentation for architecture, processes, pipelines, and operational procedures
- Establish best practices and standards that improve team effectiveness
- Facilitate knowledge sharing and technical training across engineering teams
- Create reusable documentation that supports long-term platform sustainability
Skills
- Bachelor's degree in Computer Science, Software Engineering, Information Systems, Data Engineering, or a related technical field
- 5+ years of software engineering or data engineering experience
- Proven experience building data platforms that support customer-facing products or business-critical applications
- Strong experience designing and operating large-scale data pipelines and processing systems
- Experience working in cloud-native environments and distributed systems
- Strong proficiency in Python and Ruby on Rails
- Expertise in large-scale data ingestion, transformation, and integration
- Experience building ETL/ELT workflows using tools such as Airflow , dbt , or similar platforms
- Strong understanding of structured and unstructured data processing
- Experience with web scraping, data parsing, feed processing, and API integrations
- Deep knowledge of AWS services including cloud storage, compute, containerized workloads, serverless solutions, and search technologies
- Experience designing and maintaining data warehouses and analytical data systems
- Familiarity with observability, monitoring, logging, alerting, and distributed tracing tools
- Knowledge of modern software development practices, version control, and CI/CD pipelines
- Experience supporting large-scale SaaS or data-driven products
- Background in analytics, business intelligence, or customer-facing data applications
- Experience utilizing AI-assisted development and data engineering tools
- Knowledge of real estate, multifamily housing, market intelligence, or related domains
- Experience working in growth-stage technology environments
Benefits
- Remote work (US)
Company Overview