```html
About talentflow – Pioneering Entertainment‑Driven Data Innovation
talentflow is a global leader in creating immersive digital experiences that blend entertainment, sports, and cutting‑edge technology. Our mission is to transform raw data into actionable insights that power unforgettable experiences for millions of fans worldwide. With a portfolio that spans streaming platforms, interactive gaming, and next‑generation media services, talentflow invests heavily in data‑driven product development, predictive analytics, and real‑time personalization. As we continue to expand our footprint in the cloud‑first era, we are looking for visionary engineers who thrive on solving complex data challenges at scale.
Position Overview – Lead Data Engineer
We are seeking a highly motivated Lead Data Engineer to join our Information and Investigation group within the talentflow Web‑Based Features division (formerly known as DSS). This role sits at the intersection of data engineering, analytics, and product innovation, driving the design, development, and operational excellence of large‑scale data platforms that support a variety of consumer‑facing applications. The successful candidate will own end‑to‑end data pipelines, mentor a growing team of engineers, and collaborate closely with product owners, data scientists, and business stakeholders to deliver reliable, high‑performance data solutions.
Key Responsibilities
- Architect & Build Scalable Data Solutions: Design and implement robust data warehousing architectures, including data lakes, marts, and real‑time streaming pipelines, to support both batch and low‑latency analytics workloads.
- Develop ETL/ELT Pipelines: Create, optimize, and maintain complex ETL processes using industry‑standard tools and frameworks such as Hadoop, Spark, AWS Glue, and Snowflake.
- Leverage Cloud Platforms: Deploy and manage data services on Amazon Web Services (S3, EMR, EC2, Redshift) and integrate with on‑premise environments for hybrid solutions.
- Collaborate with Cross‑Functional Teams: Partner with Data Product Managers, Data Modelers, and Data Architects to translate business requirements into technical specifications and deliver end‑to‑end data products.
- Ensure Data Quality & Governance: Implement data validation, lineage tracking, and documentation standards to uphold data integrity, compliance, and auditability.
- Performance Tuning & Optimization: Conduct regular profiling, indexing, and query optimization to meet stringent SLAs for latency, throughput, and cost efficiency.
- Mentorship & Leadership: Guide junior engineers, conduct code reviews, and foster a culture of continuous learning and agile best practices.
- Agile Delivery: Actively participate in Scrum ceremonies, sprint planning, and retrospectives to ensure rapid, iterative delivery of high‑value features.
- Documentation & Knowledge Sharing: Maintain comprehensive technical documentation, runbooks, and knowledge‑base articles to facilitate team onboarding and cross‑team collaboration.
Essential Qualifications
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related discipline (or equivalent practical experience).
- 5+ years of hands‑on experience designing and operating large‑scale data platforms in cloud or hybrid environments.
- Deep expertise in SQL and the ability to craft performant queries across distributed data stores.
- Proficiency with big‑data technologies such as Hadoop ecosystem (HDFS, Hive, Impala, PySpark) and streaming frameworks like Apache Flink or Kafka.
- Strong programming skills in Python and either Java or Scala, with a track record of building production‑grade data pipelines.
- Experience with at least one major MPP or cloud data‑warehouse solution (Snowflake, Redshift, BigQuery, Azure Synapse).
- Hands‑on knowledge of data modeling, data warehousing best practices, and dimensional design.
- Familiarity with data orchestration tools (Airflow, Luigi, Prefect) and CI/CD pipelines for data engineering.
- Demonstrated ability to work within Scrum/Agile frameworks and deliver incremental value in fast‑paced environments.
Preferred Qualifications & Additional Skills
- Master’s degree or advanced certifications in data engineering, cloud architecture, or related fields.
- Experience with containerization (Docker, Kubernetes) for scalable data workloads.
- Exposure to machine‑learning pipelines and collaboration with data science teams.
- Knowledge of data security standards (GDPR, CCPA) and implementation of encryption, masking, and access controls.
- Proven track record of leading technical initiatives that resulted in measurable business impact.
- Excellent problem‑solving mindset, strong communication skills, and the ability to translate technical concepts for non‑technical stakeholders.
Core Skills & Competencies
- Analytical Thinking: Ability to dissect complex data problems and devise elegant, scalable solutions.
- Collaboration: Strong team player who can build relationships across product, analytics, and engineering domains.
- Leadership: Comfortable taking ownership of critical data initiatives and mentoring junior talent.
- Adaptability: Thrive in a dynamic environment where priorities shift rapidly and new technologies emerge.
- Attention to Detail: Commitment to high‑quality code, thorough testing, and meticulous documentation.
Career Growth & Learning Opportunities
At talentflow, your career trajectory is shaped by the impact you create. As a Lead Data Engineer, you will have direct exposure to senior leadership, influence product roadmaps, and drive strategic data initiatives that affect millions of users. We invest heavily in continuous learning—offering tuition reimbursement, access to industry conferences, internal tech talks, and a robust mentorship program. Over time, you can evolve into roles such as Data Architecture Director, Head of Analytics, or Technical Fellow, depending on your aspirations.
Compensation, Perks, & Benefits
We offer a competitive hourly rate of $24 per hour, commensurate with experience, along with a comprehensive benefits package that includes:
- Health, dental, and vision insurance with multiple plan options.
- 401(k) matching and financial wellness programs.
- Generous paid time off, holidays, and flexible work arrangements.
- Employee assistance programs, mental‑health resources, and wellness stipends.
- On‑site amenities such as catered meals, fitness centers, and relaxation zones.
- Opportunities for remote work and hybrid schedules to support work‑life balance.
Work Environment & Culture at talentflow
talentflow fosters an inclusive, collaborative culture where curiosity is celebrated and diverse perspectives drive innovation. Our San Francisco office is a vibrant hub featuring open workspaces, quiet focus rooms, and state‑of‑the‑art collaboration zones. We champion a growth mindset, encouraging employees to experiment, fail fast, and iterate. Diversity, equity, and inclusion are core to our values, and we actively support employee resource groups, mentorship circles, and community outreach initiatives.
Why Join talentflow?
If you are passionate about turning massive data streams into meaningful insights, love working with cutting‑edge technologies, and thrive in a fast‑moving entertainment‑tech environment, talentflow offers the perfect platform to accelerate your career. You will be part of a mission‑driven team that shapes the future of digital experiences for a global audience.
Ready to Make an Impact?
Take the next step in your data engineering journey and help talentflow redefine how entertainment meets data. Apply now to become a pivotal member of our innovative team.
``` Apply for this job