```html
About skillspath – Pioneering the Future of Financial Technology
skillspath is a global leader in innovative financial services, delivering cutting‑edge solutions that empower millions of customers worldwide. Our mission is to blend technology and finance to create seamless, secure, and intelligent experiences. As a forward‑thinking organization, skillspath invests heavily in emerging technologies, open‑source collaboration, and continuous learning, ensuring that every team member has the tools and support needed to thrive in a rapidly evolving industry.
Why This Role Matters
In today’s data‑driven world, the ability to process, analyze, and act on massive volumes of information is a strategic differentiator. As a Senior Big Data Engineer at skillspath, you will be at the heart of this transformation, designing and delivering robust data pipelines that power critical business decisions, enhance customer experiences, and drive operational efficiency. This is a remote, full‑time position offering a competitive rate of $28 per hour, with the flexibility to work from anywhere in the United States.
Key Responsibilities
- Architect, develop, and maintain scalable data pipelines using Hadoop, Spark, Hive, and PySpark to ingest, process, and store petabyte‑scale datasets.
- Collaborate closely with product owners, data scientists, and analytics teams to translate business requirements into technical specifications.
- Lead code reviews, enforce best practices, and mentor junior engineers on clean coding, testing, and documentation standards.
- Design and implement automated testing frameworks, continuous integration/continuous deployment (CI/CD) pipelines, and monitoring solutions to ensure high reliability and rapid delivery.
- Identify performance bottlenecks, troubleshoot complex data processing issues, and proactively propose optimizations.
- Drive the migration of legacy batch processes to real‑time streaming architectures using Kafka, Flink, or similar technologies.
- Contribute to open‑source initiatives and internal knowledge‑sharing forums, reinforcing skillspath’s commitment to community engagement.
- Participate in agile ceremonies, including sprint planning, daily stand‑ups, and retrospectives, to maintain transparent communication and iterative progress.
- Document architectural decisions, data models, and operational procedures to support cross‑functional collaboration and future scalability.
Essential Qualifications
- Bachelor’s degree in Computer Science, Software Engineering, or a related discipline (or equivalent professional experience).
- Minimum of 3 years of hands‑on experience in big data engineering, with a proven track record of delivering production‑grade solutions.
- Deep expertise in Hadoop ecosystem components (HDFS, YARN, MapReduce) and Spark (core, SQL, Structured Streaming).
- Strong command of SQL, including complex query optimization, Hive, and PySpark DataFrames.
- Proficiency in programming languages such as Java, Python, or Scala, with a focus on clean, maintainable code.
- Solid understanding of data warehousing concepts, dimensional modeling, and ETL best practices.
- Experience working in Unix/Linux environments and scripting with Bash or similar shells.
- Demonstrated ability to design and implement end‑to‑end data pipelines that handle both batch and streaming workloads.
- Excellent problem‑solving skills, with the capacity to diagnose and resolve intricate data processing challenges.
Preferred Qualifications & Nice‑to‑Have Skills
- Hands‑on experience with cloud platforms such as Google Cloud Platform (GCP) or Amazon Web Services (AWS), especially services like BigQuery, Dataflow, or Redshift.
- Familiarity with microservices architecture and containerization technologies (Docker, Kubernetes).
- Experience building and managing Kafka streams or similar event‑driven pipelines.
- Knowledge of NoSQL databases (HBase, Couchbase, MongoDB) and their integration within big data workflows.
- Exposure to CI/CD tools such as GitHub Actions, Bitbucket Pipelines, Jenkins, or Azure DevOps.
- Background in financial services data environments, including an understanding of regulatory compliance and data security standards.
- Certifications related to big data, cloud platforms, or data engineering (e.g., Cloudera Certified Professional, AWS Certified Data Analytics).
Core Skills & Competencies
- Technical Acumen: Mastery of distributed computing, data modeling, and performance tuning.
- Leadership: Ability to guide a small team of engineers, foster collaboration, and inspire innovative thinking.
- Communication: Clear articulation of complex technical concepts to both technical and non‑technical stakeholders.
- Adaptability: Comfort working in a fast‑paced environment where priorities shift based on business needs.
- Continuous Learning: Commitment to staying current with emerging technologies, industry trends, and best practices.
- Analytical Mindset: Strong data‑driven decision‑making, with an eye for detail and quality.
Career Growth & Learning Opportunities
skillspath invests heavily in the professional development of its employees. As a Senior Big Data Engineer, you will have access to:
- Mentorship programs pairing you with senior architects and industry experts.
- Funding for certifications, conferences, and advanced coursework in data engineering, cloud technologies, and leadership.
- Opportunities to lead high‑visibility projects that directly impact skillspath’s product roadmap and market positioning.
- Cross‑functional rotations that broaden your skill set across data science, product management, and security.
- A clear promotion pathway from Senior Engineer to Lead Engineer, Architecture Manager, and eventually Director of Data Engineering.
Work Environment & Culture at skillspath
Our remote‑first culture is built on trust, autonomy, and collaboration. We provide:
- A flexible work schedule that respects work‑life balance while delivering results.
- Regular virtual team‑building events, hackathons, and knowledge‑sharing sessions.
- An inclusive environment where diverse perspectives are celebrated and every voice is heard.
- State‑of‑the‑art collaboration tools (Slack, Microsoft Teams, Confluence) to keep teams connected across time zones.
- Access to a home‑office stipend for ergonomic equipment, high‑speed internet, and productivity accessories.
Compensation, Perks & Benefits
skillspath offers a competitive hourly rate of $28, complemented by a comprehensive benefits package that includes:
- Health, dental, and vision insurance with multiple plan options.
- Retirement savings plan with company matching contributions.
- Generous paid time off, holidays, and sick leave.
- Professional development budget for courses, certifications, and conferences.
- Employee assistance program and mental‑health resources.
- Performance‑based bonuses and recognition programs.
- Access to a global network of experts and a vibrant community of innovators.
How to Apply
If you are passionate about turning massive data sets into actionable insights and thrive in a collaborative, remote environment, we want to hear from you. Click the link below to submit your application, including a resume and a brief cover letter highlighting your most relevant experience.
Apply Now – Join skillspath’s Data Engineering Team!
Closing Statement
skillspath is more than a financial services company; we are a technology‑driven ecosystem where data engineers shape the future of digital finance. By joining our team, you will work alongside some of the brightest minds in the industry, tackle challenging problems at scale, and accelerate your career in a supportive, forward‑looking environment. Take the next step in your professional journey—apply today and become a pivotal part of TeamSkillspath!