Note: The job is a remote job and is open to candidates in USA. Sapphire Software Solutions Inc is recruiting a Senior Azure Data Engineer for an unnamed client. The role supports GenAI solutions for enterprise products by building and maintaining large-scale analytics and data pipelines, including ingestion, streaming, batch processing, ETL, and SQL performance tuning.
Responsibilities
- Responsible for the building, deployment, and maintenance of mission-critical analytics solutions that process terabytes of data quickly at big-data scales
- Contributes design, code, configurations, manage data ingestion, real-time streaming, batch processing, ETL across multiple data storages
- Responsible for performance tuning of complicated SQL queries and Data flows
Skills
- At least 9+ years of experience required
- The candidate must have excellent communication skills
- Experience coding in SQL/Python(\•), with solid CS fundamentals including data structure and algorithm design
- Hands-on implementation experience working with a combination of the following technologies: Hadoop, Map Reduce, Kafka, Hive, Spark(\•), SQL and NoSQL data warehouses (\•)
- Experience in Azure cloud data platform(\•)
- Must have: understanding of LLM Pipelines, including data preprocessing for GenAI Models
- Experience coding in SQL/Python(\•), with solid CS fundamentals including data structure and algorithm design
- Hands-on implementation experience working with a combination of the following technologies: Hadoop, Map Reduce, Kafka, Hive, Spark(\•), SQL and NoSQL data warehouses (\•)
- Experience in Azure cloud data platform(\•)
- Experience working with vector databases (Milvus, Postgres, etc.) (\•)
- Knowledge of embedding models and retrieval-augmented generation (RAG) architectures
- Understanding of LLM pipelines, including data preprocessing for GenAI models (\•)
- Experience deploying data pipelines for AI/ML workloads(\•), ensuring scalability and efficiency
- Familiarity with model monitoring(\•), feature stores (Feast, Vertex AI Feature Store), and data versioning(\•)
- Experience with CI/CD for ML pipelines(\•) (Kubeflow, MLflow, Airflow, SageMaker Pipelines)
- Understanding of real-time streaming for ML model inference (Kafka, Spark Streaming) (\•)
- Knowledge of Data Warehousing, design, implementation and optimization
- Knowledge of Data Quality testing, automation and results visualization
- Knowledge of BI reports and dashboards design and implementation (PowerBI)(\•)
- Experience with supporting data scientists and complex statistical use cases
Benefits
- 100% remote, with the candidate required to sit in the EST/CST time zone
- Opportunity to build a GenAI solution embedded within the platform and work on the latest technology for an industry tax solution leader
Company Overview
Company H1B Sponsorship