Note: The job is a remote job and is open to candidates in USA. adorsys is a software company developing digital solutions for the banking and insurance sectors. The Senior Professional Data Engineer will design, build, and optimize scalable data pipelines, lakehouse architectures, and streaming solutions for intelligent automation services in finance, while ensuring data quality, governance, security, and performance.
Responsibilities
- Design, build, and maintain scalable ETL/ELT pipelines using Apache NiFi
- Implement and manage data lakehouse structures based on Apache Iceberg, including schema evolution, partitioning, and time travel features
- Develop and optimize SQL queries and data access layers with Trino, ensuring high performance and reliability
- Integrate diverse data sources (Kafka, S3, RDBMS, REST APIs, etc.) into unified pipelines
- Ensure data quality, security, and governance across the data infrastructure
- Perform performance tuning, monitoring, and troubleshooting of data pipelines and distributed systems
- Collaborate with cross-functional teams (data scientists, platform engineers, business stakeholders) to deliver data-driven solutions
- Document technical designs, concepts, and processes
Skills
- * Proven experience with Apache Iceberg, Apache NiFi, and Trino
- * Understanding and ability to work with Apache Spark code
- * Experience with ETL/ELT processes and integrating APIs
- * Hands-on expertise in real-time streaming with Apache Kafka
- * Strong background in large-scale data processing and distributed systems
- * Cloud platforms (AWS, Azure, or GCP), especially S3, IAM, Glue
- * Programming languages: Python, Java, or Scala
- * Containerization and orchestration: Docker, Kubernetes
- * CI/CD & DevOps tools: Git, Jenkins, Terraform, GitHub Actions
- * Monitoring & Logging: Prometheus, Grafana, ELK Stack, Splunk
- * Strong knowledge of SQL and performance optimization techniques
- * Structured and solution-oriented working style
- * Strong communication and collaboration skills within interdisciplinary teams
- * Ability to document and explain complex technical processes in a clear manner
- * Experience with Azure cloud services
- * Familiarity with CI/CD automation and Infrastructure as Code
- * Practical knowledge of security configurations in data processing
Benefits
- An international and agile work environment that thrives on diversity, creativity, and collaboration.
Company Overview