Note: The job is a remote job and is open to candidates in USA. Bright Vision Technologies is seeking a Senior Data Engineer to design, build, and operate large-scale data processing pipelines and analytics platforms using Hadoop and related big-data technologies. The role supports enterprise analytics, machine learning, and reporting workloads by ingesting, transforming, and analyzing structured and unstructured data.
Responsibilities
- We are seeking an experienced Senior Data Engineer - Hadoop to design, build, and operate large-scale data processing pipelines and analytics platforms on Hadoop and related big-data ecosystems
- In this role you will be responsible for ingesting, transforming, and analyzing massive volumes of structured and unstructured data to support enterprise analytics, machine learning, and reporting workloads
Skills
- Bachelor's degree in Computer Science, Engineering, or a related technical discipline
- Five or more years of professional experience designing and operating big-data pipelines on Hadoop
- Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments
- Solid experience with Hive, HDFS, Sqoop, HBase, and the broader Hadoop ecosystem
- Hands-on experience with streaming data platforms such as Kafka, Spark Streaming, or Flink
- Strong SQL skills and experience working with both relational and NoSQL data stores
- Experience with workflow orchestration tools such as Airflow or Oozie
- Solid understanding of distributed systems concepts, including partitioning, replication, and fault tolerance
- Strong scripting skills in Python or Shell
- Excellent troubleshooting, debugging, and documentation skills
- U.S. Citizens, Green Card Holders, EAD Holders, and H-1B transfer candidates
- We are unable to sponsor new H-1B visa petitions for this position
- Experience operating Hadoop on cloud platforms such as AWS EMR, Azure HDInsight, or Databricks
- Familiarity with modern lakehouse formats (Delta, Iceberg, Hudi)
- Exposure to data governance tooling such as Apache Atlas or Collibra
- Experience with Kubernetes-based data platforms (Spark-on-K8s, Trino)
- Hands-on experience with CI/CD and infrastructure-as-code in data engineering workflows
Benefits
- 100% Remote (U.S.)
- Tremendous career growth potential
Company Overview