Note: The job is a remote job and is open to candidates in USA. West 4th Strategy is hiring a Data Engineer to support the National Guard Bureau’s Minuteman project team. The role designs, develops, and maintains data pipelines and infrastructure for the NGB’s enterprise data environment, while supporting data processing, analysis, and governance.
Responsibilities
- Designs, develops, and implements end to end data pipelines utilizing ETL processes and technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, and SQL
- Works closely with cross functional teams to understand data requirements and design optimal data models and architectures
- Creates and optimizes data pipelines from scratch, ensuring scalability, reliability, and high performance processing
- Performs data cleansing, data integration, and data quality assurance activities to maintain the accuracy and integrity of large datasets
- Leverages big data technologies to efficiently process and analyze large datasets
- Collaborates with data scientists, analysts, and stakeholders to provide timely and accurate data insights and support decision making processes
- Troubleshoots data related problems and provides innovative solutions to complex data challenges
- Implements and enforces data governance policies and procedures, ensuring compliance with regulatory requirements and industry best practices
Skills
- Three (3) to ten (10) years of directly related experience
- 3+ years of experience as a Data Engineer
- Proficiency in ETL processes and firsthand experience with technologies such as Databricks, Python, Spark, Scala, JavaScript/JSON, SQL, and Jupyter Notebooks
- Experience working with large datasets and leveraging big data technologies to process and analyze data efficiently
- Understanding of data modeling/visualization, database design principles, and data governance practices
- B.A./B.S. or M.A./M.S. in a related field
- Active Secret security clearance
- U.S. citizenship is required since this role supports the U.S. Federal government
- Experience with the Department of War (DoW) mission and associated datasets
- Previous experience with Advana or the War Data Platform
- Experience with APIs
- Experience with Machine Learning models and/or Generative AI
- Experience building within Palantir Foundry applications, including Workshop, Ontology Manager, and Gaia
Benefits
- Options for health, dental, and vision insurance coverage
- 401k contribution options
Company Overview