Note: The job is a remote job and is open to candidates in USA. LDSS Corp is a Woman-Owned Small Business based in Dayton, Ohio, committed to delivering high-impact solutions to federal agencies. They are seeking a skilled Data Engineer to build and optimize ETL data pipelines, develop scalable data models, and collaborate with cross-functional teams to support the Air Force’s Maintenance, Repair, and Overhaul program.
Responsibilities
- Build and optimize ETL data pipelines using Databricks and Apache Spark
- Develop scalable data models leveraging Spark Data Frames and Spark SQL for efficient querying, aggregation, and integration against numerous diverse data sources
- Utilize Python and Regular Expressions for advanced text parsing, pattern matching, and data cleansing to prepare structured datasets for downstream ingestion
- Use APIs for seamless data exchange and automation across systems
- Manage SharePoint data (lists, libraries, files, metadata) programmatically using Python and Microsoft Graph API
- Leverage Microsoft 365 tools for documentation and collaboration
- Develop and maintain Amazon Web Services (AWS) including S3
- Fully utilize the Microsoft 365 environment for cross-functional data sharing and productivity
- Use Jira to track and manage issues, tasks, and project workflows
- Collaborate with cross-functional teams and technical stakeholders to define data transformations of legacy systems to modern schemas
- Leverage SME collaboration to interpret business rules and build business logic into data extracts and transformation processes
- Build and maintain strong working relationships with SME’s, helping to identify requirements for data transformations based upon iterative product feedback
Skills
- Bachelor's degree in computer science, engineering, or a related field
- Experience in Databricks
- Experience in Apache Spark
- Experience in Python
- Experience in AWS
- Experience in Microsoft 365
- Experience in SharePoint
- Experience in developing and consuming APIs for data exchange between different systems
- Experience in Jira or other agile workflow management systems for issue and project tracking
- Proven ability to build trust and maintain strong partnerships with customers and stakeholders
- Takes initiative and drives projects forward with minimal supervision, anticipating challenges before they arise
- Meticulous in managing complex tasks and deadlines, ensuring accuracy and consistency across deliverables
- Thrives in cross-functional environments, fostering open communication and teamwork across diverse groups
- Rapidly assimilates new technologies and processes, adapting to evolving requirements in dynamic settings
- Ability to work collaboratively with a diverse team of contractors and government personnel
- Skilled in building strong customer relationships and engaging with Subject Matter Experts (SMEs) to derive business logic for data transformation methods and rules
- AWS Certified Solutions Architect
- Databricks Certified Data Engineer
- U.S federal government security clearance
Benefits
- 401(k)
- 401(k) matching
- Dental insurance
- Flexible spending account
- Health insurance
- Health savings account
- Life insurance
- Paid time off
- Referral program
- Tuition reimbursement
- Vision insurance
Company Overview