Note: The job is a remote job and is open to candidates in USA. SMX Services & Consulting, Inc. is seeking a Data Engineer to design, develop, and optimize scalable data pipelines for advanced fraud analytics. The role involves working with various data types and ensuring data quality and governance within modern data-platform environments.
Responsibilities
- Design, build, test, deploy, and maintain scalable ETL and ELT pipelines
- Ingest and transform data from flat files, JSON, XML, Excel, APIs, relational databases, graph databases, streaming sources, and other formats
- Develop reusable SQL and Python processes for data ingestion, validation, standardization, transformation, enrichment, and loading
- Load, manage, and optimize data within Databricks Unity Catalog, Microsoft SQL Server managed instances, and comparable platforms
- Support streaming and batch ingestion frameworks within a modern Lakehouse architecture
- Conduct source-system profiling and document source structures, data definitions, relationships, limitations, and quality issues
- Develop data mappings, transformation logic, reconciliation procedures, and exception-handling processes
- Implement data-quality checks, completeness checks, validity rules, duplicate detection, and reconciliation controls
- Develop logging, monitoring, alerting, retry, recovery, and error-handling procedures
- Maintain data lineage, metadata, data dictionaries, schema documentation, and operational runbooks
- Support entity resolution, record linkage, data matching, graph ingestion, and fraud-model feature development
- Optimize SQL queries, transformation processes, storage structures, and pipeline performance
- Implement enterprise data-management, data-governance, and data-quality standards
- Support role-based access, data segregation, auditability, and approved information-handling requirements
- Work with data owners and government stakeholders to resolve access, quality, interpretation, and integration issues
- Collaborate with data scientists, graph data scientists, investigative analysts, forensic accountants, and business analysts
- Support transition and continued operation of existing data pipelines without service interruption
- Maintain code and technical artifacts within government-approved repositories and version-control environments
- Support production releases, configuration management, testing, and operational maintenance
Skills
- Minimum of three years of professional experience in data engineering or a closely related discipline
- Demonstrated experience designing, building, and maintaining scalable ETL or ELT pipelines
- Strong SQL and Python skills or equivalent data-engineering capabilities
- Experience ingesting and transforming flat files, JSON, XML, Excel, APIs, relational data, and graph data
- Experience with Databricks Unity Catalog, Microsoft SQL Server managed instances, or comparable platforms
- Experience with streaming and batch data-ingestion frameworks
- Experience with modern Lakehouse, cloud-data, or distributed-data architecture
- Experience implementing data-quality, lineage, reliability, monitoring, and performance controls
- Familiarity with enterprise data management, metadata, data governance, and data-quality practices
- Experience documenting schemas, transformation logic, pipelines, data mappings, and operational procedures
- Ability to troubleshoot data, pipeline, performance, and production issues
- Ability to collaborate with technical, investigative, and business stakeholders
- Strong analytical, documentation, communication, and problem-solving skills
- Ability to complete federal suitability, HSPD-12/PIV credentialing, and system-access requirements
- Experience supporting fraud detection, anomaly detection, financial oversight, investigative analytics, or government-benefit programs
- Experience with Azure Databricks, Apache Spark, Delta Lake, Unity Catalog, Microsoft SQL Server, Neo4j, Power BI, or comparable technologies
- Experience with orchestration, automated testing, CI/CD, Git-based repositories, and production monitoring
- Experience supporting entity resolution, graph-data ingestion, machine-learning feature pipelines, or investigative analytics
- Degree in computer science, data engineering, information systems, software engineering, or a related discipline
Benefits
- W2 option
- Primarily remote, with occasional onsite work in Washington, DC
Company Overview