Note: The job is a remote job and is open to candidates in USA. Nasscomm is seeking a Data Engineer to review and improve data lake documentation in Unity Catalog. The role involves analyzing data pipelines, documenting use cases, and ensuring data quality and governance for responsible AI applications.
Responsibilities
- Review existing Unity Catalog table and field documentation and identify gaps, inaccuracies, and missing business context
- Analyze upstream data pipelines and SQL logic to understand how data is sourced, transformed, and published
- Document approved data use cases, including how specific datasets should be used for reporting, analysis, and AI/LLM scenarios
- Document data restrictions, including sensitive data considerations, inappropriate use cases, and areas where definitions or quality are not strong enough for broad agent or LLM use
- Help identify and structure use cases where LLMs or agents could responsibly interact with the data
- Test LLM responses against documented definitions and use cases to evaluate whether outputs are accurate, safe, and grounded in the intended business context
- Flag gaps in metadata, definitions, lineage, data quality, and ownership that must be addressed before wider AI enablement
Skills
- Strong SQL skills
- Experience working with data lake, warehouse, or analytics environments
- Ability to read and reason through data pipelines and transformation logic
- Strong documentation skills, especially translating technical logic into business-friendly definitions
- Comfortable leading discovery conversations with technical and business stakeholders
- Familiarity with data governance, data quality, semantic definitions, and responsible AI / LLM validation
Company Overview