Azure Document Intelligence/Data Engineer

Role Summary

This role owns the document ingestion and extraction pipeline: turning scanned forms, PDFs, and other unstructured records into clean, structured, validated data ready for indexing, search, and downstream workflows. The person in this role builds the confidence-scoring and human-in-the-loop review mechanisms that keep extracted data trustworthy at scale.

Key Responsibilities

• Build and maintain document ingestion pipelines using Azure AI Document Intelligence (Form Recognizer).

• Implement OCR, document classification, and metadata/field extraction for scanned PDFs and mixed-format records.

• Design confidence-scoring logic and exception/review queues for low-confidence extractions.

• Build batch processing pipelines using Azure Functions, Logic Apps, or Data Factory.

• Model and maintain extraction and metadata data structures in SQL.

• Implement data quality checks, duplicate detection, and validation rules across the pipeline.

• Partner with the Search Engineer to ensure extracted data feeds cleanly into indexing.

Required Qualifications

• Production experience with Azure AI Document Intelligence / Form Recognizer.

• Strong OCR and document classification experience, including scanned/handwritten forms.

• Experience with confidence scoring and human-in-the-loop validation design.

• Proficiency in Python and SQL for data processing and pipeline development.

• Experience with Azure Functions, Logic Apps, or Data Factory (or equivalent batch/serverless tooling).

• Demonstrated experience handling data quality issues: duplicate detection, exception handling, reconciliation.

Preferred Qualifications

• Experience with LLM-assisted structured extraction (JSON schema outputs, prompt-based field extraction).

• Healthcare, insurance, legal, or government document-processing background.

• Familiarity with HIPAA-, FedRAMP-, or similarly regulated data-handling practices.