Role & Responsibilities Overview:
- Prompt Design & Optimization
- Design prompts for - summarization, risk insights, adverse media analysis, underwriter assistant agent etc.
- duplicate explanation, decision support
- Optimize prompts for accuracy, grounding, and consistency
Evaluation & Continuous Improvement
- Define evaluation scenarios and validation datasets
- Use LLM-as-judge and feedback loops to improve responses
- Continuously refine prompts based on - model performance, user feedback
RAG & Assistant Enablement
- Work with AI engineers to - improve retrieval quality, design context injection strategies
- Ensure outputs are – grounded, explainable, aligned with underwriting logic
Business Alignment
- Translate underwriting workflows into prompt logic
- Incorporate domain-specific language, risk indicators, and business rules
- Collaborate with underwriters and SMEs
Prompt Governance
- Maintain prompt library and versioning
- Define templates for different LOBs and use cases
- Ensure consistency across assistant and workflows
Candidate Profile:
- Experience: 5–8+ years with 2–4+ years in GenAI / prompt engineering
- Domain (Mandatory): Strong insurance / underwriting knowledge
- Education: Bachelor's/Master's in Engineering, Data Science, Business Analytics, or related
- Profile Type: Hybrid business + technical, strong analytical thinker
Technical skills:
- GenAI & Prompting - prompt engineering techniques (few-shot, chain-of-thought, structured prompts);Experience with Azure OpenAI / LLM APIs; Prompt versioning and optimization
- RAG & Retrieval Awareness - understanding of Azure AI Search and retrieval behavior; Context design and grounding strategies; Ability to reduce hallucinations via prompt + retrieval design
- Evaluation & Quality - LLM-as-judge / evaluation frameworks; Output validation and benchmarking; A/B testing of prompts
- Safety & Governance - Awareness of guardrails (NeMo Guardrails); PII handling (Presidio); Responsible AI practices
- Collaboration & Tools - Familiarity with APIs, JSON, and structured outputs; Basic Python; Working with AI engineers and data teams
Originally posted on