Note: The job is a remote job and is open to candidates in USA. GDIT is a technology and professional services company delivering technology solutions and mission services to government, defense, and intelligence organizations. The AI/ML Engineer Associate will develop, implement, maintain, evaluate, and test AI-powered RAG and LLM solutions, while supporting model serving, application integration, infrastructure, documentation, and collaboration with senior engineers.
Responsibilities
- Implement and iterate document ingestion, chunking, and embedding pipelines (e.g., Nomic Embed v1.5)
- Tune retrieval parameters (chunk size, overlap, top-k, similarity thresholds) against evaluation sets
- Maintain and troubleshoot the vector store (PGVector on PostgreSQL) - indexing, query performance, schema updates
- Support day-to-day operation of the LLM serving layer
- Assist with model updates, version testing, and rollback procedures
- Monitor GPU utilization, memory usage, and inference latency
- Work within front-end integrations to wire up new features, prompt templates, or tool-calling workflows
- Build and maintain API integrations between the LLM layer and downstream applications (via PGBouncer/Postgres, Redis caching, etc.)
- Write and refine system prompts, few-shot examples, and prompt-engineering iterations for specific use cases
- Build/run evaluation harnesses to test retrieval accuracy and generation quality (hallucination checks, relevance scoring)
- Track regressions when models, embeddings, or chunking strategies change
- Document known failure modes and edge cases
- Assist with environment setup, dependency management, and container/service configuration in development environments
- Support basic troubleshooting of Redis, PostgreSQL, PGAdmin as they relate to the RAG pipeline
- Escalate deeper infra/networking issues to senior engineers or platform team
- Contribute to a test strategy for the RAG/LLM pipeline covering three distinct layers: retrieval quality (are the right chunks being pulled), generation quality (is the LLM producing accurate, grounded, non-hallucinated answers), and system/integration (does the pipeline work end-to-end under real conditions)
- Help define acceptance criteria for "good enough" retrieval and generation - e.g., minimum relevance score thresholds, acceptable hallucination rate, latency SLAs
- Participate in test planning for new features or model/embedding swaps - identify what could break (retrieval drift, prompt regressions, latency changes) before rollout
- Maintain a golden/reference dataset of representative queries and expected answers or expected retrieved sources, used as a stable benchmark across changes
- Execute manual exploratory testing for new features or edge cases automation doesn't yet cover - adversarial prompts, out-of-scope questions, ambiguous queries, multi-turn context handling
- Run pre-deployment validation checklists before pushing model, prompt, or pipeline changes to production
- Execute periodic regression passes on a schedule (not just at release time) to catch silent drift - since RAG/LLM systems can degrade without any code change (e.g., underlying model provider updates, data staleness)
- Validate fixes against the original defect/failure case plus the broader regression suite
- Maintain technical documentation for pipelines, configs, and architecture decisions
- Document runbooks for common operational tasks (restarting services, common errors, model swap procedures)
- Collaborate with senior engineers on architecture decisions
- Participate in code review, both giving and receiving feedback
- Communicate technical constraints/tradeoffs to non-technical stakeholders as required
Skills
- Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP)
- 0 + years of related experience
- Python, Machine Learning, Deep Learning, SQL, Data Science, PyTorch, Docker, TensorFlow, Artificial Intelligence, Natural Language Processing, Linux, JavaScript, MATLAB, Architecture, Data Analytics, Kubernetes, MSFT Azure Platform, Big Data, Hadoop, Visualization, Software Development, and Agile
- Scheduled Weekly Hours: 30
- Travel Required: Less than 10%
Benefits
- Medical plan options, some with Health Savings Accounts, for all US-based employees
- Dental plan options for all US-based employees
- Vision plan for all US-based employees
- 401(k) plan with the ability to contribute both pre- and post-tax dollars up to IRS annual limits and receive a company match
- Full flex work weeks where possible
- Paid time off plans, including vacation, sick and personal time
- Paid holidays
- Paid parental leave
- Paid military leave
- Paid bereavement leave
- Paid jury duty leave
- Typically 15 days of paid leave per calendar year for vacations, personal business, and illness
- An additional 10 paid holidays per year
- Up to 160 hours of paid leave in a rolling 12-month period through the GDIT Paid Family Leave program for eligible employees
- Short- and long-term disability benefits
- Life insurance
- Accidental death and dismemberment insurance
- Personal accident insurance
- Critical illness insurance
- Business travel and accident insurance
Company Overview