Cloudary is a cloud consulting firm founded and led by senior cloud architects. We embed Principal-Vetted engineers directly into high-performing North American tech teams — acting as agile, hands-on experts driving critical data and AI initiatives.
We are looking for senior generalist engineers with real depth in LLM and agentic systems. This is not a ticket execution role: you get the problem and the context, you propose the solution, you build it, you ship it, and you defend the technical decisions in front of the client.
Requirements
The Role: What You Are Actually Doing
You will take a production-grade LLM agent from read-only insight toward supervised action: hardening it for scale and staging it up a capability ladder (Explains → Recommends → Orchestrates → Acts). That means agent orchestration, evaluation you can trust, observability, cost and latency engineering, resilience, and safe write-actions with a human in the loop, all on AWS. Fluency with AI-assisted engineering (Claude Code, Cursor, Copilot, or equivalent) is the baseline here, not a differentiator — but you sign the code, and "the AI wrote it" is never an answer when something breaks in production.
Key Responsibilities
Own End-to-End Delivery: Take a production LLM agent from requirement to production deploy, and defend the architecture and trade-offs directly with the client.
Agent Orchestration: Build and harden orchestration (LangGraph / LangChain or equivalent) — routing, tool-calling, planning, synthesis, and state management.
Tool & Retrieval Integration: Integrate tools over MCP and keep a growing tool surface fast and correct, including BM25, hybrid, or vector retrieval as scale demands.
AWS Bedrock & AgentCore: Run models on Bedrock and Bedrock AgentCore — model selection/routing, guardrails, memory, and regional residency profiles.
Evaluation & Observability: Build the eval harness (golden sets, LLM-as-judge, quality gates wired into CI) and instrument the system with OpenTelemetry for per-session token, cost, and latency attribution.
Cost, Latency & Resilience: Drive down cost and latency with real levers (model routing, prompt caching, payload pruning, parallelizing independent calls) behind a regression gate, and build in circuit breakers, fallbacks, and dead-letter handling.
Safe Write-Actions: Design and stage write-actions with least-privilege permissions, human-in-the-loop approval, plan versioning, audit trail, and rollback — released behind feature flags to a small cohort first.
What We Are Looking For
5+ Years in Software Engineering: At least 2 of them genuinely at a senior level, with strong Python in production and comfort picking up TypeScript or Go when a project calls for it.
Hands-On LLM Application Engineering: Prompt design, tool/function calling, structured output, context management, and token budgeting, in a system real users hit.
Agent Orchestration Experience: Built or operated orchestration with a framework like LangGraph or LangChain, or hand-rolled, beyond single-prompt calls.
Managed LLM/Agent Platform in Production: AWS Bedrock, Google Vertex AI, or Azure AI Foundry — model invocation, streaming, guardrails, and agent tooling. We use Bedrock and Bedrock AgentCore; equivalent depth on Vertex AI or Azure transfers directly.
Evaluation, Retrieval & Observability: Eval harnesses and golden/reference sets, vector or hybrid search in production, and OpenTelemetry-based distributed tracing with token/cost/latency attribution.
Production AWS, CI/CD & IaC: Real IAM, networking, storage, and observability experience, CI/CD pipelines versioned as code, and Terraform in production.
Working English: Comfortable defending system design and technical decisions directly on client calls.
Our Tech Stack
AWS Bedrock, Bedrock AgentCore, LangGraph/LangChain, MCP, Terraform, GitHub Actions/GitLab CI, OpenTelemetry
Highlights
Engagement details
Remote. Full Time
Contractor, USD
Contract length: 2-3 months
Possibility to work Pacific Time hours (Vancouver)
Originally posted on