Note: The job is a remote job and is open to candidates in USA. Nexera is an AI-native consulting firm that builds custom intelligent solutions for organizations navigating the age of AI. The AI Infrastructure & LLM Inference Engineer will build, deploy, optimize, secure, and operate production AI infrastructure and model-serving platforms across on-premises, private-cloud, hybrid, and restricted-network environments. The role also involves automation, observability, client collaboration, knowledge transfer, and operational documentation.
Responsibilities
- Design and Deploy AI Infrastructure: Build and configure secure, production-ready platforms for running large language models and other AI workloads across on-premises, private-cloud, hybrid, and restricted-network architectures
- Operate Model-Serving Platforms: Deploy, configure, qualify, and maintain models, inference runtimes, gateways, and supporting services throughout their operational lifecycle
- Optimize Performance and Capacity: Measure and improve latency, throughput, concurrency, resource utilization, and model placement using repeatable benchmarks and real workload evidence
- Build for Reliability and Security: Implement monitoring, alerting, access controls, auditability, incident response, recovery procedures, and secure software and model lifecycle practices
- Automate Platform Delivery: Use infrastructure-as-code, configuration management, testing, and release automation to make deployments, upgrades, scaling, and rollback repeatable and boring
- Partner and Transfer Knowledge: Work directly with client and application teams, communicate technical tradeoffs, document decisions and operating procedures, and leave engineers genuinely able to run and extend the platform
- Innovate How We Work: Experiment with and define the agentic workflows, automation, and tooling that improve how Nexera builds and operates AI platforms. Your patterns become part of our operating model
Skills
- Systems experience that shows. Infrastructure, Linux systems, SRE, platform engineering, HPC, distributed systems, MLOps, or production ML engineering. Roughly three to five years, or equivalent capability you can demonstrate
- Hands-on LLM and local-inference work. You have deployed or operated LLMs, inference runtimes, or GPU-backed workloads in production
- Strong Linux fundamentals. You are comfortable troubleshooting below the application layer: services, processes, networking, storage, permissions, resource limits, containers, drivers, and OS behavior
- Practical GPU knowledge. You understand GPU memory, model placement, multi-GPU communication, utilization, thermal and power behavior, and how model architecture, precision, context length, and batching translate into capacity
- Production ownership. You have run systems other people depend on and know what monitoring, alerts, incident response, upgrades, backups, rollback, and handoff actually require
- Performance discipline. You measure before you optimize, you state the workload behind your numbers, and you can walk someone through how you found a bottleneck across application, runtime, memory, communication, or hardware
- Security-minded engineering. Least privilege, network isolation, artifact integrity, identity propagation, auditability, vulnerability management, and restricted-network operations are familiar territory
- Daily AI tool user. You already use AI tools (Claude, Claude Code, Copilot, Cursor, or similar) in how you code, research, and solve problems, and you can clearly show and demonstrate how you apply them to day-to-day engineering work
- Relentless curiosity. You follow new model and runtime releases because you find it interesting, not because someone asked you to. You have opinions about where local inference is headed and you want to prove them out
- Strong communication skills. You can explain a GPU bottleneck to an engineer, summarize risk and options for a program lead, work constructively with security and data-center teams, and document decisions clearly
- Consulting mindset. You can work in a client's environment, navigate ambiguity, surface blockers early, make pragmatic recommendations, and deliver against agreed outcomes
- U.S. citizenship and residency: Required for this role due to the customer environments and information the work may support
- Location: Remote-first (US-based), with scheduled on-site work at client sites and data centers
- Travel: Expect approximately 25% travel, concentrated in bursts around platform bring-up, qualification, deployment, and major troubleshooting events, with lighter travel between deployments. On-site work is scheduled in advance wherever possible
- Eligibility: U.S. citizenship and current U.S. residency are required
- Access Requirements: Must be willing and able to complete applicable client background investigations, satisfy site-access and data-handling requirements, and obtain and maintain a U.S. government security clearance if required for assigned work
- Military or cleared experience preferred: Prior U.S. military service, experience supporting defense or national-security organizations, and/or a current or previously held U.S. government security clearance are preferred but not required
- Operating systems and hardening: RHEL, Rocky Linux, AlmaLinux, SELinux, FIPS mode, DISA STIG, OpenSCAP, systemd
- NVIDIA GPU stack: H100/A100/HGX-class systems, CUDA, NCCL, NVLink/NVSwitch, Fabric Manager, NVML, DCGM/DCGM Exporter, Nsight Systems, Nsight Compute
- Inference and ML runtimes: vLLM, SGLang, TensorRT-LLM, NVIDIA Triton or Dynamo, Hugging Face Transformers/TGI, PyTorch, Ray Serve
- Inference concepts: tensor and pipeline parallelism, BF16/FP8/INT4, quantization, continuous batching, KV and prefix caching, prefill/decode behavior, speculative decoding, long-context serving, structured output, tool calling
- Containers and orchestration: Podman, Docker, containerd, Kubernetes, OpenShift, NVIDIA GPU Operator, Slurm, Helm
- Gateway and identity: NGINX, LiteLLM or comparable gateways, OpenAI-compatible APIs, Keycloak, FreeIPA, OIDC, JWT, service accounts, TLS
- Observability and operations: Prometheus, Grafana, OpenTelemetry, Elastic, Wazuh, Splunk, Loki, alerting and SLO tooling
- Automation and supply chain: Python, Bash, Ansible, Terraform or equivalent infrastructure-as-code, Git, private RPM/Python/container repositories, SBOMs, cosign, artifact signing and offline promotion
- AI CLI and agentic tools: Claude Code, Gemini CLI, OpenAI Codex, GitHub Copilot
- Additional differentiators: C++, CUDA or Triton kernel work, inference-runtime contributions, InfiniBand/RDMA/RoCE, regulated or classified environments, and on-premises or air-gapped deployments
Benefits
- Performance bonus
- Flexible PTO plus federal holidays
- Annual learning and certification budget
- Access to premium AI tooling (Claude Code and the frontier tools you will use daily)
- Home-office setup
- Sensible travel policy — you book what makes the trip productive, and travel time is respected as work time
Company Overview