Note: The job is a remote job and is open to candidates in USA. Parallel Wireless is a U.S.-based company advancing Open RAN innovation and software-centric mobile network solutions. The company seeks a senior technical leader to architect, build, and operate a secure local LLM and generative AI platform, including model serving, retrieval-augmented generation, enterprise search, evaluation, security, integrations, and production operations.
Responsibilities
- Own the architecture and technical roadmap for a secure, reliable, and maintainable local LLM platform deployed in Parallel Wireless-controlled infrastructure
- Partner with engineering, product, support, IT, information security, legal, and domain experts to prioritize high-value use cases and translate them into measurable product and platform requirements
- Build a modular inference and model-gateway layer with stable APIs, model routing, streaming, concurrency controls, quotas, and the ability to change models or serving backends without rewriting every application
- Evaluate open-weight language, code, embedding, reranking, and, where useful, multimodal models against PW-specific tasks; document model provenance, licenses, limitations, security posture, hardware needs, and total cost of ownership
- Optimize serving across available CPU, GPU, and accelerator resources using techniques such as continuous batching, caching, parallelism, quantization, and right-sized context limits while protecting output quality
- Design and operate RAG and enterprise-search pipelines for approved repositories, wikis, tickets, standards, design documents, test results, logs, and support content, including parsing, chunking, metadata, embeddings, hybrid retrieval, reranking, freshness, citations, and deletion
- Enforce source-system permissions throughout ingestion and retrieval so that the platform never exposes content a user is not authorized to access; integrate with company identity, SSO, role-based access control, secrets management, and audit logging
- Establish versioned evaluation datasets and automated offline and online evaluation for retrieval quality, groundedness, factual accuracy, citation quality, code correctness, task completion, latency, safety, and refusal behavior
- Create release gates and reproducible regression tests for changes to models, prompts, tools, embeddings, retrieval logic, indexes, and serving configurations; support canary releases, rollback, and clear approval paths
- Implement end-to-end observability for model and agent workflows, including traces, errors, time to first token, inter-token latency, throughput, queue time, resource utilization, saturation, availability, and user feedback
- Design safe tool-calling and agent workflows with least-privilege access, sandboxing, input and output validation, bounded execution, human approval for consequential actions, and complete traceability
- Integrate the platform into the tools employees already use—such as developer environments, source-control and CI workflows, knowledge systems, ticketing systems, and internal applications—through reusable SDKs, APIs, and reference implementations
- Build the operational foundations for production use: CI/CD, configuration and model registries, backups, disaster recovery, capacity planning, dependency and vulnerability management, incident response, and lifecycle policies for models and data
- Protect proprietary and personal information through network isolation, encryption, retention controls, redaction where appropriate, secure logging, and defenses against prompt injection, data poisoning, unsafe output handling, and model-supply-chain risks
- Determine when prompt or retrieval improvements are sufficient and when parameter-efficient fine-tuning, distillation, or other adaptation is justified by measured quality gains
- Make the platform usable beyond the core AI team through documentation, examples, training, office hours, and hands-on collaboration; use telemetry and structured feedback to improve adoption and effectiveness
- Communicate architecture decisions, quality evidence, risk, capacity, and roadmap tradeoffs clearly to technical and business stakeholders
Skills
- BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; exceptional candidates with equivalent depth are welcome
- Strong Python engineering skills and experience designing maintainable APIs, services, libraries, and data pipelines
- Strong understanding of transformer-based language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, tool calling, and common model failure modes
- Demonstrated experience building production RAG or enterprise-search systems using embeddings, vector and/or lexical search, metadata filtering, reranking, source attribution, and systematic retrieval evaluation
- Experience defining task-specific LLM evaluations using representative datasets, strong baselines, domain-expert review, automated metrics, human feedback, error analysis, and regression thresholds
- Experience deploying and operating containerized services on Linux using Docker and Kubernetes or an equivalent orchestration environment
- Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering
- Strong knowledge of distributed-system fundamentals, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls
- Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response
- Sound technical judgment about quality, security, maintainability, hardware efficiency, and total cost—not just model benchmark scores
- Ability to lead an ambiguous, cross-functional initiative, explain complex AI behavior in plain language, and help other teams ship safely on a shared platform
- Experience operating LLMs in on-premises, private-cloud, restricted-network, or air-gapped environments
- Hands-on experience with current inference runtimes and serving systems such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, Triton, or equivalent technologies
- Experience optimizing inference on NVIDIA and/or AMD GPUs using CUDA, ROCm, profiling tools, tensor parallelism, pipeline parallelism, speculative decoding, prefix/KV caching, or related techniques
- Experience with model and experiment registries, LLM tracing and evaluation platforms, vector databases, hybrid-search engines, and production data-orchestration frameworks
- Experience with parameter-efficient fine-tuning methods such as LoRA/QLoRA, dataset curation, synthetic-data generation, distillation, and post-training evaluation
- Experience building code intelligence, repository-aware assistants, developer tools, or IDE and CI integrations for large C/C++ and Python codebases
- Familiarity with Active Directory or another enterprise identity provider, fine-grained document authorization, data-loss prevention, secure software supply chains, model licensing, and AI governance
- Experience red-teaming LLM or agent systems for prompt injection, sensitive-data disclosure, poisoned retrieval content, excessive agency, and insecure output handling
- Knowledge of telecommunications, 3GPP, RAN/Open RAN, cloud-native network functions, or technical-support workflows
- Experience working across heterogeneous compute platforms and making performance, energy, and TCO tradeoffs for enterprise AI workloads
- Contributions to relevant open-source AI, search, MLOps, or infrastructure projects
Benefits
- Remote work arrangement
Company Overview
Company H1B Sponsorship