Note: The job is a remote job and is open to candidates in USA. SoTalent is seeking a Data Platform Engineer to build and operate scalable AI inference infrastructure for production machine learning workloads. The role focuses on distributed systems, model-serving optimization, GPU utilization, cloud-native platforms, observability, and platform reliability.
Responsibilities
- Design, build, and operate production-grade inference platforms for large machine learning models
- Develop scalable systems that support diverse AI and model-serving workloads
- Enhance platform performance, reliability, and operational efficiency
- Ensure consistent, low-latency model serving across large-scale deployments
- Drive continuous improvements in system architecture and infrastructure
- Design and implement distributed systems that support high-volume AI inference requests
- Build resilient services capable of handling large-scale traffic and workload variability
- Improve system scalability, fault tolerance, and resource utilization
- Optimize communication and coordination between distributed components
- Solve complex performance and availability challenges
- Implement intelligent request routing mechanisms
- Develop and optimize:
- Dynamic batching
- Request scheduling
- Workload distribution
- Inference caching
- Traffic management
- Improve throughput and reduce response latency
- Maximize inference efficiency across heterogeneous workloads
- Optimize GPU utilization for large model inference workloads
- Analyze GPU performance and identify resource bottlenecks
- Improve hardware efficiency and workload placement strategies
- Support deployment and scaling of GPU-intensive applications
- Collaborate on infrastructure decisions involving accelerated computing environments
- Build and manage containerized AI infrastructure
- Design scalable deployment patterns using Kubernetes
- Implement automated scaling and workload management solutions
- Support high-availability platform architectures
- Maintain production-grade cloud-native environments
- Establish end-to-end observability for AI serving platforms
- Develop monitoring, logging, tracing, and alerting capabilities
- Track system performance, resource utilization, and service health
- Create dashboards and operational insights for platform stakeholders
- Improve incident detection and response capabilities
- Maintain highly available AI infrastructure supporting production workloads
- Troubleshoot performance, reliability, and scaling issues
- Conduct root cause analysis and implement preventative improvements
- Improve platform resilience through automation and reliability engineering practices
- Participate in operational excellence initiatives
- Partner with:
- Machine Learning Engineers
- Research Scientists
- Data Engineers
- Infrastructure Teams
- Software Engineers
- Support deployment and operationalization of machine learning models
- Help bridge the gap between research and production systems
- Improve developer experience for ML practitioners
- Measure and optimize:
- Throughput
- Latency
- Availability
- Resource Efficiency
- Cost Performance
- Conduct benchmarking and capacity planning activities
- Implement platform enhancements that improve operational outcomes
- Drive engineering decisions through performance analysis and data
Skills
- Bachelor's or Master's degree in: • Computer Science • Software Engineering • Computer Engineering • Related technical field
- 6+ years of experience in: • Distributed Systems Engineering • ML Platform Engineering • Infrastructure Engineering • Large-Scale Backend Systems
- Experience operating production-grade platforms at scale
- Strong system design and problem-solving abilities
- Experience supporting machine learning workloads in production environments
- Python
- Go
- Rust
- C++
- Backend Systems Development
- High-Performance Distributed Systems
- Service Architecture
- Scalability Engineering
- Fault-Tolerant Systems
- Performance Optimization
- Large Language Models (LLMs)
- Model Serving Platforms
- Inference Systems
- ML Infrastructure
- Training-to-Production Pipelines
- LLM Inference Frameworks
- Request Routing
- Dynamic Batching
- Caching Strategies
- Autoscaling
- Traffic Management
- Kubernetes
- Containerization
- Infrastructure Automation
- Platform Operations
- Cloud Infrastructure
- GPU Architecture
- Accelerated Computing
- Resource Scheduling
- GPU Performance Optimization
- High-Performance Computing
- Monitoring
- Logging
- Distributed Tracing
- Alerting
- Incident Management
- Reliability Engineering
- Experience building LLM serving platforms at scale
- Familiarity with modern model inference frameworks
- Experience optimizing GPU-intensive applications
- Knowledge of high-performance networking and systems architecture
- Experience with platform observability and large-scale monitoring solutions
- Background in AI infrastructure, MLOps, or cloud-native platform engineering
Benefits
- Remote work in the United States.
Company Overview