GPU DevOps / Infrastructure Setup Engineer

We are looking for an experienced GPU DevOps / Infrastructure Setup Engineer to build, configure, deploy, and maintain GPU-enabled infrastructure for AI/ML and High-Performance Computing (HPC) workloads.The ideal candidate will have hands-on experience with GPU server setup, on-premises AI infrastructure, Linux administration, NVIDIA GPU drivers, CUDA, containerization, infrastructure automation, and GPU workload deployment.Experience with NVIDIA Base Command Manager (BCM), GPU cluster bring-up, InfiniBand networking, and LLM application deployment will be highly preferred. Candidates with experience across different GPU hardware vendors and cloud, on-premises, or hybrid environments are welcome.Key ResponsibilitiesSet up, configure, and maintain GPU-enabled servers, compute nodes, and AI infrastructure.Build and deploy on-premises AI infrastructure and GPU compute clusters.Perform end-to-end GPU cluster bring-up, including provisioning, configuration, validation, and testing.Configure and manage NVIDIA Base Command Manager (BCM) for cluster deployment and management, where applicable.Install and configure operating systems, GPU drivers, CUDA Toolkit, libraries, and supporting software.Deploy and maintain GPU workloads on bare-metal servers, cloud platforms, and hybrid environments.Configure GPU-enabled Docker containers and Kubernetes workloads.Deploy and manage LLM applications using frameworks such as vLLM, Ollama, Hugging Face Text Generation Inference (TGI), or NVIDIA Triton.Configure and troubleshoot GPU hardware, driver compatibility, CUDA initialization, memory utilization, and workload execution issues.Support InfiniBand networking and high-speed interconnects for multi-GPU or multi-node environments.Automate infrastructure provisioning, configuration management, and deployment using appropriate scripting and automation tools.Develop and maintain CI/CD pipelines for infrastructure and application deployments.Monitor GPU utilization, memory, temperature, power consumption, system health, and performance.Implement monitoring, logging, alerting, health checks, and automated recovery using tools such as Prometheus and Grafana.Optimize GPU utilization, VRAM consumption, inference latency, throughput, and overall infrastructure performance.Troubleshoot Linux, networking, storage, container, hardware, and application deployment issues.Perform infrastructure upgrades, patching, maintenance, capacity planning, and resource scaling.Maintain technical documentation, installation procedures, configuration guides, and troubleshooting records.Collaborate with AI/ML, development, platform, and infrastructure teams to support production workloads.Required Technical SkillsGPU Infrastructure and HardwareHands-on experience with GPU server setup, configuration, and operations.Knowledge of GPU hardware, server components, PCIe, power requirements, cooling, and hardware compatibility.Experience with GPU models such as NVIDIA A100, H100, H200, L40S, RTX series, or equivalent GPU platforms.Linux, Drivers, and CUDAStrong Linux administration and troubleshooting skills.Experience installing and configuring GPU drivers, CUDA Toolkit, and supporting libraries.Ability to validate GPU detection, driver functionality, CUDA compatibility, and workload execution.AI Infrastructure and HPCExperience building or maintaining on-premises AI infrastructure.Understanding of GPU cluster provisioning, compute nodes, and cluster validation.Experience with NVIDIA Base Command Manager (BCM) is preferred.Experience deploying on-premises GPU infrastructure without BCM is also valuable.Knowledge of InfiniBand, RDMA, and high-speed networking for GPU clusters is an advantage.DevOps and ContainerizationExperience with Docker and GPU-enabled containers.Kubernetes experience for GPU workload deployment and orchestration.Infrastructure automation using Bash, Python, Ansible, Terraform, or equivalent tools.Familiarity with CI/CD pipelines, Git, configuration management, and infrastructure monitoring.LLM Deployment and PerformanceExperience deploying LLM applications on GPU-enabled cloud instances or on-premises GPU servers.Familiarity with vLLM, Ollama, Hugging Face TGI, NVIDIA Triton, or equivalent model-serving frameworks.Understanding of API-based model serving and GPU resource allocation.Experience troubleshooting GPU memory issues and optimizing inference performance.Familiarity with Prometheus, Grafana, NVIDIA SMI, or DCGM for GPU monitoring.Experience Requirements3+ years of experience in DevOps, SRE, platform engineering, or infrastructure engineering.At least 1–2 years of hands-on experience working with GPU environments.Demonstrated experience in at least one real-world GPU infrastructure or AI deployment project.Ability to explain the infrastructure architecture, GPU hardware used, individual responsibilities, deployment process, and technical issues resolved.Experience supporting production environments is preferred.Preferred QualificationsEnd-to-end GPU cluster deployment using NVIDIA BCM.Hands-on InfiniBand configuration and troubleshooting.Bare-metal GPU server installation and configuration without Docker.Multi-GPU or multi-node cluster deployment.Production LLM serving and GPU performance optimization.Experience across on-premises, cloud, and hybrid infrastructure environments.Candidate ProfileWe are looking for engineers with practical infrastructure experience who can independently configure GPU environments, troubleshoot complex technical issues, and support reliable AI/ML and HPC deployments.Candidates must be able to explain real projects they have worked on, the GPU hardware and tools used, their individual contributions, and the challenges they successfully resolved.