AI Inference Engineer

Company Description Paranine is building a unified orchestration layer for sovereign AI inference on Huawei NPUs and Nvidia GPUs, enabling organizations to own, control, and understand their LLM token usage. The company focuses on delivering high-uptime, self-service infrastructure with optimized open-source model APIs that provide lower latency, reduced costs, and higher throughput, all while maintaining data and compute residency in chosen regions. Paranine offers dedicated inference deployments, fine-tuning and training capabilities, and bring-your-own model options. Its control panel provides full visibility into token consumption, request logs, and detailed cost attribution, with billing available in local currencies through regional paywalls. The first sovereign regional deployment is in Pakistan, launching in September 2026, with a roadmap to expand across the AMEA region by 2030.Role Description The AI Inference Engineer will design, implement, and optimize high-performance inference pipelines for large language models and other neural network architectures running on Huawei NPUs and Nvidia GPUs. This full-time role involves profiling and tuning model performance, managing model deployments, and building robust tooling for monitoring, logging, and cost attribution across the inference stack. The engineer will collaborate closely with software developers and ML specialists to integrate open-source models, ensure reliability and scalability, and maintain nines-level uptime for regional deployments. Day-to-day responsibilities include developing and maintaining orchestration services, optimizing resource utilization, troubleshooting production issues, and contributing to the evolution of Paranine’s sovereign inference platform.QualificationsStrong foundation in Computer Science concepts, including algorithms, data structures, and systems programming.Hands-on experience with Software Development, building production-grade services and APIs for distributed systems.Expertise in Neural Networks and Pattern Recognition, with a focus on optimizing inference workloads.Practical knowledge of Natural Language Processing (NLP), particularly deploying and scaling LLMs or similar models.Proficiency in languages such as Go, Python, C++, or Rust, and familiarity with GPU/NPU acceleration frameworks (e.g., CUDA, ROCm, vendor SDKs).Experience with cloud or on-prem orchestration tools and containerization (e.g., Kubernetes, Docker) for high-availability deployments.Background in monitoring and observability (metrics, logging, tracing) and performance profiling in production environments.Bachelor’s or Master’s degree in Computer Science.Ability to work collaboratively in a fast-paced environment and communicate clearly with cross-functional teams.Apply now to embark on a rewarding journey in shaping the future of AI!