Senior Data Engineer

Company DescriptionOpen Innovation AI is a global technology company that specializes in developing advanced solutions for managing AI workloads. Its flagship product, the Open Innovation Cluster Manager (OICM), orchestrates complex AI tasks efficiently across diverse infrastructures. The platform is hardware-agnostic, optimized for various GPUs and accelerators hardware, and facilitates seamless integration and scalability for enterprise AI applications. Open Innovation AI focuses on optimizing and simplifying AI workload management and making AI technologies accessible to organizations of all sizes. With its innovative solutions, companies can reduce operational costs, accelerate time to value, and maximize their return on investment, ensuring that their AI strategies contribute directly to enhanced business outcomes.Role Overview:Open Innovation AI is seeking an experienced Senior Data Engineer to contribute to the development of our MLOps platform and to work on deploying Large Language Models (LLMs). This role involves both maintaining scalable data infrastructure and actively participating in the enhancement of our machine learning operations.Role Requirements: Design, construct, install, test, and maintain scalable data management systemsActively contribute to the development and enhancement of our MLOps platformWork on deploying and managing Large Language Models, ensuring efficiency and scalabilityDevelop data set processes for data modeling, mining, and productionIntegrate new data management technologies and software engineering tools into existing structuresCollaborate with the engineering team and other stakeholders in automating the deployment of machine learning modelsRecommend improvements to data reliability, efficiency, and qualityResearch industry trends and apply this knowledge to data infrastructure projectsUtilize data to identify tasks that can be automated and streamline processesRequired QualificationsBachelor’s or master’s degree in computer science, Engineering, or a related field Hands-on experience with Apache Airflow and ClickHouse (database), along with proficiency in programming languages such as Python, Java, C++, or ScalaProven experience as a Data Engineer with a focus on MLOps platforms and big data technologiesExperience in deploying and managing Large Language Models (LLMs)Strong proficiency in SQL and NoSQL databases, data warehousing solutionsSkilled in data pipeline, data modeling, ETL and workflow management toolsKnowledge of cloud services (AWS, Azure, GCP) and their application in data engineeringExcellent problem-solving skills and ability to work in a collaborative environmentStrong project management and organizational skills