AI and Data Engineer

Responsibilities1. Data Cleaning, Curation & Pipeline Development Clean, normalize, and validate unstructured or messy data to ensure high quality inputs for analytics and AI models.Write and optimize DAX queries to programmatically extract and aggregate tabular data directly from Power BI reports and datasets.Write performant SQL queries, stored procedures, and schema migrations across both PostgreSQL and SQL Server (SSMS).Build, automate, and monitor reliable data pipelines (ETL/ELT) using Python and Azure cloud services.2. AI, GenAI & Model Performance Optimization Assist in developing and maintaining AI/GenAI workflows (RAG pipelines, prompt templates, vector search integrations with Azure OpenAI).Profile and optimize inference speed and resource usage for Machine Learning and GenAI systems: Implement caching, prompt token reduction, and smart chunking strategies to minimize LLM latency and API bills.Apply model optimization techniques (such as ONNX runtime conversion, quantization, batching, and GPU/CPU profiling).Track and monitor model accuracy, latency, and throughput in staging and production.3. Statistical Analysis & Validation Apply fundamental statistical concepts (summary statistics, distributions, hypothesis testing, data sampling, regression) to audit data distributions and evaluate model outputs.Identify anomalies, outliers, and data drift across pipeline stages.4. Version Control & CI/CD Maintain clean, documented code and participate in peer code reviews using Git and GitHub.Work with automated build and deployment pipelines (GitHub Actions or Azure DevOps).QualificationsRequired Qualifications & Technical Skills Experience: 2 to 4 years of hands-on experience in data engineering, analytics engineering, or applied machine learning.Databases & Querying: Solid working proficiency in both PostgreSQL and SQL Server Management Studio (SSMS), including writing complex joins, indexing, and analyzing basic query execution plans.Practical experience writing and debugging DAX queries to fetch data out of Power BI models.Programming & Data Wrangling: Strong Python skills with libraries like Pandas, NumPy, and SQLAlchemy for data manipulation and cleaning.AI & Optimization Foundations: Understanding of ML/DL lifecycle and basic Generative AI concepts (LLM APIs, embeddings, vector search).Hands-on exposure to latency optimization (caching, batch inference, token usage control, ONNX/runtime speedups).Mathematics & Statistics: Solid understanding of probability, distributions, data validation methods, and evaluation metrics (precision, recall, MSE, perplexity).Cloud & Collaboration Tools: Exposure to Microsoft Azure services (Azure Data Factory, Blob Storage / ADLS, Azure OpenAI).Daily proficiency with Git and GitHub (branching, PR workflows, resolving merge conflicts).Required SkillsData engineering, analytics engineering, or applied machine learning experience (2 to 4 years).PostgreSQL and SQL Server Management Studio (SSMS), including complex joins, indexing, and basic query execution plan analysis.DAX queries to fetch data out of Power BI models.Python (Pandas, NumPy, SQLAlchemy) for data manipulation and cleaning.Understanding of ML/DL lifecycle and basic Generative AI concepts (LLM APIs, embeddings, vector search).Latency optimization (caching, batch inference, token usage control, ONNX/runtime speedups).Probability, distributions, data validation methods, and evaluation metrics (precision, recall, MSE, perplexity).Microsoft Azure services (Azure Data Factory, Blob Storage / ADLS, Azure OpenAI).Git and GitHub (branching, PR workflows, resolving merge conflicts).Preferred SkillsFamiliarity with tools like DAX Studio or Tabular Editor.Experience with LangChain, LlamaIndex, or Azure AI Search.Any foundational Microsoft Azure certification (e.g., DP-900, DP-203, or AI).