Senior Data Engineer

The Senior Data Engineer is responsible for designing, building, and governing scalable cloud-based data platforms that enable analytics, business intelligence, machine learning, and AI-driven solutions. The role focuses on architecting modern Lakehouse environments, developing robust ETL/ELT pipelines, integrating enterprise data sources, implementing governance frameworks, and delivering reliable datasets that support strategic decision-making across the organization.This position works closely with business stakeholders, analysts, engineers, and leadership teams to transform raw data into trusted business assets while ensuring scalability, performance, security, and data quality.Key Responsibilities1. Data Platform Architecture– Design and maintain enterprise-scale Lakehouse and Data Warehouse architectures– Build modern data platforms using Microsoft Fabric, Azure, and AWS services– Define and implement Medallion Architecture (Bronze, Silver, Gold)– Design scalable data models supporting analytics and reporting requirements– Establish data architecture standards and best practices across the organization2. Data Engineering & Pipeline Development– Develop and maintain ETL/ELT pipelines for batch and near real-time data processing– Build scalable ingestion frameworks integrating APIs, databases, cloud storage, and third-party platforms– Automate data movement between operational systems and analytics environments– Optimize pipeline performance, reliability, and maintainability– Monitor and troubleshoot data workflows to ensure SLA compliance3. Microsoft Fabric & Analytics Engineering– Design and implement end-to-end solutions using Microsoft Fabric components:

• Data Factory, Lake house, Warehouse, Notebooks, Pipelines, OneLake, Semantic Models, Power BI– Implement Slowly Changing Dimensions (SCD Type 2)– Create curated business-ready datasets for analytics consumption– Optimize Fabric workloads using partitioning, V-Order, shortcuts, and performance tuning techniques4. Data Integration– Develop ingestion frameworks integrating data from multiple platforms including:
• Google Analytics, Google Ads, Meta Ads, Adjust, Maqsam
• Azure SQL Database, SQL Server, REST APIs, AWS S3, Google Sheets
• Internal ERP and transactional systems5. Data Governance & Security– Design and implement enterprise data governance frameworks– Manage role-based access control (RBAC) & RLS– Establish data quality monitoring and validation processes– Define metadata management and lineage standards– Support compliance and auditing requirements6. Analytics & Business Intelligence– Collaborate with stakeholders to define KPIs and business metrics– Build analytical datasets supporting executive dashboards– Deliver reporting solutions using Power BI– Enable self-service analytics across departments– Translate business requirements into scalable data solutions7. Performance Optimization– Optimize large-scale datasets containing millions of records– Improve query performance through indexing, partitioning, and storage optimization– Reduce processing costs and execution times– Identify bottlenecks and implement scalable solutions8. AI & Advanced Analytics Enablement– Build AI-ready data platforms supporting machine learning workloads– Develop Retrieval-Augmented Generation (RAG) data pipelines– Design vector-based data architectures for LLM applications– Prepare structured and unstructured data for AI use cases9. Leadership & Mentorship– Mentor junior and mid-level data engineers– Establish coding standards and engineering best practices– Conduct architecture reviews and technical design sessions– Lead cross-functional initiatives involving data, analytics, and business teams– Drive adoption of modern data engineering practicesRequirementsEducation– Bachelor's Degree in Engineering, Computer Science, Information Systems, Data Science, or related fieldExperience– 4+ years of experience in Data Engineering and Analytics– Experience designing enterprise data platforms– Proven experience with Microsoft Fabric implementations– Experience building large-scale ETL/ELT pipelines– Experience working with both structured and semi-structured data– Experience collaborating with business and executive stakeholdersTechnical SkillsCore Engineering: Advanced SQL, Python, PySpark, data modeling, dimensional modeling, ETL/ELT, and data warehousing.Microsoft Fabric: Lakehouse, Warehouse, Data Factory, Pipelines, Notebooks, OneLake, Semantic Models, and Power BI integration.Azure: Azure Data Factory, Azure SQL Database, Azure Synapse Analytics, and Azure Storage.Business Intelligence: Power BI, KPI design, dashboard development, and analytics engineering.AI & Emerging Technologies: Large Language Models (LLMs), RAG architectures, vector databases, and AI data pipelines.Preferred CertificationsMicrosoft Certified: Fabric Data Engineer Associate Microsoft Certified: Fabric Analytics Engineer Associate