Senior Data Engineer

Job Description

We are looking for a hands-on Sr. Data Engineer to build the engineering backbone of Osol's Data & Analytics practice. You will prove and standardise how we engineer data through proofs of concept, pilots and reusable accelerators and apply that standard to client engagements, industrial data platforms and the Industrial AI program we are delivering in our group's manufacturing plants.We are platform-agnostic: our work runs on any major cloud, on premises or both, and you will ingest any kind of data batch or real-time, from ERP systems to plant sensors securely and cost-efficiently. This is a senior individual contributor role; as the practice grows, you will mentor engineers and lead a data engineering team.

Key Responsibilities

Methodology & standards: Define, with the Head of Data & Analytics, Osol's standard data engineering methodology reference patterns (lakehouse layering, change data capture, streaming), coding and testing standards, definition of done and estimation guides.Proofs of concept: Plan and run proofs of concept and pilots across cloud, on-premises and hybrid platforms against agreed success criteria, and publish evidence-based comparisons of capability, performance, cost and operability.Reusable accelerators: Build metadata-driven ingestion frameworks (batch, CDC, streaming and files), data quality rule libraries, CI/CD and infrastructure-as-code templates, and demo environments for presales and partners.Multi-source ingestion: Build pipelines for structured, semi-structured and unstructured data databases, SAP and Oracle ERP, APIs, files, documents, logs, event streams and IoT sensors in batch, micro-batch and real time, across industries such as manufacturing, finance, telecom and retail.Streaming & CDC: Implement change data capture and event-driven integration, handling late, duplicated and out-of-order data and backfilling long histories.Quality by design: Build in security (identity and access, encryption, masking), governance (catalogue, lineage, classification, retention), data quality checks, reliability and cost control, and operate pipelines with orchestration, automated testing, CI/CD and SLA-based monitoring.Portable, reusable pipelines: Build collection and storage patterns that run the same on premises and in the cloud, are configured rather than recoded for each plant or client, and keep each client's data separate.Industrial AI use cases: Deliver reliable, analysis-ready datasets and feature pipelines for use cases such as predictive maintenance, and take them from pilot into dependable production.Presales & delivery: Support presales with technical discovery, sizing, estimates and demos; lead technical delivery through design and code reviews; and hand over solutions clients can run themselves.Industrial data acquisition experience is preferred: Acquire data from plant historians, control, maintenance and laboratory systems through read-only access in segmented plant networks, buffering through outages and backfilling years of history without disturbing plant systems.Key Skills:We do not expect in-depth experience with every tool/technology mentioned below. But we expect deep, production-grade expertise in at least one modern data platform and one leading CSP tech stack, hands-on experience in multiple others in same category, and the ability to learn a new platform and technology quickly.Production data engineering: Proven, hands-on design, build and operation of production data pipelines and platforms end to end.Platform depth: Deep, production-grade expertise in at least one modern data platform (cloud and on-premises) and practical experience with at least one other.Programming & processing: Expert SQL, strong Python and Apache Spark (PySpark / Spark SQL).Batch, streaming & CDC: Proven delivery of batch and real-time pipelines using Kafka or a cloud equivalent, with change data capture.Semi-structured & unstructured data: JSON, XML, documents and logs, not only relational data.Data modelling for engineering: Dimensional models, Data Vault and lakehouse layer design, and turning logical models into efficient physical designs (partitioning, file formats, indexing).Security, governance & DataOps: Access control, encryption, masking, lineage, cataloguing and automated quality checks; Git, CI/CD, infrastructure as code and pipeline monitoring.Technical evaluation: Running proofs of concept and writing clear, evidence-based recommendations.

Requirements

Bachelor's or Master's degree in Computer Science, Software Engineering, Information Technology, Engineering or a related discipline.7-10 years in data engineering, data integration or data warehousing, including 3+ years as a technical lead or lead engineer.Has led the technical build of at least one data platform or major integration program that reached production. Multi-industry experience is strongly preferred; industrial or OT data experience is a plus.Data engineering certifications on a major platform (e.g., Databricks, Microsoft Fabric, Snowflake, AWS or Google Cloud) are an advantage.