Engineering Manager – Data Engineering

About The Job :This is an opportunity in which you will be responsible for overall implementation and delivery of projects and programmes, bringing new capabilities to the market, or implementing large scale change to existing products, covering the full lifecycle. You will oversee/manage the technicaldelivery management for a business area. You will supervise and coordinate the development teams (both internal employees and external partners) during their implementation. You will work closely with the businesses to understand their business objectives, product strategies and roadmaps and requirements so they can be continually translated into priorThis is an opportunity in which you will be responsible for overall implementation and delivery of projects and programmes, bringing new capabilities to the market, or implementing large scale change to existing products, covering the full lifecycle. You will oversee/manage the technicaldelivery management for a business area. You will supervise and coordinate the development teams (both internal employees and external partners) during their implementation. You will work closely with the businesses to understand their business objectives, product strategies and roadmaps and requirements so they can be continually translated into prioritised technology deliverables. Web development, with experience in JavaScript frameworksCloud Integrations, SecurityECS/EKS, API Gateways, Route53/F5, VPC/ELB, REST / SOAP frameworksAirflow, Glue, Lambdas, Redshift – with Node/Python.Liaise with business/data analysts and business stakeholders to review new requirements and enhancements and help identify the best technical solution for the data ingestion/distribution tasksDesign data extraction and transformation logic based on prevailing ETL/ELT technologies, languages and open-source frameworks (e.g., Python, Airflow, AWS Data Wrangler, AWS Sage Maker, AWS EMR, AWS Glue etc.)Supervise, coach and monitor the development teams and vendors in the implementation of designed data pipelines and transformationsEnsure all the data extraction and transformation jobs are successfully scheduled and executed, to guarantee data timeliness and qualityDesign, build and run data quality metrics for early identification of anomalies and erroneous data pipeline executionsReport progress, risks and issues to the line manager and other relevant stakeholders (e.g., business users, senior management team, leadership team etc)Evaluate new trends and technologies in the market and prepare suggestions andrecommendations to be presented to the line manager and other members of the senior management teamDrive a culture of collaborative reviews of designs, code and test plans.Ability to arrive at actionable insights leveraging the pro-active or pre-emptive measures and algorithms around the data driven decision.What we’re looking for:Excellent understanding of enterprise data warehousing principles and ETL/ELT solutionsFlexible and able to adjust to changing priorities in a fast-paced environment.Experience working on or with API's for data delivery/ingestion, e.g., REST APIs, GraphQL, etc.Experience working on or with batch-based data delivery/ingestion paradigms, e.g., csv files, ODBC connections, etc.Experience in Code management tools (GitHub) Build and Deployment tool (TravisCI).Excellent knowledge and working experience on data management technologies and frameworks, preferably on Python, Airflow, AWS API Gateway, SNS, SQS, LambdaGood understanding of Lambda and Kappa Architecture patternsAbility to work with large and complex data sets, data lakes and data warehousesSQL query skills. Hands-on experience with AWS Redshift, PostgreSQL and other database technologies like MySQL, Microsoft SQL Server, etc. and ability to write complex queries to extract/inject data into those databasesProven ability to manage a team of developers and to work in an objective oriented environmentExperience with Agile methodology including WIP management, backlog grooming/prioritization, reporting and dashboards.Key Skills:-Strong hands-on experience with PythonApache Airflow or similar workflow orchestration toolsAWS Data Services – Glue, Lambda, Redshift, EMRETL/ELT, Data Lakes & Data WarehousingREST/SOAP APIs and API integrationsPostgreSQL, MySQL, SQL Server & strong SQLAWS API Gateway, SNS, SQSECS/EKS, VPC, ELB, Route53/F5JavaScript frameworks / Web DevelopmentGitHub, CI/CD and Agile methodologiesEngineering team leadership and stakeholder management