Data Developer

Position DetailsJob Title: Data Developer (Databricks)Location: Abu Dhabi, UAEWork Model: Onsite (5 Days Per Week)Start Date: ASAPEnd Date: 31 OctoberPosition OverviewThe Data Developer Consultant will support the final phase of the organisation's Data Platform Modernisation Programme.This role is focused on delivering the remaining technical activities required to:Complete the migration from legacy technologies.Standardise engineering practices.Prepare the platform for long-term operational support.Ensure full operational readiness before transition into business-as-usual (BAU) support.The engagement is intended to provide experienced technical expertise to execute the "last mile" migration activities, address technical debt, complete platform remediation work, and finalise platform stabilisation efforts.Key Responsibilities The successful consultant will be responsible for the following:Data Platform ModernizationRefactor legacy data models and workflows to align with current engineering standards and best practices.Migrate data ingestion processes from legacy implementations to modern Databricks-based solutions.Support the decommissioning of legacy catalogues and platform components.Complete migration and remediation activities required to finalise the organisation's data platform transformation.Databricks EngineeringReview and optimise Databricks cluster utilisation.Implement cluster policies and performance optimisation improvements.Design, develop, and maintain scalable data pipelines.Develop and maintain data ingestion frameworks.Support the migration of ingestion processes to approved Databricks patterns and frameworks.Development & Quality AssuranceParticipate in testing, validation, deployment activities, and code reviews.Ensure all deliverables meet defined requirements, standards, and acceptance criteria.Perform testing, validation, and quality assurance activities before delivery.Contribute to continuous improvement initiatives.Documentation & Knowledge TransferDocument technical processes, standards, and operational workflows.Develop and maintain the required business context layer to support operational readiness.Contribute to knowledge transfer and support future operational teams.Team CollaborationSupport existing team members with:New data ingestion pipelinesData modelsPlatform enhancementsWork closely with the Data Engineering team to ensure alignment with engineering standards and governance frameworks.Support activities required to transition the platform from implementation into steady-state operations.Mandatory ExperienceRequired Skills & ExperienceDatabricks (Minimum 5 Years Required)Candidates must have a minimum of 5 years' hands-on Databricks development experience, including the following:Delta Live Tables (DLT) / Spark Declarative PipelinesAuto LoaderDatabricks Asset Bundles (DABs)Databricks Cluster OptimizationCluster Policy ManagementCloud EnvironmentMinimum 5 years of Databricks development experience on AWS environments.Strong understanding of cloud-based data engineering architectures.Additional Technical SkillsAzure Data Factory (ADF)1 to 3 years of experience.Source ControlStrong Git experience (mandatory).Strong understanding of source control best practices.Database & Programming SkillsStrong SQL development skills.Experience in SQL optimisation and performance tuning.Strong Python programming skills.Testing & DevOpsExperience with testing and validation practices.Experience with CI/CD deployment processes.Familiarity with DevOps methodologies and engineering practices.Working ModelThe consultant will work within the existing data engineering organisation under the team's established Agile delivery framework.Delivery StructureScoped work packages will be assigned to the consultant.Each consultant will be assigned to and managed by an existing member of the Data Engineering team.Consultants will operate within established governance, standards, and delivery processes.Task prioritisation and workload management will be coordinated through the assigned team lead.Agile ParticipationConsultants Will Actively Participate InSprint PlanningBacklog RefinementDaily Stand-UpsSprint ReviewsSprint RetrospectivesOther Agile delivery ceremonies as required