Data Engineer - Global Hedge Fund - 300k+

This is a data engineering role inside a global hedge fund where the data platform is not a support function, it is the thing the investment process runs on. The role owns pipelines that feed research, risk, and portfolio construction, which means the engineer sits in constant contact with the people actually making decisions with the output. Databricks and Spark are the core of the stack, and the work spans ingestion from vendors and exchanges through to the curated datasets researchers query every morning. If a pipeline fails silently and a dataset lands wrong, a portfolio manager makes a call on bad numbers before anyone notices. That is the standard the role is held to.What You'll DoDesign, build, and maintain large scale batch and streaming pipelines in Spark on DatabricksOwn the full lifecycle of datasets, from vendor onboarding and validation through to the models and tables researchers consumeWork directly with researchers, portfolio managers, and risk to translate vague data questions into concrete engineering workBuild the monitoring, testing, and data quality controls that catch problems before consumers doImprove performance and cost of existing Spark workloads, and make decisions about how the platform evolvesDocument and explain your work to technical and non technical audiencesMust-haves2 to 10 years of data engineering experienceStrong production experience with Databricks and Spark, including performance tuningStrong Python and SQLDemonstrated ability to communicate clearly with non engineers, defend technical decisions in plain language, and manage stakeholders directly. This is non-negotiable for the role, the engineer who cannot hold a conversation with a portfolio manager will not succeed hereExperience with cloud data infrastructure (AWS, Azure, or GCP)Nice-to-havesExposure to financial or market data (tick data, reference data, corporate actions)Delta Lake, Unity Catalog, or lakehouse architecture experienceOrchestration tooling (Airflow, Dagster, Databricks Workflows)Infrastructure as code and CI/CD for data pipelinesExperience in a buy side or trading environmentWhy This RoleThe data platform here has direct, traceable impact on investment performance, and the engineers who build it are known by name to the people using it. The team is small enough that individual decisions shape the architecture rather than disappearing into a backlog, and the pace reflects a business where data problems are urgent by default. For a data engineer who is tired of building pipelines for people they never meet, and who wants their technical judgment to be visible to the desk, this is that.Who should applyStrong data engineers from top technology companies are actively encouraged to apply. No finance background required, but genuine curiosity about markets is expected.