Senior Data Engineer (LATAM based)
Arionkoder · Greater Rio de Janeiro
Apply & track with Apply EdgeArionkoder is an AI consulting firm helping companies build, embed, and own AI systems that drive real business outcomes. We combine Product Development, Artificial Intelligence, and Team Augmentation to craft digital products and solutions that create lasting impact — guided by a human-centric approach and a commitment to pushing the boundaries of what technology can do.Arionkoder is an AI consulting firm helping companies build, embed, and own AI systems that drive real business outcomes. We combine Product Development, Artificial Intelligence, and Team Augmentation to craft digital products and solutions that create lasting impact, guided by a human-centric approach and a commitment to pushing the boundaries of what technology can do.We’re hiring a Senior Data Engineer to build the structured core of a governed data foundation on Databricks, used by AI agents, applications, and business teams. You own everything that has to be exact: business entities reconciled across systems that use different identifiers, facts with their source and validity period, business definitions turned into certified tables, access control, and the APIs and MCP tools that serve this data with the caller’s permissions.Conflicting definitions and identifiers are business problems as much as joins. The right person has spent years on Databricks making data correct and consistent, has enforced access control in production, and knows where platform controls stop and application-level enforcement has to start. You work autonomously with business stakeholders and ask the uncomfortable questions about definitions and permissions early.You’ll work in a small team (solution architect, AI engineer, full-stack engineer, data quality lead, product and project managers) alongside the client’s data platform team. You follow their conventions and hand over what you build. The AI engineer owns the parts that need a model: knowledge from documents, retrieval, agent behavior, and evals. You give them the trusted data those parts depend on. AI tools speed up your modeling, SQL, testing, and documentation every day, and you know exactly when to trust the output and when not to.Required Qualifications6+ years of professional data engineering, building production-grade pipelines and data models3+ years hands-on with Databricks in production, including Unity CatalogExpert SQL and Python: writes, debugs, optimizes, and refactors pipelines unaided. Spark/PySpark a plusStrong data modeling, including history-aware techniques (SCD Type 2, effective dating, temporal validity), with provenance and lineage designed in from the startData quality and testing discipline: tests and expectations in pipelines (dbt tests, Great Expectations, or similar) and CI/CD for dataReconciling data across systems: entity resolution or master dataIncremental and change-driven processing (CDC, streaming, or incremental frameworks) against freshness targetsPractical data governance: has implemented access control in production (row-level security, column masking, or role/attribute-based access) and knows where platform-level controls stop and application-level enforcement has to startHas built APIs or services that serve dataSenior judgment: works autonomously, challenges unclear definitions, prioritizes by risk, owns outcomes without micromanagementDaily use of AI coding tools (Claude, Cursor, or similar), with the judgment to know when to trust the outputC1 English; comfortable working fully remote and directly with a client teamPreferrableExposing data to applications through APIs or MCPsMulti-tenant data isolation for customer-facing products, with security and negative testingFinance or CRM data (NetSuite, Salesforce)Consolidating data across acquired companies or business unitsIngesting and governing unstructured content (documents, wikis, contracts) alongside structured dataTurning business definitions into a shared semantic or metrics layer with business or finance stakeholdersFinOps / cost attribution on data platformsWhat This Person Will Own And Do Day-to-dayIngestion and source strategyBuild and maintain ingestion and ETL pipelines from business systems (ERP, CRM, product databases), operational tools, and document repositories. Build data lakes in Databricks that software, people, and AI agents can all query. Decide per source whether to ingest, index and link, or query live, based on volatility, volume, and which system is the record. Work with raw data in its source-system shape without waiting on upstream cleanup.Entities and factsImplement the entity model defined with the solution architect and resolve identity across systems, reusing existing matching work and sending ambiguous matches to the owner who decides. Model facts with source, validity period, validation status, and documented exceptions. Turn business definitions validated by their owners into certified tables, recording who signed off. Implement authority rules per data item and keep unresolved conflicts visible. Deliver finance numbers that are deterministic, reproducible, and traceable to source records.Access and isolationImplement access policies in Databricks and carry them into derived data. Enforce isolation at the database, tool, and application layers for consumers who aren’t Databricks users, such as external customers. Build isolation tests at the data and API layer, including similarly named customers, and run them as a release gate.Structured sources and freshnessIntegrate the structured sources each increment needs (CRM, ERP, payment processors, product data, spreadsheets), starting from what’s already in Databricks. Meet freshness targets and expose freshness with each fact. Track dependencies so facts and certified tables are updated or flagged when a source or definition changes. Own data quality checks, observability, and cost visibility.Data interfacesBuild the APIs, webhooks, and MCP tools that query and filter facts deterministically, carrying the caller’s identity and permissions on every request. Log every access with the sources it used.Collaboration and handoverGive the AI engineer the facts, entity IDs, and access rules that knowledge and agents depend on. Work with the full-stack engineer on the owners’ review interface and with the data quality lead on acceptance criteria. Be the technical voice on data feasibility in discovery and design sessions, and document and hand over your work to the client’s team.AI-augmented engineeringUse AI tools daily for SQL and Python, schema mapping, definition drafting, test generation, and documentation. Review every AI-assisted output with care: a force multiplier, not a shortcut.Perks💵 Monthly USD Service Fee🌴 20 business days of paid Agreed Time Off—Annual + 6 Local Holidays🐣Caregiver Time Off💻A dynamic remote work culture. You can work from anywhere!🚀An entrepreneurial environment.About UsIn 2010, our founders dreamed up a company that could positively impact the world. Since then, we've been in permanent evolution: from business consulting to innovation and design, and now to building amazing digital products powered by AI. We scaled globally, launched AI Labs and Reshape Studios, and claimed our space as an AI consulting firm — helping organizations move from pilots to production through accountable AI. Today, we're expanding our impact across industries, driving real AI adoption through accountability, governance, and value at scale. Dreamers first, builders always.