Freelance Manager Data Engineering - Azure Databricks

Apply now Refer a friend

Job Description

Freelance Manager Data Engineering

Note this is a contract role that is FT 40 hours a day, onsite in Houstion 3 days a week.

We are seeking a skilled Data Engineer with extensive, hands-on expertise in Azure Databricks to design, develop, and manage our enterprise data platform. In this architecture, all core transformations will be built directly within Databricks using Databricks Workflows and Jobs. The ideal candidate must have deep experience with Unity Catalog, DevOps (CI/CD), and the ability to drive Synapse migrations into code-driven Databricks environments.

Key Responsibilities
• Core Databricks Engineering: Design, build, and optimize end-to-end data pipelines entirely within Azure Databricks and Delta Lake using PySpark, SQL, or Scala.
• Orchestration & Automation: Build, schedule, and maintain complex production pipelines utilizing Databricks Workflows and Jobs as the primary orchestration engine.
• Data Legacy Migration: Lead and execute the migration of legacy data architectures into optimized Databricks solutions.
• Pipeline Conversion: Translate and refactor legacy business logic (such as converting historical Alteryx workflows) into clean, scalable Databricks notebook code.
• Data Governance: Implement and manage data access, security, schemas, and lineage across the platform using Databricks Unity Catalog.
• CI/CD & DevOps Automation: Develop, maintain, and automate Databricks workspace deployments and jobs using YAML and Classic CI/CD pipelines in Azure DevOps.

Certifications (Highly Prioritized)
• Active Professional Certifications are strongly preferred, such as:
o Databricks Certified Data Engineer Associate / Professional
o Microsoft Certified: Azure Data Engineer Associate (DP-203)
o Microsoft Azure Fundamentals (AZ-900) or equivalent foundational Azure knowledge is required at a minimum.

Preferred / Nice-to-Have Skills (Optional)
• Alteryx Conversion: Familiarity with Alteryx workflows is a major plus, specifically to assist in reverse-engineering and converting legacy visual workflows into PySpark/SQL code. (Note: Candidates do not need to be Alteryx developers, but must be comfortable reading/migrating the ETL logic).

Additional Information

As part of our dedication to an inclusive and diverse workforce, Publicis Sapient is committed to Equal Employment Opportunity without regard for race, color, national origin, ethnicity, gender, protected veteran status, disability, sexual orientation, gender identity, or religion. We are also committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, you may contact us at hiring@publicis.sapient.com

Company Description

Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting, and customer experience with agile engineering and problem-solving creativity. United by our core values and our purpose of helping people thrive in the brave pursuit of next, our 20,000+ people in 53 offices around the world combine experience across technology, data sciences, consulting, and customer obsession to accelerate our clients’ businesses through designing the products and services their customers truly value.